3 ms·
Intuitively looks like models should be close enough, or sparse enough for merge to work. I wonder if MoE experts can be merged(?)
by zingelshuher 2y ago
Intuitively looks like models should be close enough, or sparse enough for merge to work. I wonder if MoE experts can be merged(?)