2 ms·
See my poorly educated answer above. I don’t think that’s how MoE actually works. A new mixture of experts is chosen for every new context.
by ttul 3y ago
See my poorly educated answer above. I don’t think that’s how MoE actually works. A new mixture of experts is chosen for every new context.