4 ms·
We're looking at a MoE with 50B active params, each inference pass only requires the compute of a 50B dense model.
by dannyw 3mo ago
We're looking at a MoE with 50B active params, each inference pass only requires the compute of a 50B dense model.