2 ms·
35B (35 billion) is the number of parameters this model has. Its a Mixture of Experts model (MoE) so A3B means that 3B parameters are Active at any moment.
by JLO64 6mo ago
35B (35 billion) is the number of parameters this model has. Its a Mixture of Experts model (MoE) so A3B means that 3B parameters are Active at any moment.
- zshn25 6mo ago~I see. What’s the 6?~ Nevermind, the other reply clears it