3 ms·
Gemma's dense is bigger than its moe's total parameters. You could totally expect the moe to do terribly by comparison.
by zozbot234 7mo ago
Gemma's dense is bigger than its moe's total parameters. You could totally expect the moe to do terribly by comparison.