4 ms·
As far as I know, EVERY 1t+ LLM is a MoE. Switch-c-2048, Wi Dao 2.0, GLAM, Pangu-Σ, presumably GPT4. Am I missing any?
by COAGULOPATH 3y ago
As far as I know, EVERY 1t+ LLM is a MoE. Switch-c-2048, Wi Dao 2.0, GLAM, Pangu-Σ, presumably GPT4. Am I missing any?