3 ms·
That doesn't explain how we know that GPT-4 is a sparse MoE model with X experts of Y size and using Z of them during inference.
by GaggiX 3y ago
That doesn't explain how we know that GPT-4 is a sparse MoE model with X experts of Y size and using Z of them during inference.
- Laaas 3y agoIIRC it was leaked/confirmed by accident or something like that