3 ms·
The Mistral Mixed Expert model has way fewer parameters active during inference and Groq has special purpose hardware (and probably less concurrent demand).
by tosh 3y ago
The Mistral Mixed Expert model has way fewer parameters active during inference and Groq has special purpose hardware (and probably less concurrent demand).
- kkielhofner 3y ago> probably less concurrent demand This is a significant understatement. ChatGPT has an estimated 100m monthly active users. Groq gets featured on HN from time to time but is otherwise almost completely unknown. According to their stats they have done something like 15m requests total since launch. ChatGPT likely does this in hours (or less).