6 ms·
and Groq does 485.08 T/s on mixtral 8x7B-32k I am not sure local models have any future other than POC/research. Depends on the cost of course.
by attentive 3y ago
and Groq does 485.08 T/s on mixtral 8x7B-32k
I am not sure local models have any future other than POC/research. Depends on the cost of course.
- tome 3y ago(Groqster here) For anyone who wants to try it, you can go to https://chat.groq.com/ https://chat.groq.com/ and choose Mixtral from the drop-down menu. Also, feel free to ask me any questions about Groq hardware or service.