3 ms·
Look at what a company called Groq does. Their chip can generate 500 tokens/second from a model like Llama 2 70b. https://groq.com/ https://groq.com/
by elorant 3y ago
Look at what a company called Groq does. Their chip can generate 500 tokens/second from a model like Llama 2 70b.
https://groq.com/ https://groq.com/