Groq is claiming 284 tokens/second on Llama 3.1 70b, so they’re in the same ballpark. https://groq.com/12-hours-later-groq-is-running-llama-3-instruct-8-70b-by

Groq is claiming 284 tokens/second on Llama 3.1 70b, so they’re in the same ballpark. https://groq.com/12-hours-later-groq-is-running-llama-3-instruct-8-70b-by-meta-ai-on-its-lpu-inference-enginge/ https://groq.com/12-hours-later-groq-is-running-llama-3-inst...