3 ms·

have you tried https://groq.com/ https://groq.com/? I get ~800 tokens per second.

• GaggiX 2y ago
  With Llama 7 8B, Groq still has high API limits, so you cannot use it at scale for a project.