3 ms·
The nice thing is that you can now use Llama 3 70B for very cheap ($0.9/M tokens) using FireworksAI, TogetherAI or similar providers.
by GaggiX 3y ago
The nice thing is that you can now use Llama 3 70B for very cheap ($0.9/M tokens) using FireworksAI, TogetherAI or similar providers.
- mritchie712 3y agohave you tried https://groq.com/ https://groq.com/? I get ~800 tokens per second.
- GaggiX 3y agoWith Llama 7 8B, Groq still has high API limits, so you cannot use it at scale for a project.