3 ms·
The standard is not what you can do with it, the standard is what is the free alternative. Local LLMs need to be able to beat the rate limits on all the free mo
by ActorNightly 1mo ago
The standard is not what you can do with it, the standard is what is the free alternative. Local LLMs need to be able to beat the rate limits on all the free models like Gemini to be useful. The only way to do that is to have high enough tok/sec, especially for agentic loops.