2 ms·
The models feel pretty snappy when interacting with them directly via ollama, not sure about the TPS However I've also ran into 2 things: 1) most models don't
by nico 1y ago
The models feel pretty snappy when interacting with them directly via ollama, not sure about the TPS
However I've also ran into 2 things: 1) most models don't support tools, sometimes it's hard to find a version of the model that correctly uses tools, 2) even with good TPS, since the agents are usually doing chain-of-thought and running multiple chained prompts, the experience feels slow - this is even true with Cursor using their models/apis