4 ms·
The benchmark also doesn't include speed. You almost think something has gone wrong when using it because it returns full responses so incredibly fast.
by WarmWash 2mo ago
The benchmark also doesn't include speed. You almost think something has gone wrong when using it because it returns full responses so incredibly fast.
- scrlk 2mo agoNot just speed, also reliability. IME, Gemini's speed and quality doesn't degrade badly during weekday working hours compared to OAI, and especially Anthropic.
- ford 2mo agoI've had Gemini model API use degrade the most out of OAI/Anthropic/Google (often "over capacity" vs true failures) Not sure on consumer/product use though
- scrlk 2mo agoThat's interesting to hear. I should have added that I use Gemini through Google AI Studio as my general chat model, which probably explains our wildly different experiences.
- sotix 2mo agoThis one uses that as a priority weight: https://winstonrc.github.io/ai-coding-agents-leaderboard/ https://winstonrc.github.io/ai-coding-agents-leaderboard/