3 ms·
AI benchmarks and leaderboards are complete nonsense though. Find something you like, use it, be ready to look again in a month or two.
by SV_BubbleTime 2y ago
AI benchmarks and leaderboards are complete nonsense though.
Find something you like, use it, be ready to look again in a month or two.
- falcor84 2y agoWith the accelerating progress, the "be ready to look again" is becoming a full time job that we need to be able to delegate in some way, and I haven't found anything better than benchmarks, leaderboards and reviews. EDIT: Typo
- siliconc0w 2y agoFWIW I've found the 'coding' 'category' of the leaderboard to be reasonably accurate. Claude was the best, o1-mini then was typically stronger, now the Gemini Exp 1206 is at the top. I find myself just paying a la carte via the API rather than paying the $20/mo so I can switch between the models.
- hombre_fatal 2y agopoe.com has a decent model where you buy credits and spend them talking to any LLM which makes it nice to swap between them even during the same conversation instead of paying for multiple subscriptions. Though gpt-4o could say "David Mayer" on poe.com but not on chat.openai.com which makes me wonder if they sometimes cheat and sneak in different models.