3 ms·
If this is the hierarchy, why does 4o score so much higher than o1 on LLM Arena? Worrisome for OpenAI that Gemini's mini/flash reasoning model outscores both o
by gundmc 2y ago
If this is the hierarchy, why does 4o score so much higher than o1 on LLM Arena?
Worrisome for OpenAI that Gemini's mini/flash reasoning model outscores both o1 and 4o handily.
- crazysim 2y agoIs it possible people are voting for speed of responsiveness too?
- kgeist 2y agoI suspect people on LLM Arena don't ask complex questions too often, and reasoning models seem to perform worse than simple models when the goal is just casual conversation or retrieving embedded knowledge. Reasoning models probably 'overthink' in such cases. And slower, too.
- imtringued 2y agoThe LLM Arena deletes your prompt when you restart so what's the point in trying to write a complicated prompt and testing an exhaustive number of pairs? It's easy to pin this on the users, but that website is hostile to putting in any effort. This is something I've noticed a lot actually. A lot of AI projects just give you an input field and call it a day. Expecting the user to do the heavy lifting.
- energy123 2y agoo1 on LLM Arena often times out (network error) while thinking. But they still allow you to vote and they make it seem as if your vote is registered.