3 ms·
Probably naive questions: Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time? Does that make Gemini 1.5x better, or m
by supermatt 11mo ago
Probably naive questions:
Does that also mean that Gemini-3 (the top ranked model) loses to mistral 3 40% of the time?
Does that make Gemini 1.5x better, or mistral 2/3rd as good as Gemini, or can we not quantify the difference like that?
- esafak 11mo agoYes, of course.
- uejfiweun 11mo agoWow. If all the trillions only produces that small of a diff... that's shocking. That's the sort of knowledge that could pop the bubble.
- JustFinishedBSG 11mo agoI wouldn't trust LMArena results much. They measure user preference and users are highly skewed by style, tone etc. You can litteraly "improve" your model on LMArena by just adding a bunch of emojis.