3 ms·
What do you use for evaluation? gemini-2.5-pro is at the top of MMLU and has been best for me but always looking for better.
by lukasb 11mo ago
What do you use for evaluation? gemini-2.5-pro is at the top of MMLU and has been best for me but always looking for better.
- vunderba 11mo agoRecently I've found myself getting the evaluation simultaneously from to OpenAI gpt-5, Gemini 2.5 Pro, and Qwen3 VL to give it a kind of "voting system". Purely anecdotal but I do find that Gemini is the most consistent of the three.
- lukasb 11mo agoInteresting, I'll give voting a shot, thanks.
- motbus3 11mo agoI am running similar experiment but so far, changing the seed of openai seems to give similar results. Which if that confirms, is concerning to me on how sensitive it could be
- dangoodmanUT 11mo agoI found the opposite. GPT-5 is better at judging along a true gradient of scores, while Gemini loves to pick 100%, 20%, 10%, 5%, or 0%. Like you never get a 87% score.