2 ms·
Just don't listen to anecdata, and use objective metrics instead: https://chat.lmsys.org/?leaderboard https://chat.lmsys.org/?leaderboard
by lostmsu 2y ago
Just don't listen to anecdata, and use objective metrics instead: https://chat.lmsys.org/?leaderboard https://chat.lmsys.org/?leaderboard
- PhilippGille 2y agoYou might also want to look into other benchmarks: https://old.reddit.com/r/LocalLLaMA/comments/1ean2i6/the_final_straw_for_lmsys/ https://old.reddit.com/r/LocalLLaMA/comments/1ean2i6/the_fin...
- usaar333 2y agoGPT-4o being only 7 ELO above GPT-4o-mini suggests this is measuring something a lot different than "capabilities".
- Chilko 2y agoAnecdata seems quite valid for LLM comparison when trying to evaluate 'usefullness' for users. The lmsys chat leaderboard is literally just mass anecdata.
- lostmsu 2y agoYes, "mass anecdata" + blind collection is usually called "data".