4 ms·
I'm delighted to see that at least someone else here is cognizant of this. I don't think it's the scifi AI anchoring bias though - it's the "LLM leaderboard ar
by maeil 3y ago
I'm delighted to see that at least someone else here is cognizant of this.
I don't think it's the scifi AI anchoring bias though - it's the "LLM leaderboard arena user bias".
Reset all to the same Elo, put a group of people actually representative of global society in front of the arena for an hour, and you end up with a very different leaderboard, especially at the top end.
- refulgentis 3y agoThat's silly cope justifying a bad argument made initially in ignorance of what the actual metric was. "It's meaningless because we need the perfectly unbiased representative sample of raters doing rating right, instead of the biased raters doing it wrong that I'm currently imagining" isn't an appealing or honest argument.