2 ms·

The top comment to an LMSys leaderboard link on HN is always a variation of this song. And it's always not even wrong, in the Pauli sense of the phrase. OP, y

The top comment to an LMSys leaderboard link on HN is always a variation of this song. And it's always not even wrong, in the Pauli sense of the phrase. OP, your assignment, if you choose to accept it, is to look into how the LMSys leaderboard works and report back what its metric(s) are. [SPOILER] There's a really absurdly narrow argument you can make where all of its users are engineers, making engineer queries, and LLM makers are optimizing for it thus it's bad. But...it's humans asking queries then picking the better answer, blind. You can't narrowly optimize for that when making an LLM and it's hard to see how optimizing for "people think the answer is better" is the wrong metric here. It's just ELO. Might as well argue chess/checkers/pick your poison is bad because ELO optimizes for wins but actually talent is based on more than winning.

kromem 3y ago
The argument would be more that for the Chatbot arena there's an inherent sampling bias where users battling the chatbots are more likely to ask questions in line with what they would expect a chatbot to perform well at and less likely to prompt with things that would be rejected or outside the expected domains of expertise. If we wanted the chatbot arena to be more representative of a comprehensive picture of holistic knowledge and wisdom, we'd need to determine some way to classify the prompts and then normalize the scores against those classifications so there wasn't a weighting bias towards more common and expected behaviors. Alternatively, we might want a leaderboard that represents assessments of what users would expect a chatbot to perform well at relative to the frequency with which they expect it. But in that case, we shouldn't kid ourselves that the leaderboard is representing broad and comprehensive measures of performance outside of the targets against which we are optimizing models and effectively training model users in what types of prompts to ask for in expecting successful results.
- int_19h 3y ago
  > less likely to prompt with things that would be rejected or outside the expected domains of expertise. I don't see why. If anything, it's the opposite - people spend a lot of time coming up with contrived logical puzzles etc specifically so as to see which chatbots break.
  - kromem 3y ago
    Right - 'logical' puzzles. There's an anchoring bias from decades of sci-fi that most people don't even realize they've internalized around what 'AI' can and can't do. If you think about what information and information connections are modeled in social media data, there's quite a lot of things outside of "logical puzzles." The pretrained models likely picked up things like extensive modeling of ego, emotional response and contexts for generation, etc. But you'll be hard pressed to see those skills represented in what users ask models to produce, what they've been fine tuned around, or how they are being evaluated. Even though there's extensive value in those skills on the right applications.
- maeil 3y ago
  I'm delighted to see that at least someone else here is cognizant of this. I don't think it's the scifi AI anchoring bias though - it's the "LLM leaderboard arena user bias". Reset all to the same Elo, put a group of people actually representative of global society in front of the arena for an hour, and you end up with a very different leaderboard, especially at the top end.
  - refulgentis 3y ago
    That's silly cope justifying a bad argument made initially in ignorance of what the actual metric was. "It's meaningless because we need the perfectly unbiased representative sample of raters doing rating right, instead of the biased raters doing it wrong that I'm currently imagining" isn't an appealing or honest argument.

kromem 3y ago

The argument would be more that for the Chatbot arena there's an inherent sampling bias where users battling the chatbots are more likely to ask questions in line with what they would expect a chatbot to perform well at and less likely to prompt with things that would be rejected or outside the expected domains of expertise. If we wanted the chatbot arena to be more representative of a comprehensive picture of holistic knowledge and wisdom, we'd need to determine some way to classify the prompts and then normalize the scores against those classifications so there wasn't a weighting bias towards more common and expected behaviors. Alternatively, we might want a leaderboard that represents assessments of what users would expect a chatbot to perform well at relative to the frequency with which they expect it. But in that case, we shouldn't kid ourselves that the leaderboard is representing broad and comprehensive measures of performance outside of the targets against which we are optimizing models and effectively training model users in what types of prompts to ask for in expecting successful results.

int_19h 3y ago
> less likely to prompt with things that would be rejected or outside the expected domains of expertise. I don't see why. If anything, it's the opposite - people spend a lot of time coming up with contrived logical puzzles etc specifically so as to see which chatbots break.
- kromem 3y ago
  Right - 'logical' puzzles. There's an anchoring bias from decades of sci-fi that most people don't even realize they've internalized around what 'AI' can and can't do. If you think about what information and information connections are modeled in social media data, there's quite a lot of things outside of "logical puzzles." The pretrained models likely picked up things like extensive modeling of ego, emotional response and contexts for generation, etc. But you'll be hard pressed to see those skills represented in what users ask models to produce, what they've been fine tuned around, or how they are being evaluated. Even though there's extensive value in those skills on the right applications.
maeil 3y ago
I'm delighted to see that at least someone else here is cognizant of this. I don't think it's the scifi AI anchoring bias though - it's the "LLM leaderboard arena user bias". Reset all to the same Elo, put a group of people actually representative of global society in front of the arena for an hour, and you end up with a very different leaderboard, especially at the top end.
- refulgentis 3y ago
  That's silly cope justifying a bad argument made initially in ignorance of what the actual metric was. "It's meaningless because we need the perfectly unbiased representative sample of raters doing rating right, instead of the biased raters doing it wrong that I'm currently imagining" isn't an appealing or honest argument.