5 ms·

That seems way too good to be true. What's the catch?

by ai-christianson 1y ago

Vuizur 1y ago
It is not very good at hard tasks, its ranking is much worse there.
- moneywoes 1y ago
  sorry, any examples of hard tasks
refulgentis 1y ago
I used to defend LMSys/Chatbot Arena a lot but threw in the towel after events of the past three months. I can give more details if you (or anyone else!) is interested. TL;DR: it is scoring only for "How authoritative did the answer look? How much flattering & emojis?"
- Jowsey 1y ago
  Is this not what Style Control (which IIRC they're making default soon) aims to mitigate?
  - refulgentis 1y ago
    I'm not 100% sure what their rationale is for it, the launch version of style control was a statistical model that penalized a few (4?) markdown shibboleths (lists, headers, ?). Not sure if they've shared more since. IMVHO it won't help, at all, even if they trained a perfect model that could accurately penalize it* The main problem is its one off responses, A/B tested. There's no way to connect it into all the stuff we're using to do work these days (i.e. tools / MCP servers), so at this point its sort of skipping the hard problems we'd want to see graded. (this situation is a example: whats more likely, style control is a small idea for an intractable problem, or Google has now released multiple free models better than Sonnet, including the latest, only 4B params? To my frustration, I have to go and bench these things myself because I have an AI-agnostic app I build, but I can confirm it is not the case that Gemma 3-not-n is better than Sonnet. 12B can half-consistently make file edits, which is a major step forward for local tbh) * I'm not sure how, "correctness" is a confounding metric here: we're probably much more likely to describe a formatted answer in negative terms if the answer is incorrect. In this case I am also setting aside how that could be done, just saying it as an illustration of no matter what, it's the wrong platform for a "how intelligent is this model?" signal, at this point, post-Eliza post-Turing, couple years out from ChatGPT 1.0
int_19h 1y ago
The catch is that "does as good as X" is pretty much never representative of real world performance when it comes to LLMs. In general, all those scores are mostly useful to filter out the models that are blatantly and obviously bad. But to determine whether the model is actually good at any specific thing that you need, you'll have to evaluate them yourself to find out.

That seems way too good to be true. What's the catch?

Vuizur 1y ago

It is not very good at hard tasks, its ranking is much worse there.

moneywoes 1y ago
sorry, any examples of hard tasks

refulgentis 1y ago

I used to defend LMSys/Chatbot Arena a lot but threw in the towel after events of the past three months. I can give more details if you (or anyone else!) is interested. TL;DR: it is scoring only for "How authoritative did the answer look? How much flattering & emojis?"

Jowsey 1y ago
Is this not what Style Control (which IIRC they're making default soon) aims to mitigate?
- refulgentis 1y ago
  I'm not 100% sure what their rationale is for it, the launch version of style control was a statistical model that penalized a few (4?) markdown shibboleths (lists, headers, ?). Not sure if they've shared more since. IMVHO it won't help, at all, even if they trained a perfect model that could accurately penalize it* The main problem is its one off responses, A/B tested. There's no way to connect it into all the stuff we're using to do work these days (i.e. tools / MCP servers), so at this point its sort of skipping the hard problems we'd want to see graded. (this situation is a example: whats more likely, style control is a small idea for an intractable problem, or Google has now released multiple free models better than Sonnet, including the latest, only 4B params? To my frustration, I have to go and bench these things myself because I have an AI-agnostic app I build, but I can confirm it is not the case that Gemma 3-not-n is better than Sonnet. 12B can half-consistently make file edits, which is a major step forward for local tbh) * I'm not sure how, "correctness" is a confounding metric here: we're probably much more likely to describe a formatted answer in negative terms if the answer is incorrect. In this case I am also setting aside how that could be done, just saying it as an illustration of no matter what, it's the wrong platform for a "how intelligent is this model?" signal, at this point, post-Eliza post-Turing, couple years out from ChatGPT 1.0

int_19h 1y ago

The catch is that "does as good as X" is pretty much never representative of real world performance when it comes to LLMs. In general, all those scores are mostly useful to filter out the models that are blatantly and obviously bad. But to determine whether the model is actually good at any specific thing that you need, you'll have to evaluate them yourself to find out.