3 ms·

> When using GPT-2 vs GPT-3 vs GPT-4, a human can easily tell that each is leaps and bounds "better" than its predecessor, with "deeper" understanding of the in

> When using GPT-2 vs GPT-3 vs GPT-4, a human can easily tell that each is leaps and bounds "better" than its predecessor, with "deeper" understanding of the input and "more human-like" responses and reasoning. There is a strong impression that those models aren't just progressing along a scale, but changing qualitatively. I agree with you about all of this. > I simply doubt that any of the proposed metrics captures this intuitively observable quality. Furthermore, I claim that it is this quality that actually matters. I think there are important qualitative observations about LLMs that create understanding and guide future research, but I think it's bad science/engineering to rely on some indescribable quality of "goodness" when it comes to developing new models. Metrics may not be perfect, but they are all we have. You could train a model, send it to a bunch of human crowdworkers, and ask them to rate it on how good it is (Anthropic does this!), but the result of that is... another metric. That said, yeah, I think we definitely could use better metrics. OpenAI agrees, which is why they're pushing their Evals project super hard. The paper we're commenting on agrees, which is why they're proposing alternatives to these step-function metrics. > We don't need a language model in order to multiply two numbers. Any clearly defined and algorithmically solvable (and thus readily quantifiable) task is trivial for regular software. A language model multiplying two numbers is the whole point of emergence. Transformers were designed to translate between different natural languages and trained on completing sentences with some words masked out. As we scaled them bigger and bigger, we found that the same model architecture was suddenly capable of more than completing sentences: it was answering questions about high school geography (MMLU), writing computer programs, and - yes - doing basic arithmetic. New capabilities were emerging with scale. This means that a "language model," at sufficient scale, is more than a language model. This is the closest thing we have right now to generalized AI, where one model can complete a variety of disparate tasks. The arithmetic thing is exciting and unexpected because of what it represents, and understanding it should be a priority, even if we have better non-ML ways of doing that particular task.

p-e-w 3y ago
> Metrics may not be perfect, but they are all we have. You could train a model, send it to a bunch of human crowdworkers, and ask them to rate it on how good it is (Anthropic does this!), but the result of that is... another metric. A metric that is an aggregate of human intuition is not the same as the usual metrics though. It's just a semi-formalized way to capture those intuitive observations, rather than trying to replace them with much-simpler piecemeal mechanical evaluations. > but I think it's bad science/engineering to rely on some indescribable quality of "goodness" when it comes to developing new models. IMO, bad science is what is currently happening across the entire field. Astonishing high-level behavior is being observed from models, but the tools to analyze it don't exist, so instead, people are pushing out papers at a record pace that analyze every low-level property imaginable, as if such analysis would eventually yield high-level insights. It's okay to not know. I wish every paper dealing with LLMs would start and end with the sentence "Overall, we have no idea what is happening." Instead, we get papers that add a few numbers and then wax philosophical about how LLMs supposedly do things (I'm exaggerating here, but the gist is accurate). Not a day goes by without a new article claiming that LLMs have reached their limit or similar, while we have no clue how they even work! This is really bad science, and yes, I know that much of it is coming from non-experts, but I've seen lots and lots of experts contribute to this nonsense by making completely unjustified claims of a similar nature.
- dyang165 3y ago
  I think both of what you said is valid. I think it's important not to confuse "emergent" and "expected." Who said emergence shouldn't be expected? This paper I think more argues that nonlinear metrics creating sharp plots is "expected," but in my point of view it is still and emergent property. As you said, some metrics like accuracy are important and unreplaceable. As

p-e-w 3y ago

> Metrics may not be perfect, but they are all we have. You could train a model, send it to a bunch of human crowdworkers, and ask them to rate it on how good it is (Anthropic does this!), but the result of that is... another metric. A metric that is an aggregate of human intuition is not the same as the usual metrics though. It's just a semi-formalized way to capture those intuitive observations, rather than trying to replace them with much-simpler piecemeal mechanical evaluations. > but I think it's bad science/engineering to rely on some indescribable quality of "goodness" when it comes to developing new models. IMO, bad science is what is currently happening across the entire field. Astonishing high-level behavior is being observed from models, but the tools to analyze it don't exist, so instead, people are pushing out papers at a record pace that analyze every low-level property imaginable, as if such analysis would eventually yield high-level insights. It's okay to not know. I wish every paper dealing with LLMs would start and end with the sentence "Overall, we have no idea what is happening." Instead, we get papers that add a few numbers and then wax philosophical about how LLMs supposedly do things (I'm exaggerating here, but the gist is accurate). Not a day goes by without a new article claiming that LLMs have reached their limit or similar, while we have no clue how they even work! This is really bad science, and yes, I know that much of it is coming from non-experts, but I've seen lots and lots of experts contribute to this nonsense by making completely unjustified claims of a similar nature.

dyang165 3y ago
I think both of what you said is valid. I think it's important not to confuse "emergent" and "expected." Who said emergence shouldn't be expected? This paper I think more argues that nonlinear metrics creating sharp plots is "expected," but in my point of view it is still and emergent property. As you said, some metrics like accuracy are important and unreplaceable. As