4 ms·

There is something about this that seems unscientific. What defines an "ability"? It sort of feels like the Guinness-Book type records for "most laps done aroun

by lsy 4y ago

There is something about this that seems unscientific. What defines an "ability"? It sort of feels like the Guinness-Book type records for "most laps done around the Washington monument with a duck on your head". Who is making up these ability tests and why? As an example, I clicked through to "english russian proverbs" which gives the model an english input and four russian proverbs, and asks the model to pick the closest one in meaning. There are 80 of these questions. My question is "why this task? why 80? why 4 options?" The authors give some nonspecific reasoning in the github page, but it really feels like people are out there creating random tasks as they think of them, running them all against models, and the ones that do better get added to a list to make headlines like "137 emergent abilities of language models", when the number of abilities models can do seems dominated by the number of "abilities" that have been invented, rather than any property of the model. I suspect that if we continued to use the same process to create twice as many "abilities", we would see something like twice as many that models can do. The fact that the models get better at these constructed tasks when they have more parameters also seems like a no-brainer. You are adding coefficients to a function. A function with 1 coefficient will not be able to describe much. A function with 2 describes more. If you are memorizing the internet, having more coefficients means you can memorize more stuff, and even stuff with only a few examples can be better-fit in the billions of parameters. With growing training set size, and companies hiding their data, we also run into the issue of nobody really knowing what is in the training set. Did someone clone the big-bench repository and it got sucked into a training set and that's why the model does really well on its tasks after a certain number of parameters?

zamfi 4y ago
I can't speak to this particular paper, but the field of NLP is full of these kinds of tasks, it's in some sense a historical accident of where LLMs originated. These tasks are all noisy, all pretty messy, but the interesting part here is not that performance improves on these constructed tasks -- NLP researchers have been improving performance on constructed tasks for decades, by customizing models and curating datasets. Rather, the interesting part is that performance improves despite no specific model or data shifts except for scale. It seems that you're arguing two things: first, that these tasks aren't that interesting -- on that we agree. But, second, that it's obvious that performance would improve on arbitrary tasks because of memorization and an unknown training set -- but this is easy to validate: make up a new constructed task, and see how it performs.
- lsy 4y ago
  I'm not saying everything is memorized, I'm guessing there are some tasks you can succeed on by having very similar things, or maybe even some kind of transfer knowledge. But how do you know that a "new" task you are making up doesn't have very similar examples in the (pre-existing) training set? Or that if you could isolate the data that plays into the "ability", that a much smaller model trained purely on that data wouldn't have a similar success rate? Or that your "ability" is a good proxy for the similar ability in people? In the extreme case, you can create a model that has the "ability" to perfectly solve 80 four-option questions in 20 bytes by using a truth table, but it wouldn't be a very general model. My complaint is that we have the veneer of empirical evidence but almost every aspect is loosey-goosey and full of potential confounders. We don't know the data sets, we don't really know why we want to test the tasks we test, we don't know if the tasks are similar or different to other tasks, the internal operations are illegible so we can't inspect its "thinking", and yet we are willing to list a specific number of tasks with detailed charts as though the concept of "emergence" is scientifically grounded rather than potentially being an artifact of something else.
- gcanyon 4y ago
  > the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that: With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly). With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes. As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.

by lsy 4y ago

zamfi 4y ago

I can't speak to this particular paper, but the field of NLP is full of these kinds of tasks, it's in some sense a historical accident of where LLMs originated. These tasks are all noisy, all pretty messy, but the interesting part here is not that performance improves on these constructed tasks -- NLP researchers have been improving performance on constructed tasks for decades, by customizing models and curating datasets. Rather, the interesting part is that performance improves despite no specific model or data shifts except for scale. It seems that you're arguing two things: first, that these tasks aren't that interesting -- on that we agree. But, second, that it's obvious that performance would improve on arbitrary tasks because of memorization and an unknown training set -- but this is easy to validate: make up a new constructed task, and see how it performs.

lsy 4y ago
I'm not saying everything is memorized, I'm guessing there are some tasks you can succeed on by having very similar things, or maybe even some kind of transfer knowledge. But how do you know that a "new" task you are making up doesn't have very similar examples in the (pre-existing) training set? Or that if you could isolate the data that plays into the "ability", that a much smaller model trained purely on that data wouldn't have a similar success rate? Or that your "ability" is a good proxy for the similar ability in people? In the extreme case, you can create a model that has the "ability" to perfectly solve 80 four-option questions in 20 bytes by using a truth table, but it wouldn't be a very general model. My complaint is that we have the veneer of empirical evidence but almost every aspect is loosey-goosey and full of potential confounders. We don't know the data sets, we don't really know why we want to test the tasks we test, we don't know if the tasks are similar or different to other tasks, the internal operations are illegible so we can't inspect its "thinking", and yet we are willing to list a specific number of tasks with detailed charts as though the concept of "emergence" is scientifically grounded rather than potentially being an artifact of something else.
gcanyon 4y ago
> the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that: With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly). With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes. As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.