4 ms·

I can't speak to this particular paper, but the field of NLP is full of these kinds of tasks, it's in some sense a historical accident of where LLMs originated.

by zamfi 4y ago

I can't speak to this particular paper, but the field of NLP is full of these kinds of tasks, it's in some sense a historical accident of where LLMs originated. These tasks are all noisy, all pretty messy, but the interesting part here is not that performance improves on these constructed tasks -- NLP researchers have been improving performance on constructed tasks for decades, by customizing models and curating datasets. Rather, the interesting part is that performance improves despite no specific model or data shifts except for scale. It seems that you're arguing two things: first, that these tasks aren't that interesting -- on that we agree. But, second, that it's obvious that performance would improve on arbitrary tasks because of memorization and an unknown training set -- but this is easy to validate: make up a new constructed task, and see how it performs.

lsy 4y ago
I'm not saying everything is memorized, I'm guessing there are some tasks you can succeed on by having very similar things, or maybe even some kind of transfer knowledge. But how do you know that a "new" task you are making up doesn't have very similar examples in the (pre-existing) training set? Or that if you could isolate the data that plays into the "ability", that a much smaller model trained purely on that data wouldn't have a similar success rate? Or that your "ability" is a good proxy for the similar ability in people? In the extreme case, you can create a model that has the "ability" to perfectly solve 80 four-option questions in 20 bytes by using a truth table, but it wouldn't be a very general model. My complaint is that we have the veneer of empirical evidence but almost every aspect is loosey-goosey and full of potential confounders. We don't know the data sets, we don't really know why we want to test the tasks we test, we don't know if the tasks are similar or different to other tasks, the internal operations are illegible so we can't inspect its "thinking", and yet we are willing to list a specific number of tasks with detailed charts as though the concept of "emergence" is scientifically grounded rather than potentially being an artifact of something else.
gcanyon 4y ago
> the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that: With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly). With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes. As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.

by zamfi 4y ago

lsy 4y ago

I'm not saying everything is memorized, I'm guessing there are some tasks you can succeed on by having very similar things, or maybe even some kind of transfer knowledge. But how do you know that a "new" task you are making up doesn't have very similar examples in the (pre-existing) training set? Or that if you could isolate the data that plays into the "ability", that a much smaller model trained purely on that data wouldn't have a similar success rate? Or that your "ability" is a good proxy for the similar ability in people? In the extreme case, you can create a model that has the "ability" to perfectly solve 80 four-option questions in 20 bytes by using a truth table, but it wouldn't be a very general model. My complaint is that we have the veneer of empirical evidence but almost every aspect is loosey-goosey and full of potential confounders. We don't know the data sets, we don't really know why we want to test the tasks we test, we don't know if the tasks are similar or different to other tasks, the internal operations are illegible so we can't inspect its "thinking", and yet we are willing to list a specific number of tasks with detailed charts as though the concept of "emergence" is scientifically grounded rather than potentially being an artifact of something else.

gcanyon 4y ago

> the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that: With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly). With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes. As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.