> the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a funda

> the field of NLP is full of these kinds of tasks Isn't part of the issue that everyone is looking at these task results for signs of AGI, but there's a fundamental difference between human intelligence and artificial intelligence in that: With human intelligence we expect that it generalizes across just about any task -- if I create a new type of puzzle, the expectation is that if a human can think it up, a human can solve it. Cases where the puzzle fools otherwise-intelligent people are interesting because the people are fooled (unexpectedly). With artificial intelligence the exact opposite is generally true: we by default don't expect that competence at one task generalizes to competence at others, and cases are interesting where competence unexpectedly generalizes. As a result of that second part, we come up with more and more benchmarks to try models on, but we're still only looking at a small subset of all the weird, random tasks that a human is by default capable of either handling immediately, or picking up with some reasonable effort.