2 ms·
It’s a significant misunderstanding of Chomsky’s life’s work to think that ChatGPT would prove it wrong. Chomsky’s primary claim is a claim about how language a
by foldr 3y ago
It’s a significant misunderstanding of Chomsky’s life’s work to think that ChatGPT would prove it wrong. Chomsky’s primary claim is a claim about how language acquisition works in humans. He argues, for example, that certain locality constraints on linguistic dependencies are ‘built in’ and not learned inductively. Thus a human does not ‘learn’ that (i) is ambiguous and (ii) is not:
(i) How often did you tell John that he should take out the trash? [how often did you tell, or how often to take it out]
(ii) How often did you tell John why he should take out the trash? [only means how often did you tell]
Nothing that ChatGPT can do suggests that Chomsky was wrong about this kind of thing. It’s really more of a blow to a certain kind of work in AI that was partly inspired by Chomsky – but not something that he himself ever took much interest in.
Now it’s true that Chomsky appears to be in the camp that says ChatGPT doesn’t really understand anything. But the focus of his own work has never been on debunking AI, or making claims about the true nature of understanding, or anything of that ilk.
- Turing_Machine 3y ago> He argues, for example, that certain locality constraints on linguistic dependencies are ‘built in’ and not learned inductively. Checking in late here, but one of the pillars of Chomsky's argument is the so-called "poverty of the stimulus" -- basically, that human babies simply don't receive enough training data to acquire language as rapidly and correctly as they demonstrably do. Chomsky therefore concludes that there must be some kind of pre-existing "language module" in the brain to account for this. Now, not everyone accepted this idea even at the time, but surely the argument is much less plausible for an LLM which is likely exposed to more training data than even an adult human.
- foldr 3y ago>Now, not everyone accepted this idea even at the time, but surely the argument is much less plausible for an LLM which is likely exposed to more training data than even an adult human. Yes indeed. Of course this doesn't show that Chomsky was wrong about humans. In any case, I've seen no evidence that current LLMs successfully learn the kinds of constraints I was talking about.