3 ms·

I think Chomsky's opinions held back linguistics by decades, and we are only making progress because of the ascension of statistical methods. Chomsky didn't ju

I think Chomsky's opinions held back linguistics by decades, and we are only making progress because of the ascension of statistical methods. Chomsky didn't just argue that the statistical method lacked insight, but that humans can't really learn language statistically. Given that our artificial neural networks are much worse at learning than humans are, and yet they can simulate language much better than expected, this refutes Chomsky's arguments. Non-statistical approaches have basically zero successes, commercial or otherwise. ML at least gives us evidence of what aspects of language statistical models can explain and what they cannot, which is a hint for future research if linguists are just willing to take it.

mellosouls 6y ago
Chomsky didn't just argue that the statistical method lacked insight, but that humans can't really learn language statistically. Given that our artificial neural networks are much worse at learning than humans are, and yet they can simulate language much better than expected, this refutes Chomsky's arguments How does demonstrating what artificial neural networks can do refute the argument that humans can't learn something in the same way? All it does is refute the idea that artificial neural networks can't do that. I'm not commenting on how we learn or Chomsky's correctness or otherwise; just pointing out the unjustified leap from what AI tools can do to what actual intelligence does.
- QuesnayJr 6y ago
  Chomsky argued that it was literally impossible for a statistical model to learn language, so to the extent it's possible, it refutes Chomsky's argument. We won't know how humans learn language until we understand how humans learn language, but the fact that a crude general-purpose model like ANN works as well as it does is certainly evidence that humans can learn language statistically. We don't know for sure, but we rarely do.
  - mellosouls 6y ago
    Chomsky argued that it was literally impossible for a statistical model to learn language, so to the extent it's possible, it refutes Chomsky's argument. That's a different claim to the one I countered. I agree with your last point with the caution that there is no reason yet to think the hyped current AI tools have great relation to the way the human mind works, solely on the basis of the lack of progress towards AGI.
  - coolgeek 6y ago
    > but the fact that a crude general-purpose model like ANN works as well as it does is certainly evidence that humans can learn language statistically Humans learn language intuitively (subconsciously), as well as formally (through academic training). Humans understand language (to a greater or lesser extent in any particular case). Statistical NLP is not, in any human-translatable way, understanding. It is approximation and prediction, in a purely mathematical sense. Perhaps one of us misunderstands, or is imprecisely using, the phrase "learning language statistically", but I am unaware of any evidence that humans do so.
    - QuesnayJr 6y ago
      Chomsky's linguistic theories are not about understanding. Hence his famous example of "colorless green ideas sleep furiously", which is a meaningless sentence that we recognize as grammatical. Actually understanding language the way humans do would require AGI, I suspect.

mellosouls 6y ago

Chomsky didn't just argue that the statistical method lacked insight, but that humans can't really learn language statistically. Given that our artificial neural networks are much worse at learning than humans are, and yet they can simulate language much better than expected, this refutes Chomsky's arguments How does demonstrating what artificial neural networks can do refute the argument that humans can't learn something in the same way? All it does is refute the idea that artificial neural networks can't do that. I'm not commenting on how we learn or Chomsky's correctness or otherwise; just pointing out the unjustified leap from what AI tools can do to what actual intelligence does.

QuesnayJr 6y ago
Chomsky argued that it was literally impossible for a statistical model to learn language, so to the extent it's possible, it refutes Chomsky's argument. We won't know how humans learn language until we understand how humans learn language, but the fact that a crude general-purpose model like ANN works as well as it does is certainly evidence that humans can learn language statistically. We don't know for sure, but we rarely do.
- mellosouls 6y ago
  Chomsky argued that it was literally impossible for a statistical model to learn language, so to the extent it's possible, it refutes Chomsky's argument. That's a different claim to the one I countered. I agree with your last point with the caution that there is no reason yet to think the hyped current AI tools have great relation to the way the human mind works, solely on the basis of the lack of progress towards AGI.
- coolgeek 6y ago
  > but the fact that a crude general-purpose model like ANN works as well as it does is certainly evidence that humans can learn language statistically Humans learn language intuitively (subconsciously), as well as formally (through academic training). Humans understand language (to a greater or lesser extent in any particular case). Statistical NLP is not, in any human-translatable way, understanding. It is approximation and prediction, in a purely mathematical sense. Perhaps one of us misunderstands, or is imprecisely using, the phrase "learning language statistically", but I am unaware of any evidence that humans do so.
  - QuesnayJr 6y ago
    Chomsky's linguistic theories are not about understanding. Hence his famous example of "colorless green ideas sleep furiously", which is a meaningless sentence that we recognize as grammatical. Actually understanding language the way humans do would require AGI, I suspect.