4 ms·
Chomsky has already addressed Norvig's objections before he even made them as anyone how has actually listened to what Chomsky has said would know. > Norvig's
by foobarqux 4y ago
Chomsky has already addressed Norvig's objections before he even made them as anyone how has actually listened to what Chomsky has said would know.
> Norvig's reply: He agrees, but engineering success often facilitates scientific success.
There has been virtually no progress in understanding of the human language faculty from probabilistic models or LLMs.
> Norvig's reply: Science is both description and explanation; you can't have one without the other; in the history of science, the laborious accumulation of data is the usual mode of operation.
Probabilistic models don't describe anything in a way that leads to understanding (as the fact that no progress in understanding has been made shows).
> people actually generate and understand language in some rich statistical sense (maybe with statistical models several layers deep, like the modern AI models of speech recognition).
They do not; there are studies which Chomsky cites involving trying to learning "impossible" non-structural languages that give strong evidence that this is not the case.
> Norvig's reply: Certain advances in statistical learning methods provide reason to believe that such learning methods will be able to do the job.
It has nothing to do with the human language faculty.
> My conclusion is that 100% of these articles and awards are more about "accurately modeling the world" than they are about "providing insight," although they all have some theoretical insight component as well.
If you have a black box machine and you write a paper that says the black box reproduces some natural phenomenon with a billion of its knobs to turned to these specific settings you have wasted everyone's time.
> Norvig illustrates that rules about language do not always capture the right phenomena. (i before e)
The fundamental character of human language has nothing to do with spelling.
> [Norvig]: so any valid criticism of probabilistic models would have to be because they are too expressive, not because they are not expressive enough.
Yes Chomsky has explicitly said this. Any model that accepts virtually everything is a bad model.
I don't have time to go through the rest.
- morelisp 4y agoAs a connoisseur of the Chomsky-Foucault debate summarized as one intellectual giant completely missing the other's point, it's quite funny to see Chomsky in the opposite role here. The more operationalist you are, the more depressingly wrong you get - but also it seems the more likely you are to win debate club antics.
- voidhorse 4y agoUnfortunately I think the person who presents the simpler theory, even if it's less correct or less powerful, typically wins debates on theory. In some sense, theorists are working at different levels of abstraction across temporal dimensions. Foucault was concerned about deeper structures and a longer time horizon than Chomsky was at the time they debated, Chomsky is concerned about deeper issues and a longer time horizon than Norvig. This is why they wind up talking past each other.
- alecst 4y ago> They do not; there are studies which Chomsky cites involving trying to learning "impossible" non-structural languages that give strong evidence that this is not the case. I was looking for these studies. I found some similar stuff by Jennifer Culbertson, along these lines https://www.annualreviews.org/doi/pdf/10.1146/annurev-linguistics-011718-012329 https://www.annualreviews.org/doi/pdf/10.1146/annurev-lingui..., but didn't quite know what to Google. Can you point me to something?
- foobarqux 4y agoI believe Chomsky mentioned two studies in one of the 4 episodes in the "Closer to Truth" series, you'll have to search the transcript for the exact timestamp. The first is an fMRI study that shows that the brain doesn't engage the language processing centers when trying to understand a made-up non-structural language (i.e. a "statistical language") but does when trying to learn a made-up structural language. The second is about a man who had brain damage except in the language processing centers. A similar study showed that he could learn made-up structural languages but not "statistical" languages. Poverty of stimulus arguments might also be relevant. There might be an energy argument in his book "Why Only Us" as well.