4 ms·

Yeah, Norvig has a rather famous rebuttal of Chomsky up on his blog from years ago, and everything that's happened since then has supported Norvig's position.

drdeca 5y ago
Is this the one you mean : https://norvig.com/chomsky.html https://norvig.com/chomsky.html ?
- kleiba 5y ago
  If it is then I wouldn't necessarily call it a rebuttal because Norvig broadly agrees with Chomsky on the points under discussion. And that is that statistical models are not useful scientific theories about language in themselves. Where they disagree is what consequences to draw from it: Chomsky discards studying language by statistical means altogether, but Norvig offers a more nuanced position: he argues that there might still be a merit in statistical approaches to language, but not as a result so much in itself but rather as a valid tool that can ultimately help derive a deeper understanding. I think this is a valuable distinction. Unfortunately, though, it is one that the research landscape seems to have somehow forgotten about over the last decade or so. More tangibly speaking using an example, while the capabilities of a language model like GPT-3 are amazing, Norvig's point is that science should not just stop there - it should ask the question "what does GPT-3 teach us about human language?"
  - foldr 5y ago
    >Chomsky discards studying language by statistical means altogether Chomsky nowhere does this. That is at best a straw man constructed by Norvig of Chomsky's position. If you just look at the transcript of the Chomksy interview in the Norvig piece, you can see Chomsky describing a case where statistical analysis is useful.
foldr 5y ago
This rebuttal has always bugged me because of the egregious error Norvig makes in his analysis of pro drop. The dropping of subjects in English matrix clauses (e.g. "Not gonna do it") is known to linguists as "diary drop", and is a totally different phenomenon from the dropping of subjects in so-called "null subject" or "pro drop" languages like Spanish. The most obvious difference is that diary drop is completely impossible in embedded clauses: I think I'm not gonna do it. [good in English] I think not gonna do it. [bad in English] (Yo) creo que no voy a hacerlo. [good in Spanish] To linguists it's immensely frustrating that someone who's considered by the compsci crowd to be a leader in the field of NLP can't even get some basic facts of linguistic analysis correct. "Linguists can argue over the interpretation of these facts for hours on end", Norvig says dismissively. But any competent syntactician could have explained to him in five minutes why his examples are irrelevant to the point he's trying to make. The rest of Norvig's post just consists in misunderstanding what Chomsky is saying, as far as I can see. Chomsky is saying that statistical analysis alone is not sufficient to achieve scientific progress. I don't think Norvig actually disagrees with Chomsky on this point, but he seems to think that Chomsky is saying that scientists should never use statistical methods. The real kicker here is that Norvig is an engineer, not a scientist. He's contributed very little to the scientific study of language, and yet is lecturing someone who's contributed vastly more on how it ought to be done. Of course, that doesn't necessarily mean that he's wrong, but it does grate. Not to mention that the Chomsy/O'Reilly comparison is little more than trolling – it hardly seems calculated to stimulate an intelligent response.

drdeca 5y ago

Is this the one you mean : https://norvig.com/chomsky.html https://norvig.com/chomsky.html ?

kleiba 5y ago
If it is then I wouldn't necessarily call it a rebuttal because Norvig broadly agrees with Chomsky on the points under discussion. And that is that statistical models are not useful scientific theories about language in themselves. Where they disagree is what consequences to draw from it: Chomsky discards studying language by statistical means altogether, but Norvig offers a more nuanced position: he argues that there might still be a merit in statistical approaches to language, but not as a result so much in itself but rather as a valid tool that can ultimately help derive a deeper understanding. I think this is a valuable distinction. Unfortunately, though, it is one that the research landscape seems to have somehow forgotten about over the last decade or so. More tangibly speaking using an example, while the capabilities of a language model like GPT-3 are amazing, Norvig's point is that science should not just stop there - it should ask the question "what does GPT-3 teach us about human language?"
- foldr 5y ago
  >Chomsky discards studying language by statistical means altogether Chomsky nowhere does this. That is at best a straw man constructed by Norvig of Chomsky's position. If you just look at the transcript of the Chomksy interview in the Norvig piece, you can see Chomsky describing a case where statistical analysis is useful.

foldr 5y ago

This rebuttal has always bugged me because of the egregious error Norvig makes in his analysis of pro drop. The dropping of subjects in English matrix clauses (e.g. "Not gonna do it") is known to linguists as "diary drop", and is a totally different phenomenon from the dropping of subjects in so-called "null subject" or "pro drop" languages like Spanish. The most obvious difference is that diary drop is completely impossible in embedded clauses: I think I'm not gonna do it. [good in English] I think not gonna do it. [bad in English] (Yo) creo que no voy a hacerlo. [good in Spanish] To linguists it's immensely frustrating that someone who's considered by the compsci crowd to be a leader in the field of NLP can't even get some basic facts of linguistic analysis correct. "Linguists can argue over the interpretation of these facts for hours on end", Norvig says dismissively. But any competent syntactician could have explained to him in five minutes why his examples are irrelevant to the point he's trying to make. The rest of Norvig's post just consists in misunderstanding what Chomsky is saying, as far as I can see. Chomsky is saying that statistical analysis alone is not sufficient to achieve scientific progress. I don't think Norvig actually disagrees with Chomsky on this point, but he seems to think that Chomsky is saying that scientists should never use statistical methods. The real kicker here is that Norvig is an engineer, not a scientist. He's contributed very little to the scientific study of language, and yet is lecturing someone who's contributed vastly more on how it ought to be done. Of course, that doesn't necessarily mean that he's wrong, but it does grate. Not to mention that the Chomsy/O'Reilly comparison is little more than trolling – it hardly seems calculated to stimulate an intelligent response.