3 ms·
The way Norvig tells it, language lends itself perfectly well to smoothing and even Chomsky's favourite absurd phrase "colorless green ideas sleep furiously" is
by YeGoblynQueenne 10y ago
The way Norvig tells it, language lends itself perfectly well to smoothing and even Chomsky's favourite absurd phrase "colorless green ideas sleep furiously" is assigned a (very small but non-zero) probability in some statistical models of English.
Truth is you can't do much with statistical models of language without some sort of way to account for what's missing from your data, which is always most of language. On the other hand, anything you might do is never going to be enough when that's the case: that you're missing the majority of language from your training data.
- fauigerzigerk 10y agoSome NLP tasks do indeed lend themselves well to smoothing, but I was thinking of language understanding tasks like question answering where changing or misunderstanding a single word in an entire novel can easily make a correct answer completely incorrect. I agree with your second paragraph, and if I understand Chomsky correctly, that is part of why he argues in favor of a generative grammar. I can't say that I completely understand how such a grammar would be linked to semantics and experience though.
- YeGoblynQueenne 10y agoAh, OK, sorry for misunderstanding.
- mcguire 10y agoWas that a pun? Out of curiosity, how does Chomsky's generative model account for language understanding?
- foobarqux 10y agoIt demonstrates that human languages all have a certain structure, so it must be an innate faculty of human beings (as opposed to acquired). A neural network could not tell you that. It also suggests further avenues of inquiry (Why that particular structure?)
- mcguire 10y agoI think you misunderstood my question. Here's my understanding of Chomskyian linguistics: 1. You have a certain concept you wish to express. 2. You apply a generative grammar to the concept, producing a linguistic statement. 3. You express the statement in a linguistic performance. 4. I perceive the linguistic performance. 5. ¿I reverse the generative grammar to produce the concept? 6. I understand the concept. My understanding is that Chomsky is only interested in steps 2 and 5 (and that he is explicitly uninterested in 3 and 4). But how does step 5 work?
- foobarqux 10y agoI don't think much is understood about how linear externalizations of language are deserialized into symbolic structures, what those structures are, and how those are symbol structures are mapped into mental representations.
- YeGoblynQueenne 10y agoHah. No pun :)
- foobarqux 10y agoThe issue Chomsky raises isn't that statistical models aren't predictive enough, but that you don't understand they don't typically yield any understanding of the underlying system.
- mcguire 10y agoSimilarly (and I understand this argument went through physics long before quantum mechanics showed up and made the argument seem silly), the ideal gas law, say, is predictive, but doesn't yield any understanding of the underlying system.