3 ms·

I think Norvig acknowledges the point you are making here, namely that the statistical approach does not explain the cognitive systems behind language. However

by lkozma 15y ago

I think Norvig acknowledges the point you are making here, namely that the statistical approach does not explain the cognitive systems behind language. However (if I understand correctly) he implies that those systems might be too complex to be adequately explained, let alone emulated and we can achieve more by observing them as black boxes, analyzing their outputs, i.e. language as it is used. "if a computer could - like humans - learn to produce infinite, novel, contextual, and meaningful grammatical utterances" To perfectly achieve this goal, you might have to simulate 4 billion years of evolution under the same conditions as it happened on Earth, and a few thousand years of cultural evolution as it led to our languages and our cultural context. Language is incredibly complex and changing, many of its details might be incidental, i.e. results of random events, so it seems unreasonable to pretend that we can deduce it all from some elegant first principles. At least that is my reading of Norvig's argument.

brockf 15y ago
Since when has science been reasonable? :) In situations like this, I tend to speak in theoretical absolutes. A computer that "could - like humans - learn to produce infinite, novel, contextual, and meaningful grammatical utterances" isn't even on the timeline right now, but it's the theoretical goal in showing that we understand language acquisition (ontogenetic development), evolution (phylogenetic development), and production. Just because that goals seems unattainable doesn't, to me, mean that we need to aim any lower. Now, this is premised on my belief that mimicking phenomena with statistical learning is not as intellectually satisfying as understanding the underlying cognitive systems, but that's not believed by everyone.
- lkozma 15y ago
  I agree that learning to produce and interpret varied utterances is a worthy goal, but the fact is that (far as it still is today) lowly statistical methods have gotten us closer to this goal than the other, chomskian approach. It could be a situation where aiming lower lets you shoot higher.
  - anghyflawn 15y ago
    This is a fundamental misunderstanding of what modern generative linguistics is all about (to be fair, it is extremely widespread). The aim of this branch of science is expressly not to "learn to produce and interpret varied utterances" (called E-language in the jargon), but to understand the cognitive processes behind the production and interpretation of utterances (called I-language). Now you may agree or disagree with the methods and assumptions used in the pursuit of this goal, but it is patently unfair to accuse the field of failing to do something it never set out to do.
pealco 15y ago
> I think Norvig acknowledges the point you are making here, namely that the statistical approach does not explain the cognitive systems behind language. If that is the case, then the argument that Norvig is making is irrelevant to the argument Chomsky is making. Chomsky simply makes the point that statistical accounts lack explanatory adequacy. As someone who has worked closely with many of his students and who has received extensive training on his scientific program, I can say with confidence that Chomsky would have no objection whatsoever about the usefulness of statistical approaches to linguistic engineering problems. The results speak for themselves. He would go on to say, however, that how well a statistical approach solves a linguistic engineering problem is irrelevant to the question of how humans do what they do. The answer to the question may well be statistically grounded. That is a valid hypothesis and a logical possibility which should be taken seriously. However, it is incumbent on the proponents of such an answer to provide evidence that it is what humans are doing. Here are some examples of the kinds of evidence necessary: * evidence that humans are capable of performing the kinds of computations that the statistical approach requires, * evidence that the statistical approach works with the relatively limited amount of data that a human receives, * evidence that the statistical approach fails in ways that humans fail How well a statistical approach succeeds at an engineering task is not an item on this list, simply, again, because engineering tasks are irrelevant to what humans actually do. Let me specifically say that statistical approaches are not, from the start, ruled out as potential candidates for the algorithms underlying human language. It's just that a case has to be made for them using the right kind of evidence. Finally, I'll reiterate what others have pointed out: from a scientific perspective, that something is hard to explain doesn't mean that we shouldn't try. And, those that have given up (as you suggest Norvig has) shouldn't fault those who haven't for calling them out on it.

by lkozma 15y ago

brockf 15y ago

Since when has science been reasonable? :) In situations like this, I tend to speak in theoretical absolutes. A computer that "could - like humans - learn to produce infinite, novel, contextual, and meaningful grammatical utterances" isn't even on the timeline right now, but it's the theoretical goal in showing that we understand language acquisition (ontogenetic development), evolution (phylogenetic development), and production. Just because that goals seems unattainable doesn't, to me, mean that we need to aim any lower. Now, this is premised on my belief that mimicking phenomena with statistical learning is not as intellectually satisfying as understanding the underlying cognitive systems, but that's not believed by everyone.

lkozma 15y ago
I agree that learning to produce and interpret varied utterances is a worthy goal, but the fact is that (far as it still is today) lowly statistical methods have gotten us closer to this goal than the other, chomskian approach. It could be a situation where aiming lower lets you shoot higher.
- anghyflawn 15y ago
  This is a fundamental misunderstanding of what modern generative linguistics is all about (to be fair, it is extremely widespread). The aim of this branch of science is expressly not to "learn to produce and interpret varied utterances" (called E-language in the jargon), but to understand the cognitive processes behind the production and interpretation of utterances (called I-language). Now you may agree or disagree with the methods and assumptions used in the pursuit of this goal, but it is patently unfair to accuse the field of failing to do something it never set out to do.

pealco 15y ago

> I think Norvig acknowledges the point you are making here, namely that the statistical approach does not explain the cognitive systems behind language. If that is the case, then the argument that Norvig is making is irrelevant to the argument Chomsky is making. Chomsky simply makes the point that statistical accounts lack explanatory adequacy. As someone who has worked closely with many of his students and who has received extensive training on his scientific program, I can say with confidence that Chomsky would have no objection whatsoever about the usefulness of statistical approaches to linguistic engineering problems. The results speak for themselves. He would go on to say, however, that how well a statistical approach solves a linguistic engineering problem is irrelevant to the question of how humans do what they do. The answer to the question may well be statistically grounded. That is a valid hypothesis and a logical possibility which should be taken seriously. However, it is incumbent on the proponents of such an answer to provide evidence that it is what humans are doing. Here are some examples of the kinds of evidence necessary: * evidence that humans are capable of performing the kinds of computations that the statistical approach requires, * evidence that the statistical approach works with the relatively limited amount of data that a human receives, * evidence that the statistical approach fails in ways that humans fail How well a statistical approach succeeds at an engineering task is not an item on this list, simply, again, because engineering tasks are irrelevant to what humans actually do. Let me specifically say that statistical approaches are not, from the start, ruled out as potential candidates for the algorithms underlying human language. It's just that a case has to be made for them using the right kind of evidence. Finally, I'll reiterate what others have pointed out: from a scientific perspective, that something is hard to explain doesn't mean that we shouldn't try. And, those that have given up (as you suggest Norvig has) shouldn't fault those who haven't for calling them out on it.