3 ms·

It feels harsh to call him naïve because I'm sure I wouldn't have had any more insight, but it is surprising how little we knew about the science of language pr

by sjy 7y ago

It feels harsh to call him naïve because I'm sure I wouldn't have had any more insight, but it is surprising how little we knew about the science of language pre-Chomsky, considering that we have been learning languages for millions of years. Now that we have had several generations of computer technology fall short of the abilities of human translators, it seems obvious that you're probably not going to be able to write a machine translation program on a computer with a few megabytes of memory.

skybrian 7y ago
Did you mean post-Chomsky? It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars (or anything Chomsky would recognize) and switched to statistics.
- cr0sh 7y ago
  > It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars...and switched to statistics. What's curious about this statement (and forgive me if I am in error here; I am not extremely familiar with the machine translation field) is that the community chose to go down the route of "traditional grammars" and their rules, for the purposes of translation, rather than down the route of statistics to begin with. Because as it was noted, this happened after WW2 and the advances in cryptography, which were by and large also driven by statistical analysis and other similar mathematically based advances; one would think that such insights would have carried over into the machine translation realm. But they didn't - which is simply a curious historical footnote. It's also something we see often in history - particularly as it relates to technological advances; that there are certain paths that are taken that lead to virtual dead-ends decades later, but which the path that should have been taken was presaged by earlier work, yet for one reason or another wasn't pursued. If we only had a way to avoid such "wrong turns", we could be much further along technologically - but of course, that would also be tantamount to "telling the future"...almost.
  - mcguire 7y ago
    The downside of statistical models has always been understanding and explaining why they do whatever they do. Any statistical model will generate a result, either right or wrong, but it's impracticable to figure out how it came up with that result: the only knob you have is training data. "Symbolic" techniques are the usual first choice because in order to get them to work reasonably well (i.e. beyond the 60%-80% threshold), you have to understand what they're doing.
- burnte 7y ago
  "It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars (or anything Chomsky would recognize) and switched to statistics." The same thing happened in speech recognition. IBM started throwing statistical modeling and horsepower at the speech recognition problem, and folks like Kurzweil thought it was inelegant and not "true" speech recognition. However, it serves the purpose, and we know that humans use similar tricks when listening to augment pure recognition. It's harder to hear someone in a loud room if you can't see their mouth, or don't know the context of what they're saying.
- sjy 7y ago
  I did mean pre-Chomsky. I was using him as a rough reference point for when linguistics started to get serious as a scientific discipline. Because people have been informally studying linguistics for all of human history (by trying to learn new languages in adulthood), it's surprising to me that we ever thought that natural language processing would be easily solved by computers. To me, it seems quite obvious that this is a fundamentally very challenging problem for computers, like theorem proving or painting, rather than accounting or printing. I was born with the benefit of hindsight here, but I find it strange that computability theory was formalised in the 1930s (before we could actually build powerful computers), and yet decades later, leading experts still failed to realise that language processing was fundamentally difficult, and intractable on the hardware they had at the time. This seems like an obvious consequence of a number of facts which they did know, like "describing an algorithm is very different to having a normal conversation," and "texts are open to interpretation and cannot be perfectly translated into other languages."
raverbashing 7y ago
To be very honest you don't need Chomsky to realize how complicated languages are People who think it's easy usually don't speak a 2nd language (amongst other problems in communicating with non-natives for example) so I'm going to bet on that.

by sjy 7y ago

skybrian 7y ago

Did you mean post-Chomsky? It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars (or anything Chomsky would recognize) and switched to statistics.

cr0sh 7y ago
> It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars...and switched to statistics. What's curious about this statement (and forgive me if I am in error here; I am not extremely familiar with the machine translation field) is that the community chose to go down the route of "traditional grammars" and their rules, for the purposes of translation, rather than down the route of statistics to begin with. Because as it was noted, this happened after WW2 and the advances in cryptography, which were by and large also driven by statistical analysis and other similar mathematically based advances; one would think that such insights would have carried over into the machine translation realm. But they didn't - which is simply a curious historical footnote. It's also something we see often in history - particularly as it relates to technological advances; that there are certain paths that are taken that lead to virtual dead-ends decades later, but which the path that should have been taken was presaged by earlier work, yet for one reason or another wasn't pursued. If we only had a way to avoid such "wrong turns", we could be much further along technologically - but of course, that would also be tantamount to "telling the future"...almost.
- mcguire 7y ago
  The downside of statistical models has always been understanding and explaining why they do whatever they do. Any statistical model will generate a result, either right or wrong, but it's impracticable to figure out how it came up with that result: the only knob you have is training data. "Symbolic" techniques are the usual first choice because in order to get them to work reasonably well (i.e. beyond the 60%-80% threshold), you have to understand what they're doing.
burnte 7y ago
"It's my understanding that machine translation started to become semi-useful after abandoning traditional grammars (or anything Chomsky would recognize) and switched to statistics." The same thing happened in speech recognition. IBM started throwing statistical modeling and horsepower at the speech recognition problem, and folks like Kurzweil thought it was inelegant and not "true" speech recognition. However, it serves the purpose, and we know that humans use similar tricks when listening to augment pure recognition. It's harder to hear someone in a loud room if you can't see their mouth, or don't know the context of what they're saying.
sjy 7y ago
I did mean pre-Chomsky. I was using him as a rough reference point for when linguistics started to get serious as a scientific discipline. Because people have been informally studying linguistics for all of human history (by trying to learn new languages in adulthood), it's surprising to me that we ever thought that natural language processing would be easily solved by computers. To me, it seems quite obvious that this is a fundamentally very challenging problem for computers, like theorem proving or painting, rather than accounting or printing. I was born with the benefit of hindsight here, but I find it strange that computability theory was formalised in the 1930s (before we could actually build powerful computers), and yet decades later, leading experts still failed to realise that language processing was fundamentally difficult, and intractable on the hardware they had at the time. This seems like an obvious consequence of a number of facts which they did know, like "describing an algorithm is very different to having a normal conversation," and "texts are open to interpretation and cannot be perfectly translated into other languages."

raverbashing 7y ago

To be very honest you don't need Chomsky to realize how complicated languages are People who think it's easy usually don't speak a 2nd language (amongst other problems in communicating with non-natives for example) so I'm going to bet on that.