3 ms·

I used to think the same way. But after spending the last few years getting frustrated trying to create more complex linguistic systems, e.g., dialog systems or

I used to think the same way. But after spending the last few years getting frustrated trying to create more complex linguistic systems, e.g., dialog systems or scientific articles understanding, I am coming to the conclusion that the current statistical approach is a dead end. It’s actually impeding the field because it’s working so well for certain tasks that when people are trying to build systems with real understanding, they can’t match the performance obtained by gigantic language models. But the answer is not Chomsky, it’s semantic grounding in the real world rather than looking at language as a sequence of symbols.

cgearhart 9y ago
I agree. I think Norvig is right that interpretation and understanding may be inherently probabilistic problems, but the accuracy of statistical models sure seems to just be a matter of "playing the odds"; correctly performing some task doesn't imply any real "understanding" by the model. If the system had "semantic grounding" as you call it, that would go a long way towards disambiguating during interpretation & understanding tasks.
- asdf83123 9y ago
  Norvig didn't really seem to address the "collecting butterflies" part. At least, not sufficiently such I noticed. Sure, science/engineering are fine with allowing for numerous failures and dead ends. Yeah that tracks with history, but is that a good justification? And what about the economy? Doesn't this imply investment into these models is going to be of diminishing value sooner than later? This is Chomsky. Not hard to see it as a tacit indictment of corporate investment these days as well. Corporate is not above manipulation curriculum for their gains. Chomsky is a hardcore proponent of academic freedom. Are we investing a lot of time and money on the wrong efforts? Would we easily acknowledge if we were?

cgearhart 9y ago

I agree. I think Norvig is right that interpretation and understanding may be inherently probabilistic problems, but the accuracy of statistical models sure seems to just be a matter of "playing the odds"; correctly performing some task doesn't imply any real "understanding" by the model. If the system had "semantic grounding" as you call it, that would go a long way towards disambiguating during interpretation & understanding tasks.

asdf83123 9y ago
Norvig didn't really seem to address the "collecting butterflies" part. At least, not sufficiently such I noticed. Sure, science/engineering are fine with allowing for numerous failures and dead ends. Yeah that tracks with history, but is that a good justification? And what about the economy? Doesn't this imply investment into these models is going to be of diminishing value sooner than later? This is Chomsky. Not hard to see it as a tacit indictment of corporate investment these days as well. Corporate is not above manipulation curriculum for their gains. Chomsky is a hardcore proponent of academic freedom. Are we investing a lot of time and money on the wrong efforts? Would we easily acknowledge if we were?