4 ms·
I'm sad that statistical methods have gained so much ground over more formal and logical methods (edit: maybe I should instead say "that formal and logical meth
by schoen 5y ago
I'm sad that statistical methods have gained so much ground over more formal and logical methods (edit: maybe I should instead say "that formal and logical methods have lost so much ground compared to statistical methods"), and, while I can see ways to construct examples that the statistical methods ought to have trouble with, I also notice how incredibly well they've done and how many barriers they've blown past in a short period of time.
Recently I was impressed to see that the Winograd schema challenge -- a particular challenge dataset of the kind you mention -- saw huge performance gains from statistical NLP methods, which implies that the statistical methods encoded (something that at least sometimes works a lot like a substitute for) common sense about the world and the context of utterances.
The Winograd schemas were my favorite example of "XYZ" here. They involve pairs of sentences where the sentence is parsed differently based on assumptions from external-world knowledge. For example
Putting the barbell on the glass table isn't a good idea, because it's too (heavy/flimsy).
What is too heavy?
What is too flimsy?
Or, from memory rather than finding the original version
The local authorities refused a permit to the demonstrators because they (feared/advocated) violence.
Who feared violence?
Who advocated violence?
The compiler confirmed that that the program was buggy because it (enforced/failed) typechecking.
What enforced typechecking?
What failed typechecking?
The doctor recommended surgery for the patient because she was (suffering from/experienced with) coronary artery blockage.
Who was suffering from coronary artery blockage?
Who was experienced with coronary artery blockage?
In these cases, the most plausible referent of a pronoun changes depending on the semantics of the other word that you fill in, for a reason that has to do with the outside world, not syntax. (I don't each case absolutely has to include a "because" clause, but it's the simplest way to supply disambiguating context.)
I totally thought statistical techniques would be horrible at this. There's good reason to think that these require some detailed knowledge about the world. The statistical techniques were horrible at this, for a while, and now they're abruptly great at it (if I remember correctly, at or above typical human performance).
- suifbwish 5y agoThe main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine if we left our understanding and working model of physics completely up to statistical and deep learning models. We would not be able to reliably design anything or model simulations ect. The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it)
- why-el 5y agoI think this is the crux of the issue. People like Chomsky argue the same way, that is, sure a statistical model can mimic a phenomenon to an increasingly likely degree, but the question he concerns himself with is "how and why it [language] works that way", not that I can somehow approach it. His example scenario of a filming bees and statistically re-engineer their dance is poignant: sure you get impressive results, but you will never understand why they dance! Once this is clear, the discussion becomes healthier: linguists continue to try and understand fundamentals; ML NLP continues to try to get impressive, possibly lucrative results. Logical research can borrow from ML results, or re-use them, and so forth, but I agree with Chomsky that it can't be a shot in the dark always, because once you deploy it and pass it some tough decisions to make, well, what then? I was pleased to see at least one idea, the recursive structures found in his theories, which is increasingly being accepted by people like Hinton, so that's great.
- visarga 5y ago> but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e...
- visarga 5y ago> but you will never understand why they dance! You can't understand why without considering the environment, the problem is that it's too expensive to train AI agents in reality and simulated environments are too simplistic. But if such a simulated environment is available, agents can learn general skills. > Deep mind: Generally capable agents emerge from open-ended play https://deepmind.com/blog/article/generally-capable-agents-emerge-from-open-ended-play https://deepmind.com/blog/article/generally-capable-agents-e... In this kind of setup the agents know why they act like they do.
- 5y ago
- im3w1l 5y agoI think all you need to do is wait. Eventually the statistical methods will stop progressing so quickly. Either because they starting hitting tough obstacles, or because they manage to solve all the problems they set out to solve and go "now what?". That's when interest in alternative methods will pick up.