7 ms·

The history of NLP is littered with people claiming on theoretical grounds that XYZ is unattainable using purely statistical methods, and that some notion of th

The history of NLP is littered with people claiming on theoretical grounds that XYZ is unattainable using purely statistical methods, and that some notion of the logical structure of language needs to be brought in. And yet one by one, the XYZ have been attained by statistical methods. If you think there's something NLP can't do using machine learning, make a challenge dataset. That would be much more useful than yet another theoretical argument.

throwawaygh 5y ago
Ironically, exactly these sorts of "statistics can't..." arguments are sort of theoretically bankrupt. Either the thing you want to do is impossible or else a learned model can do it at least almost as well as... idk what the alternative even is, something not learned? Taking this to the extreme, on an example where I have first-hand experience: I would never recommend replacing your compiler passes with transformers. The latter will be buggier, at best marginally faster, and it will take several orders of magnitude more effort to get them to work well enough for production. ML isn't the right tool. But, I mean, you can do it. You shouldn't. But you can. To be fair to the article, the title is "won't", not "can't". And that's easier to believe, at least for me. It's not that X is unattainable using ML; it's that some other approach will get there first.
- tudelo 5y ago
  Are there not situations where a learned model could be insufficient due to lack of real-time learning? It is my understanding that a lot of current applications take a ton of training time / computing power and it's not so trivial to do in the moment. I guess this is less of a theoretical problem and more of a practical problem though.
  - throwawaygh 5y ago
    > I guess this is less of a theoretical problem and more of a practical problem though. Yeah, exactly. Theoretically, ML can do anything. Practically, not so much.
- snek_case 5y ago
  The reason you wouldn't want to use ML for program transformations is that compiler analyses like constant folding are essentially fast, specialized theorem provers. When you need to know if an optimization is safe or not (is this expression equivalent to that one?), you want an exact solution, not just a pretty good guess, and there are known techniques that can give you just that. However, you can use ML in areas where compilers use heuristics. For example, how many times should you unroll this loop, or should you inline function B into function A? Your program is going to be valid with or without inlining/unrolling, and the compiler needs an "intuition" of what's going to be best for performance. Right now, in most cases, this uses simple hardcoded rules. The same goes for math. You wouldn't ask a neural network to certify whether a mathematical proof is valid or not, knowing it has been 99% accurate on the test set. You couldn't trust it to verify all the steps. However, you could use a neural network to help you guide a formal theorem prover in searching for a proof, as in which branches of the infinite tree of mathematical expressions should you search.
  - throwawaygh 5y ago
    Sure. It was an extreme example of the difference between "can't" and "won't", and a point about theory vs practice.
- didibus 5y ago
  I think that's a great example, the question is: can ML be as good as we are at compiling? And then, if ML is as good as we are, and clearly we're not good enough that we had to write a systematic proof system to compile things correctly, could ML have invented this same technique and implemented that compiler correctly? In the end, we're just back at the same place, can ML be as intelligent as we are. Is it the holy grail of AI? What are its limits?
jasonwatkinspdx 5y ago
Yeah, Norvig has a rather famous rebuttal of Chomsky up on his blog from years ago, and everything that's happened since then has supported Norvig's position.
- drdeca 5y ago
  Is this the one you mean : https://norvig.com/chomsky.html https://norvig.com/chomsky.html ?
  - kleiba 5y ago
    If it is then I wouldn't necessarily call it a rebuttal because Norvig broadly agrees with Chomsky on the points under discussion. And that is that statistical models are not useful scientific theories about language in themselves. Where they disagree is what consequences to draw from it: Chomsky discards studying language by statistical means altogether, but Norvig offers a more nuanced position: he argues that there might still be a merit in statistical approaches to language, but not as a result so much in itself but rather as a valid tool that can ultimately help derive a deeper understanding. I think this is a valuable distinction. Unfortunately, though, it is one that the research landscape seems to have somehow forgotten about over the last decade or so. More tangibly speaking using an example, while the capabilities of a language model like GPT-3 are amazing, Norvig's point is that science should not just stop there - it should ask the question "what does GPT-3 teach us about human language?"
    - foldr 5y ago
      >Chomsky discards studying language by statistical means altogether Chomsky nowhere does this. That is at best a straw man constructed by Norvig of Chomsky's position. If you just look at the transcript of the Chomksy interview in the Norvig piece, you can see Chomsky describing a case where statistical analysis is useful.
- foldr 5y ago
  This rebuttal has always bugged me because of the egregious error Norvig makes in his analysis of pro drop. The dropping of subjects in English matrix clauses (e.g. "Not gonna do it") is known to linguists as "diary drop", and is a totally different phenomenon from the dropping of subjects in so-called "null subject" or "pro drop" languages like Spanish. The most obvious difference is that diary drop is completely impossible in embedded clauses: I think I'm not gonna do it. [good in English] I think not gonna do it. [bad in English] (Yo) creo que no voy a hacerlo. [good in Spanish] To linguists it's immensely frustrating that someone who's considered by the compsci crowd to be a leader in the field of NLP can't even get some basic facts of linguistic analysis correct. "Linguists can argue over the interpretation of these facts for hours on end", Norvig says dismissively. But any competent syntactician could have explained to him in five minutes why his examples are irrelevant to the point he's trying to make. The rest of Norvig's post just consists in misunderstanding what Chomsky is saying, as far as I can see. Chomsky is saying that statistical analysis alone is not sufficient to achieve scientific progress. I don't think Norvig actually disagrees with Chomsky on this point, but he seems to think that Chomsky is saying that scientists should never use statistical methods. The real kicker here is that Norvig is an engineer, not a scientist. He's contributed very little to the scientific study of language, and yet is lecturing someone who's contributed vastly more on how it ought to be done. Of course, that doesn't necessarily mean that he's wrong, but it does grate. Not to mention that the Chomsy/O'Reilly comparison is little more than trolling – it hardly seems calculated to stimulate an intelligent response.
joe_the_user 5y ago
I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. If they could define exactly what it was, they could indeed describe a way to train for it or program it. So a theoretical argument could useful if gave us an idea what are our mysterious something it. However, I'm not sure if this article makes a contribution here. Proofs of impossibility generally aren't useful, as you say.
- schoen 5y ago
  Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it!
  - joe_the_user 5y ago
    Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it! That can be true without being demonstration that we're seeing unbounded progress in language use and understanding. A theory of the limit of NLP is still a theory of NLP and by no means better than other theories.
- trabant00 5y ago
  > "understanding" or something. But they can't say exactly what it is. Umm, you just said it yourself, understanding. The model has no clue whatsoever what any of the output means, just that it scores good. I don't understand Japanese. But given enough time and feedback I can try all the possible combinations of sounds I can come up with and answer a question in Japanese till I get a satisfying feedback from the Japanese speaker (ex: his facial expression) that asked the question. Is this satisfactory as for speaking Japanese in your opinion?
  - ivalm 5y ago
    That’s a red room style argument, but I’m guessing if you have time to get feedback/etc on all possible combinations you might just learn Japanese (because it is informationally compact to get high scores through semantic understanding rather that wrote memorization).
schoen 5y ago
I'm sad that statistical methods have gained so much ground over more formal and logical methods (edit: maybe I should instead say "that formal and logical methods have lost so much ground compared to statistical methods"), and, while I can see ways to construct examples that the statistical methods ought to have trouble with, I also notice how incredibly well they've done and how many barriers they've blown past in a short period of time. Recently I was impressed to see that the Winograd schema challenge -- a particular challenge dataset of the kind you mention -- saw huge performance gains from statistical NLP methods, which implies that the statistical methods encoded (something that at least sometimes works a lot like a substitute for) common sense about the world and the context of utterances. The Winograd schemas were my favorite example of "XYZ" here. They involve pairs of sentences where the sentence is parsed differently based on assumptions from external-world knowledge. For example Putting the barbell on the glass table isn't a good idea, because it's too (heavy/flimsy). What is too heavy? What is too flimsy? Or, from memory rather than finding the original version The local authorities refused a permit to the demonstrators because they (feared/advocated) violence. Who feared violence? Who advocated violence? The compiler confirmed that that the program was buggy because it (enforced/failed) typechecking. What enforced typechecking? What failed typechecking? The doctor recommended surgery for the patient because she was (suffering from/experienced with) coronary artery blockage. Who was suffering from coronary artery blockage? Who was experienced with coronary artery blockage? In these cases, the most plausible referent of a pronoun changes depending on the semantics of the other word that you fill in, for a reason that has to do with the outside world, not syntax. (I don't each case absolutely has to include a "because" clause, but it's the simplest way to supply disambiguating context.) I totally thought statistical techniques would be horrible at this. There's good reason to think that these require some detailed knowledge about the world. The statistical techniques were horrible at this, for a while, and now they're abruptly great at it (if I remember correctly, at or above typical human performance).
- suifbwish 5y ago
  The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine if we left our understanding and working model of physics completely up to statistical and deep learning models. We would not be able to reliably design anything or model simulations ect. The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it)
poorjohnmacafee 5y ago
Well, have you seen any industry NLP product that exercises any meaningful level of language understanding? I haven't. The Turing test has never even been close to being passed at the annual Loebner Prize. The best such industry NLP products I know of don't even use ML at their cores.
zR0x 5y ago
If it’s just stats why do I need Google? I can write code to bubble up stats across text sources in any language. I’m reminded of a behind the scenes video of Batman 1989, where they discuss a complicated layering of Joker makeup to achieve the effect in the scene where he wipes off fleshy colored face paint; white makeup on Nicholson, special coating to be able to apply the next layer, on and on. No one thought to just have him wipe white face paint onto his forehead. I can easily see a bunch of programmers making a mess out of an elegant problem given the spaghetti code I’ve worked in. Maybe English means nothing about consciousness? Many linguists take the position it’s random sounds we’ve been polishing definitions of for years.
- wutbrodo 5y ago
  > If it’s just stats why do I need Google? I can write code to bubble up stats across text sources in any language. I'm not sure I understand your framing here, but in it, aren't you just describing what Google _is_?
SavantIdiot 5y ago
Will the AI be able to ask for clarification? Translators do. In fact, even people speaking the same language need to ask for clarification sometimes. It is literally impossible to discern some sentences without follow up, why would someone think an AI can do it one shot?
- agravier 5y ago
  Why wouldn't an AI be able to ask for clarification in a live setting? I don't see your point.
  - joe_the_user 5y ago
    Well, the approach of solving a corpus (advocated by GP) doesn't leave room for the possibility of asking a follow-up question so you can't train for it that way.
    - rubatuga 5y ago
      It could list all possible solutions based on the response to the followup question it would have asked.
    - visarga 5y ago
      GPT-3 was trained on a massive text collection so it can even ask for clarifying questions if the prompt context allows for it. Why? Because there are plenty of such examples in the training data.
      - drdeca 5y ago
        Sure, but this is based on something kind of like “whether someone would be likely to ask for clarification here”, not based on whether GPT3 needs clarification. Though, I’m confident that it should be possible to make something which, when unsure, asks for clarification. In fact, I believe something at least rather similar is already done in, uh, I think it is called “delegative reinforcement learning”, where an agent takes actions, but when it is highly uncertain, it can instead choose to have an expert (who is assumed to be competent at the task, or at least has a tolerably low risk of very bad outcomes) to step in instead. (Or, wait, maybe delegative reinforcement learning is still just used theoretically? Not sure.)
fooker 5y ago
You can overfit to produce plausible looking statistical approaches, once you have a dataset.
fishmaster 5y ago
There's the HANS dataset which is used to evaluate NLI models to check if they pick up surface heuristics: https://huggingface.co/datasets/hans https://huggingface.co/datasets/hans https://arxiv.org/abs/1902.01007 https://arxiv.org/abs/1902.01007

throwawaygh 5y ago

Ironically, exactly these sorts of "statistics can't..." arguments are sort of theoretically bankrupt. Either the thing you want to do is impossible or else a learned model can do it at least almost as well as... idk what the alternative even is, something not learned? Taking this to the extreme, on an example where I have first-hand experience: I would never recommend replacing your compiler passes with transformers. The latter will be buggier, at best marginally faster, and it will take several orders of magnitude more effort to get them to work well enough for production. ML isn't the right tool. But, I mean, you can do it. You shouldn't. But you can. To be fair to the article, the title is "won't", not "can't". And that's easier to believe, at least for me. It's not that X is unattainable using ML; it's that some other approach will get there first.

tudelo 5y ago
Are there not situations where a learned model could be insufficient due to lack of real-time learning? It is my understanding that a lot of current applications take a ton of training time / computing power and it's not so trivial to do in the moment. I guess this is less of a theoretical problem and more of a practical problem though.
- throwawaygh 5y ago
  > I guess this is less of a theoretical problem and more of a practical problem though. Yeah, exactly. Theoretically, ML can do anything. Practically, not so much.
snek_case 5y ago
The reason you wouldn't want to use ML for program transformations is that compiler analyses like constant folding are essentially fast, specialized theorem provers. When you need to know if an optimization is safe or not (is this expression equivalent to that one?), you want an exact solution, not just a pretty good guess, and there are known techniques that can give you just that. However, you can use ML in areas where compilers use heuristics. For example, how many times should you unroll this loop, or should you inline function B into function A? Your program is going to be valid with or without inlining/unrolling, and the compiler needs an "intuition" of what's going to be best for performance. Right now, in most cases, this uses simple hardcoded rules. The same goes for math. You wouldn't ask a neural network to certify whether a mathematical proof is valid or not, knowing it has been 99% accurate on the test set. You couldn't trust it to verify all the steps. However, you could use a neural network to help you guide a formal theorem prover in searching for a proof, as in which branches of the infinite tree of mathematical expressions should you search.
- throwawaygh 5y ago
  Sure. It was an extreme example of the difference between "can't" and "won't", and a point about theory vs practice.
didibus 5y ago
I think that's a great example, the question is: can ML be as good as we are at compiling? And then, if ML is as good as we are, and clearly we're not good enough that we had to write a systematic proof system to compile things correctly, could ML have invented this same technique and implemented that compiler correctly? In the end, we're just back at the same place, can ML be as intelligent as we are. Is it the holy grail of AI? What are its limits?

jasonwatkinspdx 5y ago

Yeah, Norvig has a rather famous rebuttal of Chomsky up on his blog from years ago, and everything that's happened since then has supported Norvig's position.

drdeca 5y ago
Is this the one you mean : https://norvig.com/chomsky.html https://norvig.com/chomsky.html ?
- kleiba 5y ago
  If it is then I wouldn't necessarily call it a rebuttal because Norvig broadly agrees with Chomsky on the points under discussion. And that is that statistical models are not useful scientific theories about language in themselves. Where they disagree is what consequences to draw from it: Chomsky discards studying language by statistical means altogether, but Norvig offers a more nuanced position: he argues that there might still be a merit in statistical approaches to language, but not as a result so much in itself but rather as a valid tool that can ultimately help derive a deeper understanding. I think this is a valuable distinction. Unfortunately, though, it is one that the research landscape seems to have somehow forgotten about over the last decade or so. More tangibly speaking using an example, while the capabilities of a language model like GPT-3 are amazing, Norvig's point is that science should not just stop there - it should ask the question "what does GPT-3 teach us about human language?"
  - foldr 5y ago
    >Chomsky discards studying language by statistical means altogether Chomsky nowhere does this. That is at best a straw man constructed by Norvig of Chomsky's position. If you just look at the transcript of the Chomksy interview in the Norvig piece, you can see Chomsky describing a case where statistical analysis is useful.
foldr 5y ago
This rebuttal has always bugged me because of the egregious error Norvig makes in his analysis of pro drop. The dropping of subjects in English matrix clauses (e.g. "Not gonna do it") is known to linguists as "diary drop", and is a totally different phenomenon from the dropping of subjects in so-called "null subject" or "pro drop" languages like Spanish. The most obvious difference is that diary drop is completely impossible in embedded clauses: I think I'm not gonna do it. [good in English] I think not gonna do it. [bad in English] (Yo) creo que no voy a hacerlo. [good in Spanish] To linguists it's immensely frustrating that someone who's considered by the compsci crowd to be a leader in the field of NLP can't even get some basic facts of linguistic analysis correct. "Linguists can argue over the interpretation of these facts for hours on end", Norvig says dismissively. But any competent syntactician could have explained to him in five minutes why his examples are irrelevant to the point he's trying to make. The rest of Norvig's post just consists in misunderstanding what Chomsky is saying, as far as I can see. Chomsky is saying that statistical analysis alone is not sufficient to achieve scientific progress. I don't think Norvig actually disagrees with Chomsky on this point, but he seems to think that Chomsky is saying that scientists should never use statistical methods. The real kicker here is that Norvig is an engineer, not a scientist. He's contributed very little to the scientific study of language, and yet is lecturing someone who's contributed vastly more on how it ought to be done. Of course, that doesn't necessarily mean that he's wrong, but it does grate. Not to mention that the Chomsy/O'Reilly comparison is little more than trolling – it hardly seems calculated to stimulate an intelligent response.

joe_the_user 5y ago

I think a lot of people get the impression current NLP models like GPT-3 lack something - "understanding" or something. But they can't say exactly what it is. If they could define exactly what it was, they could indeed describe a way to train for it or program it. So a theoretical argument could useful if gave us an idea what are our mysterious something it. However, I'm not sure if this article makes a contribution here. Proofs of impossibility generally aren't useful, as you say.

schoen 5y ago
Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it!
- joe_the_user 5y ago
  Also, when people have tried to define exactly what it is, the NLP models have then usually turned out to be able to have or acquire it! That can be true without being demonstration that we're seeing unbounded progress in language use and understanding. A theory of the limit of NLP is still a theory of NLP and by no means better than other theories.
trabant00 5y ago
> "understanding" or something. But they can't say exactly what it is. Umm, you just said it yourself, understanding. The model has no clue whatsoever what any of the output means, just that it scores good. I don't understand Japanese. But given enough time and feedback I can try all the possible combinations of sounds I can come up with and answer a question in Japanese till I get a satisfying feedback from the Japanese speaker (ex: his facial expression) that asked the question. Is this satisfactory as for speaking Japanese in your opinion?
- ivalm 5y ago
  That’s a red room style argument, but I’m guessing if you have time to get feedback/etc on all possible combinations you might just learn Japanese (because it is informationally compact to get high scores through semantic understanding rather that wrote memorization).

schoen 5y ago

I'm sad that statistical methods have gained so much ground over more formal and logical methods (edit: maybe I should instead say "that formal and logical methods have lost so much ground compared to statistical methods"), and, while I can see ways to construct examples that the statistical methods ought to have trouble with, I also notice how incredibly well they've done and how many barriers they've blown past in a short period of time. Recently I was impressed to see that the Winograd schema challenge -- a particular challenge dataset of the kind you mention -- saw huge performance gains from statistical NLP methods, which implies that the statistical methods encoded (something that at least sometimes works a lot like a substitute for) common sense about the world and the context of utterances. The Winograd schemas were my favorite example of "XYZ" here. They involve pairs of sentences where the sentence is parsed differently based on assumptions from external-world knowledge. For example Putting the barbell on the glass table isn't a good idea, because it's too (heavy/flimsy). What is too heavy? What is too flimsy? Or, from memory rather than finding the original version The local authorities refused a permit to the demonstrators because they (feared/advocated) violence. Who feared violence? Who advocated violence? The compiler confirmed that that the program was buggy because it (enforced/failed) typechecking. What enforced typechecking? What failed typechecking? The doctor recommended surgery for the patient because she was (suffering from/experienced with) coronary artery blockage. Who was suffering from coronary artery blockage? Who was experienced with coronary artery blockage? In these cases, the most plausible referent of a pronoun changes depending on the semantics of the other word that you fill in, for a reason that has to do with the outside world, not syntax. (I don't each case absolutely has to include a "because" clause, but it's the simplest way to supply disambiguating context.) I totally thought statistical techniques would be horrible at this. There's good reason to think that these require some detailed knowledge about the world. The statistical techniques were horrible at this, for a while, and now they're abruptly great at it (if I remember correctly, at or above typical human performance).

suifbwish 5y ago
The main problem with using stat models or approaches such as deep learning is not that we are unable to do it. Though possibly trivial, the real problem is we are unable to understand how or why it works which can lead to unintended consequences or lack of ability to support/continue further development (aside from not being able to leverage the new fundamental understanding and apply it to related fields). Imagine if we left our understanding and working model of physics completely up to statistical and deep learning models. We would not be able to reliably design anything or model simulations ect. The reason we have not figured out NLP is not because we are incapable, it’s because enough of the right minds have not been looking at it as a puzzle worth solving, possibly because it and other AI related concepts are often introduced at or just before the PhD level and so most minds in science never encounter it (even though they could never encounter any other written concept without it)

poorjohnmacafee 5y ago

Well, have you seen any industry NLP product that exercises any meaningful level of language understanding? I haven't. The Turing test has never even been close to being passed at the annual Loebner Prize. The best such industry NLP products I know of don't even use ML at their cores.

zR0x 5y ago

If it’s just stats why do I need Google? I can write code to bubble up stats across text sources in any language. I’m reminded of a behind the scenes video of Batman 1989, where they discuss a complicated layering of Joker makeup to achieve the effect in the scene where he wipes off fleshy colored face paint; white makeup on Nicholson, special coating to be able to apply the next layer, on and on. No one thought to just have him wipe white face paint onto his forehead. I can easily see a bunch of programmers making a mess out of an elegant problem given the spaghetti code I’ve worked in. Maybe English means nothing about consciousness? Many linguists take the position it’s random sounds we’ve been polishing definitions of for years.

wutbrodo 5y ago
> If it’s just stats why do I need Google? I can write code to bubble up stats across text sources in any language. I'm not sure I understand your framing here, but in it, aren't you just describing what Google _is_?

SavantIdiot 5y ago

Will the AI be able to ask for clarification? Translators do. In fact, even people speaking the same language need to ask for clarification sometimes. It is literally impossible to discern some sentences without follow up, why would someone think an AI can do it one shot?

agravier 5y ago
Why wouldn't an AI be able to ask for clarification in a live setting? I don't see your point.
- joe_the_user 5y ago
  Well, the approach of solving a corpus (advocated by GP) doesn't leave room for the possibility of asking a follow-up question so you can't train for it that way.
  - rubatuga 5y ago
    It could list all possible solutions based on the response to the followup question it would have asked.
  - visarga 5y ago
    GPT-3 was trained on a massive text collection so it can even ask for clarifying questions if the prompt context allows for it. Why? Because there are plenty of such examples in the training data.
    - drdeca 5y ago
      Sure, but this is based on something kind of like “whether someone would be likely to ask for clarification here”, not based on whether GPT3 needs clarification. Though, I’m confident that it should be possible to make something which, when unsure, asks for clarification. In fact, I believe something at least rather similar is already done in, uh, I think it is called “delegative reinforcement learning”, where an agent takes actions, but when it is highly uncertain, it can instead choose to have an expert (who is assumed to be competent at the task, or at least has a tolerably low risk of very bad outcomes) to step in instead. (Or, wait, maybe delegative reinforcement learning is still just used theoretically? Not sure.)

fooker 5y ago

You can overfit to produce plausible looking statistical approaches, once you have a dataset.

fishmaster 5y ago

There's the HANS dataset which is used to evaluate NLI models to check if they pick up surface heuristics: https://huggingface.co/datasets/hans https://huggingface.co/datasets/hans https://arxiv.org/abs/1902.01007 https://arxiv.org/abs/1902.01007