5 ms·

Why should it not be possible to solve this with statistical methods? The model just needs to be able to understand the important meaning of "no" in here, in th

Why should it not be possible to solve this with statistical methods? The model just needs to be able to understand the important meaning of "no" in here, in the context of the whole sentence. I would guess that most modern NNs from the NLP area (Transformer or LSTM) would be able to correctly differentiate the meaning. The problem is, I think there is no fancy NN (yet) behind Google search, and the other web searches. To extend on that, you can think of the human brain as just another (powerful) statistical model.

animalCrax0rz 7y ago
"there is no evidence of cancer" and "there is evidence of no cancer" are two different statements with different meaning, so it's more complex a task than just understanding the importance of "no" in a sentence. It's involves semantic analysis of the sentence. The paper I linked to below describes a technique they call "deep parsing." Check it out for more context.
- dhimes 7y ago
  While it was an interesting point, it doesn't seem completely applicable here. Presumably the classifications include context. "A shirt with no stripes" should be distinguishable from "a stripes with no shirt" in the this context.
  - deleted 7y ago
    [deleted]
- londt8 7y ago
  I tried to search for "cheese without holes" on Google and it yielded good results. I think the problem here is that the query is something people would rarely search.
  - jjnoakes 7y ago
    I just searched google images for "cheese" and "cheese without holes" and I got roughly the same results (about 1/3 of the images had holes in both cases).
    - Miraste 7y ago
      "pictures" and "pictures without color" show that it does get some of these, although not the way I expected.
  - derivativethrow 7y ago
    If that's the case that's a big problem, because human children are trivially capable of both formulating and understanding uncommon sentences which still make sense. It might be hard to come up with examples on the spot, but in everyday life you will routinely come across things you need to refer to by negation which are relatively uncommon.
    - derefr 7y ago
      Human children are also capable of walking, but it’s one of the hardest problems in robotics. Things brains can do intuitively, they can do because they have millions of years of evolution behind them.
      - im3w1l 7y ago
        Of the things the human brain does, I wouldn't say walking is very interesting. The most amazing property in my opinion is the fact that it trains itself. (Whereas neural networks are trained by external systems). I suspect it's related to imprinting. To take the example of filial imprinting, the brain must have some hardcoded notion of what a parent looks like. Then this is used to to build a parent detector, and the hardcoded notion is thereafter discarded. Then the newly learnt parent detector is used in reinforcement learning (near parent = good). Keep in mind that this all happens just after birth or hatching, before the visual centres of the brain have had any chance to train. Really cool stuff.
  - worik 7y ago
    THat will be your search bubble. You clearly prefer Stilton or Chedder to Emmental cheese
- atq2119 7y ago
  This is true, but isn't really that relevant to the parent's point about statistical methods. Statistical methods (and "deep learning" is such a method) could certainly take the order of words in a sentence into account, for example.
  - animalCrax0rz 7y ago
    Which part of "The paper I linked to below describes a technique they call "deep parsing." Check it out for more context." could you not parse? I mean if us humans have a difficulty parsing each other's statements, then why should machines do any better?
    - WillPostForFood 7y ago
      The number of humans that would have difficulty parsing "shirt without stripes" is very very low.
      - animalCrax0rz 7y ago
        I keep trying to talk about the general problem and people keep focusing on this one example. I give up. :)
    - atq2119 7y ago
      None. I was merely pointing out that your comment wasn't a response to albertzeyer's point about statistical methods. That is to say, maybe you didn't parse their comment properly ;)
      - animalCrax0rz 7y ago
        Nope, I was simply countering that it's not as simple as they suggested. I added the link to the paper to show one approach to given them some context as to what might it look like.
- albertzeyer 7y ago
  Yes, sure, exactly. It's a complex task. I'm just saying that there is no reason why a statistical model should not be able to solve the task. And you seem to agree on that. You even linked such a method.
  - animalCrax0rz 7y ago
    Yes, I was just countering that it’s more involved than recognizing the importance of “no” in a sentence.
- pbhjpbhj 7y ago
  Lexical analysis is surely clever enough to make "no evidence" and "no cancer" the atoms used and should even differentiate "no cancer" from "no, cancer" pretty easily? Are we really still at 80s chatbot level functionality when it comes to string parsing? Related aside: It frustrates me no end that spellcheck still doesn't appear to use any probablistic considerations, like Markov chains, to determine the intended word. And that when I click the next to last letter to make an adjustment it doesn't then change the suggestions to alternate endings, etc.. Perhaps newer devices than I have do this.
  - zerd 7y ago
    The general problem is much harder than that. You need to understand double negatives, when do they invert the meaning and when do they underline it? "Ain't nobody got time for that" can be interpreted in different ways. And then you need to understand sarcasm. Then a later sentence can invert the meaning of an earlier one. E.g. "This is the best movie ever. Said no one". Using word-pairs as features is easy, but there are just so many exceptions and ambiguity it's a very difficult problem to solve well.
    - _bxg1 7y ago
      It gets even harder than that: https://en.m.wikipedia.org/wiki/Winograd_Schema_Challenge https://en.m.wikipedia.org/wiki/Winograd_Schema_Challenge For example: "The city councilmen refused the demonstrators a permit because they advocated violence." Which party is "they"? There is no lexical information that can possibly answer this question. It depends entirely on an actual understanding of what "city councilmen" and "demonstrators" (in the context of city councilmen and permits!) are, and which one would be more likely to be advocating violence (and in which case that would lead to a permit denial). Background: Until recently I worked at a symbolic AI company who was tackling this problem. I myself didn't work on this problem directly, but I became 100% convinced that their approach, while a long shot, was the only conceivable way of solving it in a fully generalized way.
      - chx 7y ago
        It gets even worse: "The city councilmen refused the demonstrators a permit because they advocated peace." the meaning of this sentence in 1968 is different than in 2020!
rhizome 7y ago
>The model just needs to be able to understand the important meaning of "no" in here, in the context of the whole sentence It's easy to say, isn't it? Unfortunately, sticking the word "just" in there doesn't affect the difficulty. I do it all the time, too. That said, "meaning" is not statistical.
- hammock 7y ago
  Why is it hard? Honestly. Add a "t" to "no" and you get a basic Boolean query.
  - gnulinux 7y ago
    Boolean queries aren't statistically analyzed; they're parsed and algebraically evaluate. So your point seems to support the opposite thing you're supporting.
    - hammock 7y ago
      Yeah I don't really know how github works. When I click the OP link I see a tiny bit of text and then three screenshots. Is there a whole article on this topic I'm missing?
- waltpad 7y ago
  > That said, "meaning" is not statistical. Are you sure of that? After all, we don't have all the same interpretation for every word in the dictionary. Ask a hundred person in the street - well perhaps not these days - the meaning of the word "meaning", how many different explanations will you get? And will you be able to reduce them all to the same "fundamental" meaning?
DenisM 7y ago
This very question was the subject of a lengthy debate between Norvig and Chomsky. I won't rehash it here, but here's a glimpse: Chomsky: Statistical analysis of snowflakes falling outside the window may predict the next snowflake, but it will do very little for weather prediction, and nothing for climate analysis. Norvig: Give us enough data and we will get close enough for all practical purposes.
- hammock 7y ago
  This kind of reminds me of the plot of the current season on Westworld (no spoilers). The amount of data needed to really have "enough data" is often not practical, therefore (ironically) never letting you apply it "for practical purposes."
ScoutOrgo 7y ago
The OP most likely isn't up to date on latest NN architectures which can learn from the ordering of words. Statistical methods <> ML, not anymore in NLP.
- melenaboija 7y ago
  I guess it depends on what you mean with statistical methods but as you said "not anymore in NLP" then yes, statistical methods are in NLP. Both language models and transformers use statistics to predict.
- lonelappde 7y ago
  Statistical linguistics is older than modern neural nets. Google itself promoted its research into same.
BickNowstrom 7y ago
> Why should it not be possible to solve this with statistical methods? Doing just that for 10 years, beating hand-coded systems: https://www-nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf https://www-nlp.stanford.edu/pubs/SocherLinNgManning_ICML201... [pdf] > I would guess that most modern NNs from the NLP area (Transformer or LSTM) would be able to correctly differentiate the meaning. Yes. See demos like: https://demo.allennlp.org/constituency-parsing/MTczNjYyNA== https://demo.allennlp.org/constituency-parsing/MTczNjYyNA== and https://demo.allennlp.org/dependency-parsing/MTczNjYyNg== https://demo.allennlp.org/dependency-parsing/MTczNjYyNg== > I think there is no fancy NN (yet) behind Google search, During the deep learning boom, Google made a huge push towards NN-based NLP. SEO's and their PR calls their efforts collectively RankBrain: https://en.wikipedia.org/wiki/RankBrain https://en.wikipedia.org/wiki/RankBrain I think we are on the cusp of combining symbolical/logical operations over the vectors produced by Neural Networks (or at least, major effort there). Could be by neatly tying up all these different NN-based NLP modules (parsing, semantic distance, knowledge bases, ...) with another set of decision layers stacked on top.

animalCrax0rz 7y ago

"there is no evidence of cancer" and "there is evidence of no cancer" are two different statements with different meaning, so it's more complex a task than just understanding the importance of "no" in a sentence. It's involves semantic analysis of the sentence. The paper I linked to below describes a technique they call "deep parsing." Check it out for more context.

dhimes 7y ago
While it was an interesting point, it doesn't seem completely applicable here. Presumably the classifications include context. "A shirt with no stripes" should be distinguishable from "a stripes with no shirt" in the this context.
- deleted 7y ago
  [deleted]
londt8 7y ago
I tried to search for "cheese without holes" on Google and it yielded good results. I think the problem here is that the query is something people would rarely search.
- jjnoakes 7y ago
  I just searched google images for "cheese" and "cheese without holes" and I got roughly the same results (about 1/3 of the images had holes in both cases).
  - Miraste 7y ago
    "pictures" and "pictures without color" show that it does get some of these, although not the way I expected.
- derivativethrow 7y ago
  If that's the case that's a big problem, because human children are trivially capable of both formulating and understanding uncommon sentences which still make sense. It might be hard to come up with examples on the spot, but in everyday life you will routinely come across things you need to refer to by negation which are relatively uncommon.
  - derefr 7y ago
    Human children are also capable of walking, but it’s one of the hardest problems in robotics. Things brains can do intuitively, they can do because they have millions of years of evolution behind them.
    - im3w1l 7y ago
      Of the things the human brain does, I wouldn't say walking is very interesting. The most amazing property in my opinion is the fact that it trains itself. (Whereas neural networks are trained by external systems). I suspect it's related to imprinting. To take the example of filial imprinting, the brain must have some hardcoded notion of what a parent looks like. Then this is used to to build a parent detector, and the hardcoded notion is thereafter discarded. Then the newly learnt parent detector is used in reinforcement learning (near parent = good). Keep in mind that this all happens just after birth or hatching, before the visual centres of the brain have had any chance to train. Really cool stuff.
- worik 7y ago
  THat will be your search bubble. You clearly prefer Stilton or Chedder to Emmental cheese
atq2119 7y ago
This is true, but isn't really that relevant to the parent's point about statistical methods. Statistical methods (and "deep learning" is such a method) could certainly take the order of words in a sentence into account, for example.
- animalCrax0rz 7y ago
  Which part of "The paper I linked to below describes a technique they call "deep parsing." Check it out for more context." could you not parse? I mean if us humans have a difficulty parsing each other's statements, then why should machines do any better?
  - WillPostForFood 7y ago
    The number of humans that would have difficulty parsing "shirt without stripes" is very very low.
    - animalCrax0rz 7y ago
      I keep trying to talk about the general problem and people keep focusing on this one example. I give up. :)
  - atq2119 7y ago
    None. I was merely pointing out that your comment wasn't a response to albertzeyer's point about statistical methods. That is to say, maybe you didn't parse their comment properly ;)
    - animalCrax0rz 7y ago
      Nope, I was simply countering that it's not as simple as they suggested. I added the link to the paper to show one approach to given them some context as to what might it look like.
albertzeyer 7y ago
Yes, sure, exactly. It's a complex task. I'm just saying that there is no reason why a statistical model should not be able to solve the task. And you seem to agree on that. You even linked such a method.
- animalCrax0rz 7y ago
  Yes, I was just countering that it’s more involved than recognizing the importance of “no” in a sentence.
pbhjpbhj 7y ago
Lexical analysis is surely clever enough to make "no evidence" and "no cancer" the atoms used and should even differentiate "no cancer" from "no, cancer" pretty easily? Are we really still at 80s chatbot level functionality when it comes to string parsing? Related aside: It frustrates me no end that spellcheck still doesn't appear to use any probablistic considerations, like Markov chains, to determine the intended word. And that when I click the next to last letter to make an adjustment it doesn't then change the suggestions to alternate endings, etc.. Perhaps newer devices than I have do this.
- zerd 7y ago
  The general problem is much harder than that. You need to understand double negatives, when do they invert the meaning and when do they underline it? "Ain't nobody got time for that" can be interpreted in different ways. And then you need to understand sarcasm. Then a later sentence can invert the meaning of an earlier one. E.g. "This is the best movie ever. Said no one". Using word-pairs as features is easy, but there are just so many exceptions and ambiguity it's a very difficult problem to solve well.
  - _bxg1 7y ago
    It gets even harder than that: https://en.m.wikipedia.org/wiki/Winograd_Schema_Challenge https://en.m.wikipedia.org/wiki/Winograd_Schema_Challenge For example: "The city councilmen refused the demonstrators a permit because they advocated violence." Which party is "they"? There is no lexical information that can possibly answer this question. It depends entirely on an actual understanding of what "city councilmen" and "demonstrators" (in the context of city councilmen and permits!) are, and which one would be more likely to be advocating violence (and in which case that would lead to a permit denial). Background: Until recently I worked at a symbolic AI company who was tackling this problem. I myself didn't work on this problem directly, but I became 100% convinced that their approach, while a long shot, was the only conceivable way of solving it in a fully generalized way.
    - chx 7y ago
      It gets even worse: "The city councilmen refused the demonstrators a permit because they advocated peace." the meaning of this sentence in 1968 is different than in 2020!

rhizome 7y ago

>The model just needs to be able to understand the important meaning of "no" in here, in the context of the whole sentence It's easy to say, isn't it? Unfortunately, sticking the word "just" in there doesn't affect the difficulty. I do it all the time, too. That said, "meaning" is not statistical.

hammock 7y ago
Why is it hard? Honestly. Add a "t" to "no" and you get a basic Boolean query.
- gnulinux 7y ago
  Boolean queries aren't statistically analyzed; they're parsed and algebraically evaluate. So your point seems to support the opposite thing you're supporting.
  - hammock 7y ago
    Yeah I don't really know how github works. When I click the OP link I see a tiny bit of text and then three screenshots. Is there a whole article on this topic I'm missing?
waltpad 7y ago
> That said, "meaning" is not statistical. Are you sure of that? After all, we don't have all the same interpretation for every word in the dictionary. Ask a hundred person in the street - well perhaps not these days - the meaning of the word "meaning", how many different explanations will you get? And will you be able to reduce them all to the same "fundamental" meaning?

DenisM 7y ago

This very question was the subject of a lengthy debate between Norvig and Chomsky. I won't rehash it here, but here's a glimpse: Chomsky: Statistical analysis of snowflakes falling outside the window may predict the next snowflake, but it will do very little for weather prediction, and nothing for climate analysis. Norvig: Give us enough data and we will get close enough for all practical purposes.

hammock 7y ago
This kind of reminds me of the plot of the current season on Westworld (no spoilers). The amount of data needed to really have "enough data" is often not practical, therefore (ironically) never letting you apply it "for practical purposes."

ScoutOrgo 7y ago

The OP most likely isn't up to date on latest NN architectures which can learn from the ordering of words. Statistical methods <> ML, not anymore in NLP.

melenaboija 7y ago
I guess it depends on what you mean with statistical methods but as you said "not anymore in NLP" then yes, statistical methods are in NLP. Both language models and transformers use statistics to predict.
lonelappde 7y ago
Statistical linguistics is older than modern neural nets. Google itself promoted its research into same.

BickNowstrom 7y ago

> Why should it not be possible to solve this with statistical methods? Doing just that for 10 years, beating hand-coded systems: https://www-nlp.stanford.edu/pubs/SocherLinNgManning_ICML2011.pdf https://www-nlp.stanford.edu/pubs/SocherLinNgManning_ICML201... [pdf] > I would guess that most modern NNs from the NLP area (Transformer or LSTM) would be able to correctly differentiate the meaning. Yes. See demos like: https://demo.allennlp.org/constituency-parsing/MTczNjYyNA== https://demo.allennlp.org/constituency-parsing/MTczNjYyNA== and https://demo.allennlp.org/dependency-parsing/MTczNjYyNg== https://demo.allennlp.org/dependency-parsing/MTczNjYyNg== > I think there is no fancy NN (yet) behind Google search, During the deep learning boom, Google made a huge push towards NN-based NLP. SEO's and their PR calls their efforts collectively RankBrain: https://en.wikipedia.org/wiki/RankBrain https://en.wikipedia.org/wiki/RankBrain I think we are on the cusp of combining symbolical/logical operations over the vectors produced by Neural Networks (or at least, major effort there). Could be by neatly tying up all these different NN-based NLP modules (parsing, semantic distance, knowledge bases, ...) with another set of decision layers stacked on top.