4 ms·

Looks like it can even handle garden path sentences [1] like "the complex houses married and single soldiers and their families." 1: https://en.wikipedia.org/w

by m0th87 10y ago

Looks like it can even handle garden path sentences [1] like "the complex houses married and single soldiers and their families." 1: https://en.wikipedia.org/wiki/Garden_path_sentence https://en.wikipedia.org/wiki/Garden_path_sentence

charlieegan3 10y ago
Do any dependency parsers do this well? I think 'houses' throws them off, e.g.: https://demos.explosion.ai/displacy/?text=The%20complex%20houses%20married%20and%20single%20soldiers%20and%20their%20families.&model=en&cpu=1&cph=1 https://demos.explosion.ai/displacy/?text=The%20complex%20ho... vs. https://demos.explosion.ai/displacy/?text=The%20complex%20contains%20married%20and%20single%20soldiers%20and%20their%20families.&model=en&cpu=1&cph=1 https://demos.explosion.ai/displacy/?text=The%20complex%20co...
ma2rten 10y ago
I don't think this parser parses left-to-right, like humans and other parsers like Parsley MacParseface do.
- botexpert 10y ago
  It does. left-to-right linear time parsing is the idea that's about 15 years old and turned out to be most efficient and most performant.
  - ma2rten 10y ago
    I might be wrong, but based on the parser home page it looks to me like they provide a Shift-reduce parser but it is not the default. I even says on the page Parser: englishPCFG.ser.gz.
    - botexpert 10y ago
      True, it might not be the case, although they do have the exact same beam search shift-reduce parser that mcparseface has (without the neural net).
- kuschku 10y ago
  I’m currently writing a paper on improving natural language interfaces for databases, and I’ve ran both Parsey McParseface and Stanford’s CoreNLP for that, and both have horrible performance. Their results for even the most simple english sentences are horrible, often with multiple mistagged words. As soon as you add relative clauses, they break down entirely.
  - YeGoblynQueenne 10y ago
    Well, it's not really a solved problem, is it? Saying the performance is "horrible" does some injustice to it. It's actually the state of the art in solutions to a very difficult problem. I've used the Stanford parser for university work (at Masters level). It was part of the pipeline for a sentiment extractor. I didn't get the feeling it performed badly, quite the contrary- the parses it generated were quite useful as features in my extractor. It had this rare feeling of a tool that you can actually use to do something interesting and useful. Then again, I do have my expectations set very lowly, for this sort of thing, especially after completing my Masters thesis (on grammar induction). Language learning is hard.
    - kuschku 10y ago
      I mean, it's surprising how well it works, but if you come in, and expect to get a perfect parse tree out of everything, so you can then easily pattern match on top of that, well, you'll be in for a rouch surprise. Any every slightly more complicated sentence was completely misparsed by CoreNLP and ParseyMcParseface. As I mentioned before, as soon as you introduce relative clauses it breaks down. It's hard when your research was supposed to discuss how to better handle the common knowledge problem of NLIDBs, but you have to spend a lot of time just to get a kinda useful parse out in the first place.
malaya_zemlya 10y ago
"The horse raced past the barn fell" still trips it.
invaliduser 10y ago
You usually can't tell from one single sentence if a parser handles garden path sentences, which involve backtracking (hence the name). Plus, there is a part-of-speech tagger that provides an intermediate layer of nlp analysis, which makes a backtracking involving a change of part-of-speech («houses» being either a verb of a noun) quite harder (yet not impossible). POS taggers improve parsers performance dramatically but usually are an indication that the parser have limited backtracking capabilities. You can try it with «the complex houses married soldiers», houses is erroneously marked as a noun.
- YeGoblynQueenne 10y ago
  Yup. It gets it fundamentally wrong, thinking "complex" is an adjective and "houses" is a noun: the/DT complex/JJ houses/NNS married/JJ and/CC single/JJ soldiers/NNS and/CC their/PRP$ families/NNS But, honest, that's a rotten way to try out NL parsers. Nobody claims Stanford's, or anyone's, parser can handle that sort of thing. It throws human language organs because it's specially engineered to do that. And if it can confuse humans, there's no program that it can't confuse, except for one where the right parse is hard-coded. It's an unfair test, is what I'm saying.
kwelstr 10y ago
Try: Time flies like the wind but fruit flies like bananas.

by m0th87 10y ago

charlieegan3 10y ago

Do any dependency parsers do this well? I think 'houses' throws them off, e.g.: https://demos.explosion.ai/displacy/?text=The%20complex%20houses%20married%20and%20single%20soldiers%20and%20their%20families.&model=en&cpu=1&cph=1 https://demos.explosion.ai/displacy/?text=The%20complex%20ho... vs. https://demos.explosion.ai/displacy/?text=The%20complex%20contains%20married%20and%20single%20soldiers%20and%20their%20families.&model=en&cpu=1&cph=1 https://demos.explosion.ai/displacy/?text=The%20complex%20co...

ma2rten 10y ago

I don't think this parser parses left-to-right, like humans and other parsers like Parsley MacParseface do.

botexpert 10y ago
It does. left-to-right linear time parsing is the idea that's about 15 years old and turned out to be most efficient and most performant.
- ma2rten 10y ago
  I might be wrong, but based on the parser home page it looks to me like they provide a Shift-reduce parser but it is not the default. I even says on the page Parser: englishPCFG.ser.gz.
  - botexpert 10y ago
    True, it might not be the case, although they do have the exact same beam search shift-reduce parser that mcparseface has (without the neural net).
kuschku 10y ago
I’m currently writing a paper on improving natural language interfaces for databases, and I’ve ran both Parsey McParseface and Stanford’s CoreNLP for that, and both have horrible performance. Their results for even the most simple english sentences are horrible, often with multiple mistagged words. As soon as you add relative clauses, they break down entirely.
- YeGoblynQueenne 10y ago
  Well, it's not really a solved problem, is it? Saying the performance is "horrible" does some injustice to it. It's actually the state of the art in solutions to a very difficult problem. I've used the Stanford parser for university work (at Masters level). It was part of the pipeline for a sentiment extractor. I didn't get the feeling it performed badly, quite the contrary- the parses it generated were quite useful as features in my extractor. It had this rare feeling of a tool that you can actually use to do something interesting and useful. Then again, I do have my expectations set very lowly, for this sort of thing, especially after completing my Masters thesis (on grammar induction). Language learning is hard.
  - kuschku 10y ago
    I mean, it's surprising how well it works, but if you come in, and expect to get a perfect parse tree out of everything, so you can then easily pattern match on top of that, well, you'll be in for a rouch surprise. Any every slightly more complicated sentence was completely misparsed by CoreNLP and ParseyMcParseface. As I mentioned before, as soon as you introduce relative clauses it breaks down. It's hard when your research was supposed to discuss how to better handle the common knowledge problem of NLIDBs, but you have to spend a lot of time just to get a kinda useful parse out in the first place.

malaya_zemlya 10y ago

"The horse raced past the barn fell" still trips it.

invaliduser 10y ago

You usually can't tell from one single sentence if a parser handles garden path sentences, which involve backtracking (hence the name). Plus, there is a part-of-speech tagger that provides an intermediate layer of nlp analysis, which makes a backtracking involving a change of part-of-speech («houses» being either a verb of a noun) quite harder (yet not impossible). POS taggers improve parsers performance dramatically but usually are an indication that the parser have limited backtracking capabilities. You can try it with «the complex houses married soldiers», houses is erroneously marked as a noun.

YeGoblynQueenne 10y ago
Yup. It gets it fundamentally wrong, thinking "complex" is an adjective and "houses" is a noun: the/DT complex/JJ houses/NNS married/JJ and/CC single/JJ soldiers/NNS and/CC their/PRP$ families/NNS But, honest, that's a rotten way to try out NL parsers. Nobody claims Stanford's, or anyone's, parser can handle that sort of thing. It throws human language organs because it's specially engineered to do that. And if it can confuse humans, there's no program that it can't confuse, except for one where the right parse is hard-coded. It's an unfair test, is what I'm saying.

kwelstr 10y ago

Try: Time flies like the wind but fruit flies like bananas.