6 ms·

As a linguist and software engineer I can't imagine someone doing serious NLP without ever having studied [concrete] syntax trees and such. It is easy to impres

As a linguist and software engineer I can't imagine someone doing serious NLP without ever having studied [concrete] syntax trees and such. It is easy to impress people with some tokenization but it's n-grams that are really useful in the real world, as is understanding syntax trees and all the interconnections possible inside them so you can NLP the shit out of real world text/speech, instead of simple examples with a tagger (and a good, carefully crafted for a demo like Apple's, trained set of tagged data). This is a good summary article with very good links nonetheless.

riku_iki 8y ago
> as is understanding syntax trees and all the interconnections possible inside them so you can NLP the shit out of real world text/speech But do modern DL approaches (e.g. SQuad, translation models) defy this approach? They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic..
- matachuan 8y ago
  That's how NLP can be a dead end by putting your faith on NN.
  - riku_iki 8y ago
    So far this dead end outperforms traditional ways though.
    - matachuan 8y ago
      Lol for sure any NN-based approach can claim to outperform traditional ways while half of CVPR paper results cannot be reproduced.
      - posterboy 8y ago
        From reading papers on WaveNet and AlphaGo, the problem seems to be incomplete information in the papers, but I am not going to argue with the popular results.
      - amirouche 8y ago
        CVPR?
        mindcrime 8y ago
        Conference on Computer Vision and Pattern Recognition
      - riku_iki 8y ago
        > while half of CVPR paper results cannot be reproduced. even if just another half of results are proven to outperform traditional approaches, is it strong enough trend?
        matachuan 8y ago
        You clearly did not see that was sarcastic.
- ertand 8y ago
  You could jointly train a NN to parse the sentence and tackle the NLP task you have at the same time, or instead of just using the tokens you could use a parser to add more features too. edit: clarification
- yorwba 8y ago
  > They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic. Sure, but that doesn't mean that knowing about syntax won't improve the result. If you're training a translation model on a huge database of labeled examples, it might discover syntactic relationships from scratch. But if you don't have so much data, you're probably better off using all the auxiliary information you can get.
  - riku_iki 8y ago
    > But that doesn't mean that knowing about syntax won't improve the result this is likely correct, additional high quality input will likely improve performance, but creating such input for 200 modern human languages requires significant effort, much larger than allowing NN to solve this problem, that's why researchers invest this effort into NN improvement, not syntax tree creation tooling.
    - hcorreasuarez 8y ago
      All of us have to strike the right balance between stubbornly believing that NNs will solve all problems in NLP -which, imho, underestimates what linguistics can do for this field of study- and stubbornly saying that only linguistic knowledge will lead to an improvement of the results -which, imho, means underestimating the outstanding work on deep learning techniques done so far. As you said, creating high-quality input is likely to be costly -and I'm not talking about syntax tree creation tooling only here. But, isn't it worth the while? I believe it is.
hcorreasuarez 8y ago
As a linguist, I see what you mean. However, I disagree on a couple of things. First, I wouldn’t say “n-grams and understanding syntax trees and all the interconnections possible inside them” will make your NLP skills or results any better. Understanding n-grams and syntax trees should be quite easy for linguists, but that won’t do the trick. It turns out that tagging, lemmatizing, wsd, parsing, and many other fundamental tasks in NLP are not that “simple” after all. Of course, you can use libraries to do all of those tasks -and this is pretty simple, indeed,- but most libraries will end up making flagrant mistakes if texts -either spoken or written- get too complex. Second, real world texts are complex by default, so, for someone to “NLP the shit out of real world text/speech”, they will have to find creative solutions to make tools improve -and that can get pretty difficult. Once again, understanding n-grams and syntax trees is not enough. There’s one thing I totally agree with you on: tokenization, tagging, and the like might not be that impressive. Nonetheless, creative solutions to the problems underlying those tasks are, in fact, rare and impressive.
hiker512 8y ago
Sorry, but for a lot of situations n-grams just don't scale. There are just way too many combinations. There are a lot of people dealing with GBs and TBs of text.
- deleted 8y ago
  [deleted]
- autokad 8y ago
  > "There are just way too many combinations" truncated SVD
asdsa5325 8y ago
Nowadays, deep learning powers most of NLP.
- stochastic_monk 8y ago
  The state of the art is primarily hybrid models which use deep neural networks in combination with classical parsing algorithms. For example, the Stack LSTM method uses a form of arc-std with an LSTM and a stack of sorts.
  - riku_iki 8y ago
    And on which problem this approach achieved state of art results?
    - stochastic_monk 8y ago
      Dependency Parsing [1]. [1] https://arxiv.org/abs/1505.08075 https://arxiv.org/abs/1505.08075
      - riku_iki 8y ago
        I checked this paper briefly, and couldn't find any "classical parsing algorithms" there. Maybe you could point me on what do you mean?.. Also why do you think it is state of art? Is it in top results in any recent benchmarks (e.g. http://universaldependencies.org/conll17/results.html http://universaldependencies.org/conll17/results.html)?
        stochastic_monk 8y ago
        Arc-standard [1] is the classical algorithm in question. See section 3.2 of Dyer's paper: "Our parser is based on the arc-standard transition inventory (Nivre, 2004), given in Figure 3". Perhaps it's not still state of the art, but I don't see dependency parsing as a task in that suite of benchmarks, and NLP isn't my primary field, so I'm not up-to-date on the whether or not another method has outperformed it in the last three years. (I would be surprised if one hadn't.) At the time it was, and regardless, I don't doubt that the current state of the art similarly uses neural networks only as a subset of the complete method. [1] https://www.aclweb.org/anthology/W04-0308 https://www.aclweb.org/anthology/W04-0308 Edit: I can't seem to reply to the post below me. There isn't an entry CMU/Pittsburgh in your benchmark. The paper refers to the Stanford Dependency [SD] and Penn Chinese Treebank 5 [CTB5], and reports 90.9/85.7, respectively, for LAS scores. I'm not sure if any of the treebanks listed correspond to these, but these numbers do seem in line to at least be competitive regardless of which treebank is used.
mindcrime 8y ago
Obligatory: Every time I fire a linguist, the performance of our speech recognition system goes up. -- Fred Jelinek
- caio1982 8y ago
  Dunno why you were downvoted, I heard this one multiple times during my classes and always had a good laugh :-)

riku_iki 8y ago

> as is understanding syntax trees and all the interconnections possible inside them so you can NLP the shit out of real world text/speech But do modern DL approaches (e.g. SQuad, translation models) defy this approach? They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic..

matachuan 8y ago
That's how NLP can be a dead end by putting your faith on NN.
- riku_iki 8y ago
  So far this dead end outperforms traditional ways though.
  - matachuan 8y ago
    Lol for sure any NN-based approach can claim to outperform traditional ways while half of CVPR paper results cannot be reproduced.
    - posterboy 8y ago
      From reading papers on WaveNet and AlphaGo, the problem seems to be incomplete information in the papers, but I am not going to argue with the popular results.
    - amirouche 8y ago
      CVPR?
      - mindcrime 8y ago
        Conference on Computer Vision and Pattern Recognition
    - riku_iki 8y ago
      > while half of CVPR paper results cannot be reproduced. even if just another half of results are proven to outperform traditional approaches, is it strong enough trend?
      - matachuan 8y ago
        You clearly did not see that was sarcastic.
ertand 8y ago
You could jointly train a NN to parse the sentence and tackle the NLP task you have at the same time, or instead of just using the tokens you could use a parser to add more features too. edit: clarification
yorwba 8y ago
> They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic. Sure, but that doesn't mean that knowing about syntax won't improve the result. If you're training a translation model on a huge database of labeled examples, it might discover syntactic relationships from scratch. But if you don't have so much data, you're probably better off using all the auxiliary information you can get.
- riku_iki 8y ago
  > But that doesn't mean that knowing about syntax won't improve the result this is likely correct, additional high quality input will likely improve performance, but creating such input for 200 modern human languages requires significant effort, much larger than allowing NN to solve this problem, that's why researchers invest this effort into NN improvement, not syntax tree creation tooling.
  - hcorreasuarez 8y ago
    All of us have to strike the right balance between stubbornly believing that NNs will solve all problems in NLP -which, imho, underestimates what linguistics can do for this field of study- and stubbornly saying that only linguistic knowledge will lead to an improvement of the results -which, imho, means underestimating the outstanding work on deep learning techniques done so far. As you said, creating high-quality input is likely to be costly -and I'm not talking about syntax tree creation tooling only here. But, isn't it worth the while? I believe it is.

hcorreasuarez 8y ago

As a linguist, I see what you mean. However, I disagree on a couple of things. First, I wouldn’t say “n-grams and understanding syntax trees and all the interconnections possible inside them” will make your NLP skills or results any better. Understanding n-grams and syntax trees should be quite easy for linguists, but that won’t do the trick. It turns out that tagging, lemmatizing, wsd, parsing, and many other fundamental tasks in NLP are not that “simple” after all. Of course, you can use libraries to do all of those tasks -and this is pretty simple, indeed,- but most libraries will end up making flagrant mistakes if texts -either spoken or written- get too complex. Second, real world texts are complex by default, so, for someone to “NLP the shit out of real world text/speech”, they will have to find creative solutions to make tools improve -and that can get pretty difficult. Once again, understanding n-grams and syntax trees is not enough. There’s one thing I totally agree with you on: tokenization, tagging, and the like might not be that impressive. Nonetheless, creative solutions to the problems underlying those tasks are, in fact, rare and impressive.

hiker512 8y ago

Sorry, but for a lot of situations n-grams just don't scale. There are just way too many combinations. There are a lot of people dealing with GBs and TBs of text.

deleted 8y ago
[deleted]
autokad 8y ago
> "There are just way too many combinations" truncated SVD

asdsa5325 8y ago

Nowadays, deep learning powers most of NLP.

stochastic_monk 8y ago
The state of the art is primarily hybrid models which use deep neural networks in combination with classical parsing algorithms. For example, the Stack LSTM method uses a form of arc-std with an LSTM and a stack of sorts.
- riku_iki 8y ago
  And on which problem this approach achieved state of art results?
  - stochastic_monk 8y ago
    Dependency Parsing [1]. [1] https://arxiv.org/abs/1505.08075 https://arxiv.org/abs/1505.08075
    - riku_iki 8y ago
      I checked this paper briefly, and couldn't find any "classical parsing algorithms" there. Maybe you could point me on what do you mean?.. Also why do you think it is state of art? Is it in top results in any recent benchmarks (e.g. http://universaldependencies.org/conll17/results.html http://universaldependencies.org/conll17/results.html)?
      - stochastic_monk 8y ago
        Arc-standard [1] is the classical algorithm in question. See section 3.2 of Dyer's paper: "Our parser is based on the arc-standard transition inventory (Nivre, 2004), given in Figure 3". Perhaps it's not still state of the art, but I don't see dependency parsing as a task in that suite of benchmarks, and NLP isn't my primary field, so I'm not up-to-date on the whether or not another method has outperformed it in the last three years. (I would be surprised if one hadn't.) At the time it was, and regardless, I don't doubt that the current state of the art similarly uses neural networks only as a subset of the complete method. [1] https://www.aclweb.org/anthology/W04-0308 https://www.aclweb.org/anthology/W04-0308 Edit: I can't seem to reply to the post below me. There isn't an entry CMU/Pittsburgh in your benchmark. The paper refers to the Stanford Dependency [SD] and Penn Chinese Treebank 5 [CTB5], and reports 90.9/85.7, respectively, for LAS scores. I'm not sure if any of the treebanks listed correspond to these, but these numbers do seem in line to at least be competitive regardless of which treebank is used.

mindcrime 8y ago

Obligatory: Every time I fire a linguist, the performance of our speech recognition system goes up. -- Fred Jelinek

caio1982 8y ago
Dunno why you were downvoted, I heard this one multiple times during my classes and always had a good laugh :-)