5 ms·

> as is understanding syntax trees and all the interconnections possible inside them so you can NLP the shit out of real world text/speech But do modern DL app

> as is understanding syntax trees and all the interconnections possible inside them so you can NLP the shit out of real world text/speech But do modern DL approaches (e.g. SQuad, translation models) defy this approach? They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic..

matachuan 8y ago
That's how NLP can be a dead end by putting your faith on NN.
- riku_iki 8y ago
  So far this dead end outperforms traditional ways though.
  - matachuan 8y ago
    Lol for sure any NN-based approach can claim to outperform traditional ways while half of CVPR paper results cannot be reproduced.
    - posterboy 8y ago
      From reading papers on WaveNet and AlphaGo, the problem seems to be incomplete information in the papers, but I am not going to argue with the popular results.
    - amirouche 8y ago
      CVPR?
      - mindcrime 8y ago
        Conference on Computer Vision and Pattern Recognition
    - riku_iki 8y ago
      > while half of CVPR paper results cannot be reproduced. even if just another half of results are proven to outperform traditional approaches, is it strong enough trend?
      - matachuan 8y ago
        You clearly did not see that was sarcastic.
ertand 8y ago
You could jointly train a NN to parse the sentence and tackle the NLP task you have at the same time, or instead of just using the tokens you could use a parser to add more features too. edit: clarification
yorwba 8y ago
> They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic. Sure, but that doesn't mean that knowing about syntax won't improve the result. If you're training a translation model on a huge database of labeled examples, it might discover syntactic relationships from scratch. But if you don't have so much data, you're probably better off using all the auxiliary information you can get.
- riku_iki 8y ago
  > But that doesn't mean that knowing about syntax won't improve the result this is likely correct, additional high quality input will likely improve performance, but creating such input for 200 modern human languages requires significant effort, much larger than allowing NN to solve this problem, that's why researchers invest this effort into NN improvement, not syntax tree creation tooling.
  - hcorreasuarez 8y ago
    All of us have to strike the right balance between stubbornly believing that NNs will solve all problems in NLP -which, imho, underestimates what linguistics can do for this field of study- and stubbornly saying that only linguistic knowledge will lead to an improvement of the results -which, imho, means underestimating the outstanding work on deep learning techniques done so far. As you said, creating high-quality input is likely to be costly -and I'm not talking about syntax tree creation tooling only here. But, isn't it worth the while? I believe it is.

matachuan 8y ago

That's how NLP can be a dead end by putting your faith on NN.

riku_iki 8y ago
So far this dead end outperforms traditional ways though.
- matachuan 8y ago
  Lol for sure any NN-based approach can claim to outperform traditional ways while half of CVPR paper results cannot be reproduced.
  - posterboy 8y ago
    From reading papers on WaveNet and AlphaGo, the problem seems to be incomplete information in the papers, but I am not going to argue with the popular results.
  - amirouche 8y ago
    CVPR?
    - mindcrime 8y ago
      Conference on Computer Vision and Pattern Recognition
  - riku_iki 8y ago
    > while half of CVPR paper results cannot be reproduced. even if just another half of results are proven to outperform traditional approaches, is it strong enough trend?
    - matachuan 8y ago
      You clearly did not see that was sarcastic.

ertand 8y ago

You could jointly train a NN to parse the sentence and tackle the NLP task you have at the same time, or instead of just using the tokens you could use a parser to add more features too. edit: clarification

yorwba 8y ago

> They train DL models on labeled data without knowing anything about syntax trees, and allow NN do all the magic. Sure, but that doesn't mean that knowing about syntax won't improve the result. If you're training a translation model on a huge database of labeled examples, it might discover syntactic relationships from scratch. But if you don't have so much data, you're probably better off using all the auxiliary information you can get.

riku_iki 8y ago
> But that doesn't mean that knowing about syntax won't improve the result this is likely correct, additional high quality input will likely improve performance, but creating such input for 200 modern human languages requires significant effort, much larger than allowing NN to solve this problem, that's why researchers invest this effort into NN improvement, not syntax tree creation tooling.
- hcorreasuarez 8y ago
  All of us have to strike the right balance between stubbornly believing that NNs will solve all problems in NLP -which, imho, underestimates what linguistics can do for this field of study- and stubbornly saying that only linguistic knowledge will lead to an improvement of the results -which, imho, means underestimating the outstanding work on deep learning techniques done so far. As you said, creating high-quality input is likely to be costly -and I'm not talking about syntax tree creation tooling only here. But, isn't it worth the while? I believe it is.