3 ms·

Recursive neural networks have not seen wide adoption in the deep learning community. To my limited understanding, this is because their performance depends on

by vonnik 10y ago

Recursive neural networks have not seen wide adoption in the deep learning community. To my limited understanding, this is because their performance depends on the quality of the parser you use, which makes them fragile. Has anyone had experience using them?

Eridrus 10y ago
Not really my area, but RNN variants such as LSTMs/GRUs were (are?) the state of the art in NLP tasks.
- mrdrozdov 10y ago
  Recursive Neural Networks are not the same as Recurrent Neural Networks (RNNs).
  - FreakLegion 10y ago
    The repo uses RNN for Recursive Neural Network and specifically calls attention to its LSTM implementation.
    - mrdrozdov 10y ago
      Thanks for pointing this out. The same acronym is used to refer to both versions. I like the description of Recursive Neural Nets versus Recurrent Neural Nets in this Stack Overflow post [1]. > A recursive network is just a generalization of a recurrent network. In a recurrent network the weights are shared (and dimensionality remains constant) along the length of the sequence because how would you deal with position-dependent weights when you encounter a sequence at test-time of different length to any you saw at train-time. In a recursive network the weights are shared (and dimensionality remains constant) at every node for the same reason. In the TreeRNN (Tree Recursive Neural Nets) Github project and associated paper [2], there is indeed an implementation of a TreeLSTM which is a recursive LSTM inspired by the recurrent version. [1]: http://stats.stackexchange.com/a/158995/92040 http://stats.stackexchange.com/a/158995/92040 [2]: http://www.aclweb.org/anthology/P15-1150 http://www.aclweb.org/anthology/P15-1150
bertomartin 10y ago
Parsers are pretty good and getting better, have you seen: https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html https://research.googleblog.com/2016/05/announcing-syntaxnet... This is a very successful application of recursive NNs in Question/Answering domain: https://cs.umd.edu/~miyyer/qblearn/ https://cs.umd.edu/~miyyer/qblearn/

by vonnik 10y ago

Eridrus 10y ago

Not really my area, but RNN variants such as LSTMs/GRUs were (are?) the state of the art in NLP tasks.

mrdrozdov 10y ago
Recursive Neural Networks are not the same as Recurrent Neural Networks (RNNs).
- FreakLegion 10y ago
  The repo uses RNN for Recursive Neural Network and specifically calls attention to its LSTM implementation.
  - mrdrozdov 10y ago
    Thanks for pointing this out. The same acronym is used to refer to both versions. I like the description of Recursive Neural Nets versus Recurrent Neural Nets in this Stack Overflow post [1]. > A recursive network is just a generalization of a recurrent network. In a recurrent network the weights are shared (and dimensionality remains constant) along the length of the sequence because how would you deal with position-dependent weights when you encounter a sequence at test-time of different length to any you saw at train-time. In a recursive network the weights are shared (and dimensionality remains constant) at every node for the same reason. In the TreeRNN (Tree Recursive Neural Nets) Github project and associated paper [2], there is indeed an implementation of a TreeLSTM which is a recursive LSTM inspired by the recurrent version. [1]: http://stats.stackexchange.com/a/158995/92040 http://stats.stackexchange.com/a/158995/92040 [2]: http://www.aclweb.org/anthology/P15-1150 http://www.aclweb.org/anthology/P15-1150

bertomartin 10y ago

Parsers are pretty good and getting better, have you seen: https://research.googleblog.com/2016/05/announcing-syntaxnet-worlds-most.html https://research.googleblog.com/2016/05/announcing-syntaxnet... This is a very successful application of recursive NNs in Question/Answering domain: https://cs.umd.edu/~miyyer/qblearn/ https://cs.umd.edu/~miyyer/qblearn/