3 ms·
This hits the sweet spot. It's more flexible than a DSL like Caffe which gets in the way as soon as you stray even a little bit, without requiring as much boile
by murbard2 12y ago
This hits the sweet spot. It's more flexible than a DSL like Caffe which gets in the way as soon as you stray even a little bit, without requiring as much boilerplate as a generic framework like Theano.
I have a question:
When training LSTM, how do you handle the gradient calculation?
I believe the original approach is to do a single step of BPTT for all the weights outside the unit, and do RTRL for the weights inside each unit. Is that what you've implemented?
- fchollet 12y agoIf you are familiar with Theano, you can check out the LSTM unit implementation here: https://github.com/fchollet/keras/blob/master/keras/layers/recurrent.py#L252 https://github.com/fchollet/keras/blob/master/keras/layers/r... Each input is propagated through the unit (all timesteps) before the gradient is computed. No RTRL. It works well in practice.