4 ms·
PyTorch is also much faster than Tensorflow on LSTMs http://deeplearningathome.com/2017/06/PyTorch-vs-Tensorflow-lstm-language-model.html http://deeplearningath
by option 9y ago
PyTorch is also much faster than Tensorflow on LSTMs http://deeplearningathome.com/2017/06/PyTorch-vs-Tensorflow-lstm-language-model.html http://deeplearningathome.com/2017/06/PyTorch-vs-Tensorflow-.... There is also a reason for that
- 1024core 9y ago> There is also a reason for that Don't be shy. Do share what that reason is.
- IanCal 9y agoFrom the linked article, to help others like me who mostly scan the comments > PyTorch LSTM network is faster because, by default, it uses cuRNN’s LSTM implementation which fuses layers, steps and point-wise operations. See blog-post on this here. > Tensorflow’s RNNs (in r1.2), by default, do not use cuDNN’s RNN, and their ‘call’ function describes only one time-step of computation, hence a lot of optimization opportunities are lost. On the flip side, though, this gives user much more flexibility, provided that the user knows what he is doing.
- deleted 9y ago[deleted]
- sdenton4 9y agoYeah, tf rnn's are slow. If you look at the benchmark results here, they're all comparable except for the lstm examples. There are 'fused' implementations in tf, but they aren't the default, and I haven't tried them out yet...