3 ms·
I would like to add for those of you not familiar: The Neural Turing Machine method uses a neural network called a recurrent neural net. Recurrent neural nets a
by agibsonccc 12y ago
I would like to add for those of you not familiar: The Neural Turing Machine method uses a neural network called a recurrent neural net. Recurrent neural nets are used in modeling time series data and have a neat concept of training called back propagation through time.
Here's a neat tutorial with an RBM (typically a feed forward net) as a recurrent net for those who want to just see what a recurrent net "looks like"
http://deeplearning.net/tutorial/rnnrbm.html http://deeplearning.net/tutorial/rnnrbm.html
- shawntan 12y agoNitpicking here, but while the authors do use a recurrent neural net (RNN), they do not use it exclusively. The system consists of a memory element, and a controller element. In their evaluation of the system, they use both a standard feed-forward network, as well as an RNN with long short-term memory (LSTM) units as the controller element. In certain tasks, the feed-forward network works better. +1 on the deeplearning.net tutorials, and theano. I've learnt a lot from there.
- agibsonccc 12y agoRight. Mainly just low hanging fruit for those who aren't in this stuff day to day. In a lot of my talks and day to day conversations, I've found people don't know the difference between a feed forward architecture vs, recurrent, vs recursive vs,...you get the point :P