4 ms·
I hate the format of the Nature paper. I have no idea how the algorithm is different than their neural turing machine
by jostmey 10y ago
I hate the format of the Nature paper. I have no idea how the algorithm is different than their neural turing machine
- hidden-markov 10y agoNeural Turing machine is a Turing machine. This one is a differentiable random access memory machine.
- pavledjo 10y agoIIRC, the difference is an additional read head. Also they removed location-based addressing for the read/write heads in favor of a combination of content-based adressing and a dynamic scheme to figure out empty locations.
- zardo 10y agoCould you expand on the removal of location based addressing? That's not how I understood it, but... I don't understand it very well.
- albertzeyer 10y agoOne extension is that the model can dynamically grow/reduce the external augmented memory.
- kekeblom 10y agoThe DNC has a similar structure as the neural turing machine. It has a controller network and a memory matrix with read and write heads. However, the DNC differs in it’s memory accessing mechanisms. Firstly, the DNC has a memory allocation mechanism ensuring that the network can write to memory that is not being used. Secondly, it also contains free gates that allow the network to explicitly forget data stored in it’s memory. Thirdly, the DNC employs a temporal link matrix which tracks the order in which writes are made. The temporal link matrix allows the location based addressing mechanism to recover from a jump caused by a content based memory access.