9 ms·

A TensorFlow implementation of the Differentiable Neural Computer

157 points by hurrycane 9y ago

jostmey 9y ago
I hate the format of the Nature paper. I have no idea how the algorithm is different than their neural turing machine
- hidden-markov 9y ago
  Neural Turing machine is a Turing machine. This one is a differentiable random access memory machine.
  - Aqua 9y ago
    :DDD
    - vog 9y ago
      Care to elaborate?
- pavledjo 9y ago
  IIRC, the difference is an additional read head. Also they removed location-based addressing for the read/write heads in favor of a combination of content-based adressing and a dynamic scheme to figure out empty locations.
  - zardo 9y ago
    Could you expand on the removal of location based addressing? That's not how I understood it, but... I don't understand it very well.
- albertzeyer 9y ago
  One extension is that the model can dynamically grow/reduce the external augmented memory.
- kekeblom 9y ago
  The DNC has a similar structure as the neural turing machine. It has a controller network and a memory matrix with read and write heads. However, the DNC differs in it’s memory accessing mechanisms. Firstly, the DNC has a memory allocation mechanism ensuring that the network can write to memory that is not being used. Secondly, it also contains free gates that allow the network to explicitly forget data stored in it’s memory. Thirdly, the DNC employs a temporal link matrix which tracks the order in which writes are made. The temporal link matrix allows the location based addressing mechanism to recover from a jump caused by a content based memory access.
crawfordcomeaux 9y ago
I can't wait to start playing with this. The DNC is a big part of the initial inspiration for how I started modeling and reprogramming my brain! A key difference between one of my models for how our brain works and the DNC: there's a mechanism for loading/unloading neural networks into the DNC. Also, the inputs = signals from the brain + body + mind + environment (physical & mental) = the outputs (though some outputs require iteratively applying the inputs a good bit before manifesting).
- trendia 9y ago
  > I started modeling and reprogramming my brain! How did you model and reprogram your brain?
  - NegatioN 9y ago
    Seems like we got a time traveler on our hands.
    - crawfordcomeaux 9y ago
      If very intentionally programming our brains becomes a social norm in the future, which I believe it will, then I've got no problem with this characterization.
  - crawfordcomeaux 9y ago
    I've developed a systematic framework for generating practices to change how my body, mind, and brain work. I haven't figured out a way to explain it in simple terms, but working toward it. EDIT: Here are some basic thoughts... Everything we experience flows through our perceptual sets (composites of relevant emotions, beliefs, behavioral strategies, and intentions), which we can intentionally craft over time. Each component in a perceptual set is processed with varying degrees of attention. So to program your brain, the first step is to find a functional base line perceptual set to work from & learn how to direct/focus your attention on the different aspects of your perception. To put it even more simply: go practice all the forms of Buddhism. All I seem to be doing at this point is systematically rediscovering stuff they've known for 2500 years and attempting to put a categorical structure onto it.
zmgehlke 9y ago
So, I'll own up to it, I'm kinda dumb (or at least not read up on this). Can someone explain what a DNC is? Assume I know most CS undergrad topics -- or can at least google them; eg, I know what a Turing machine is but not what a neural Turing machine is.
- p1esk 9y ago
  It's a memory bank attached to a neural network.
- deepnet 9y ago
  Neural Turing Machines by Alex Graves, Greg Wayne, Ivo Danihelka https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401 A Neural Turing machine is a type of neural network with addressable memory - it learns rather than being explicity programmed. The net also learns to boot, i.e. how to read/ write it's own memory from scratch. Stochastic Gradient Descent learning relies on backpropagation of the error corrections to the individual weights to make a slight improvement - this requires a differentiable neural net. This version innovates with content addressable memory. The authors demonstrate learning from big data sets and with reinforcement learning, (i.e. trial and error as an embodied agent).
- westbrookt 9y ago
  I'll attempt to explain what I understand from the paper. This will be higher level, and abstracted because that's the best I can do as an undergraduate currently. The DNC essentially gives the neural network external memory. This memory is controlled by read heads, and write heads. These heads are actually just part of the vector output of the neural network. Each of them has a key vector, which is used to find where in memory to read or write. This is done by finding the most similar vector in memory, using cosine distance. The neural network can then use the memory. It also has mechanisms to remember the order that the memory vectors were written in, to allow for sequential recall. The network itself is trained by reinforcement learning generally. I'm not entirely sure what is done with the parts of the vectors that are not used at a given timestep, when only input and not output is needed. Again, this is what I believe I was able to understand from the paper. Please let me know if I got anything wrong.
deleted 9y ago
[deleted]
meri_dian 9y ago
The way in which the brain solves classification tasks - a reactionary process that doesn't draw on deliberate reasoning ability - seems similar to how we recreate such abilities in ANN's. So in that sense it seems our AI systems are following the path laid out for us by our own brains. However it seems to me (and I may be wrong) that the DNC tries to implement procedural, deliberate thinking in a way divergent from the method implemented by the human brain. The brain is clearly capable of providing us with the ability to navigate through complex procedure (like interpreting subway maps), something beyond the abilities of modern AI which mainly excels at more reactionary classification. So the question becomes, why is Deep Mind diverging from the way the brain works in the case of the DNC? Is it because we don't actually understand how the brain implements reasoning, or is it because in order to do so in the brain's own way it would exceed our technical capabilities? Or both?
- rhcom2 9y ago
  > The way in which the brain solves classification tasks - a reactionary process that doesn't draw on deliberate reasoning ability Does this mean that when I see an animal my brain reactivity just knows "cat" instead of going "well it a small thing with fur and whiskers, fits cat"? edit: A longer description I found helpful: https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/
  - meri_dian 9y ago
    I'd say that the classification of a cat as a cat is a fundamentally different process from, say, solving a complex math problem. The main difference seems to be that the former is a subconscious process and the latter is conscious, but it's more than that because to solve the complex problem we're relying on explicit selective recall from our long term memory then transferring that knowledge to our short term memory for conscious processing. I don't think this transfer of information from long term to short term memory occurs when looking at a cat.
    - jacquesm 9y ago

157 points by hurrycane 9y ago

jostmey 9y ago

I hate the format of the Nature paper. I have no idea how the algorithm is different than their neural turing machine

hidden-markov 9y ago
Neural Turing machine is a Turing machine. This one is a differentiable random access memory machine.
- Aqua 9y ago
  :DDD
  - vog 9y ago
    Care to elaborate?
pavledjo 9y ago
IIRC, the difference is an additional read head. Also they removed location-based addressing for the read/write heads in favor of a combination of content-based adressing and a dynamic scheme to figure out empty locations.
- zardo 9y ago
  Could you expand on the removal of location based addressing? That's not how I understood it, but... I don't understand it very well.
albertzeyer 9y ago
One extension is that the model can dynamically grow/reduce the external augmented memory.
kekeblom 9y ago
The DNC has a similar structure as the neural turing machine. It has a controller network and a memory matrix with read and write heads. However, the DNC differs in it’s memory accessing mechanisms. Firstly, the DNC has a memory allocation mechanism ensuring that the network can write to memory that is not being used. Secondly, it also contains free gates that allow the network to explicitly forget data stored in it’s memory. Thirdly, the DNC employs a temporal link matrix which tracks the order in which writes are made. The temporal link matrix allows the location based addressing mechanism to recover from a jump caused by a content based memory access.

crawfordcomeaux 9y ago

I can't wait to start playing with this. The DNC is a big part of the initial inspiration for how I started modeling and reprogramming my brain! A key difference between one of my models for how our brain works and the DNC: there's a mechanism for loading/unloading neural networks into the DNC. Also, the inputs = signals from the brain + body + mind + environment (physical & mental) = the outputs (though some outputs require iteratively applying the inputs a good bit before manifesting).

trendia 9y ago
> I started modeling and reprogramming my brain! How did you model and reprogram your brain?
- NegatioN 9y ago
  Seems like we got a time traveler on our hands.
  - crawfordcomeaux 9y ago
    If very intentionally programming our brains becomes a social norm in the future, which I believe it will, then I've got no problem with this characterization.
- crawfordcomeaux 9y ago
  I've developed a systematic framework for generating practices to change how my body, mind, and brain work. I haven't figured out a way to explain it in simple terms, but working toward it. EDIT: Here are some basic thoughts... Everything we experience flows through our perceptual sets (composites of relevant emotions, beliefs, behavioral strategies, and intentions), which we can intentionally craft over time. Each component in a perceptual set is processed with varying degrees of attention. So to program your brain, the first step is to find a functional base line perceptual set to work from & learn how to direct/focus your attention on the different aspects of your perception. To put it even more simply: go practice all the forms of Buddhism. All I seem to be doing at this point is systematically rediscovering stuff they've known for 2500 years and attempting to put a categorical structure onto it.

zmgehlke 9y ago

So, I'll own up to it, I'm kinda dumb (or at least not read up on this). Can someone explain what a DNC is? Assume I know most CS undergrad topics -- or can at least google them; eg, I know what a Turing machine is but not what a neural Turing machine is.

p1esk 9y ago
It's a memory bank attached to a neural network.
deepnet 9y ago
Neural Turing Machines by Alex Graves, Greg Wayne, Ivo Danihelka https://arxiv.org/abs/1410.5401 https://arxiv.org/abs/1410.5401 A Neural Turing machine is a type of neural network with addressable memory - it learns rather than being explicity programmed. The net also learns to boot, i.e. how to read/ write it's own memory from scratch. Stochastic Gradient Descent learning relies on backpropagation of the error corrections to the individual weights to make a slight improvement - this requires a differentiable neural net. This version innovates with content addressable memory. The authors demonstrate learning from big data sets and with reinforcement learning, (i.e. trial and error as an embodied agent).
westbrookt 9y ago
I'll attempt to explain what I understand from the paper. This will be higher level, and abstracted because that's the best I can do as an undergraduate currently. The DNC essentially gives the neural network external memory. This memory is controlled by read heads, and write heads. These heads are actually just part of the vector output of the neural network. Each of them has a key vector, which is used to find where in memory to read or write. This is done by finding the most similar vector in memory, using cosine distance. The neural network can then use the memory. It also has mechanisms to remember the order that the memory vectors were written in, to allow for sequential recall. The network itself is trained by reinforcement learning generally. I'm not entirely sure what is done with the parts of the vectors that are not used at a given timestep, when only input and not output is needed. Again, this is what I believe I was able to understand from the paper. Please let me know if I got anything wrong.

deleted 9y ago

[deleted]

meri_dian 9y ago

The way in which the brain solves classification tasks - a reactionary process that doesn't draw on deliberate reasoning ability - seems similar to how we recreate such abilities in ANN's. So in that sense it seems our AI systems are following the path laid out for us by our own brains. However it seems to me (and I may be wrong) that the DNC tries to implement procedural, deliberate thinking in a way divergent from the method implemented by the human brain. The brain is clearly capable of providing us with the ability to navigate through complex procedure (like interpreting subway maps), something beyond the abilities of modern AI which mainly excels at more reactionary classification. So the question becomes, why is Deep Mind diverging from the way the brain works in the case of the DNC? Is it because we don't actually understand how the brain implements reasoning, or is it because in order to do so in the brain's own way it would exceed our technical capabilities? Or both?

rhcom2 9y ago
> The way in which the brain solves classification tasks - a reactionary process that doesn't draw on deliberate reasoning ability Does this mean that when I see an animal my brain reactivity just knows "cat" instead of going "well it a small thing with fur and whiskers, fits cat"? edit: A longer description I found helpful: https://deepmind.com/blog/differentiable-neural-computers/ https://deepmind.com/blog/differentiable-neural-computers/
- meri_dian 9y ago
  I'd say that the classification of a cat as a cat is a fundamentally different process from, say, solving a complex math problem. The main difference seems to be that the former is a subconscious process and the latter is conscious, but it's more than that because to solve the complex problem we're relying on explicit selective recall from our long term memory then transferring that knowledge to our short term memory for conscious processing. I don't think this transfer of information from long term to short term memory occurs when looking at a cat.
  - jacquesm 9y ago