4 ms·

Key-Value Memory Networks for Directly Reading Documents

41 points by shagunsodhani 10y ago

fpgaminer 10y ago
It is my understanding, from a sparse reading of these kinds of networks, that at a high level they basically just add memory units to the networking, and then the "normal" fully connected/convolution parts consume and control those memory units like any other input/output. So the "normal" parts of the network have to learn how to use the memory facilities that they've been provided. Much like how LSTMs are just fully connected networks with some storage values that they can gate, and thus learn memory by learning proper gating. It makes me wonder ... do human brains also have to learn how to use memories? I mean that merely as a curious question, not as a means to inform biology or vise versa. We have to learn how to use all our senses and all our muscles, so perhaps it makes sense that our brains also have to learn how to use long term memory facilities. In fact, there's the age of amnesia, which is around 6, after which the majority of people forget most of their long term memories up to that point. Perhaps that's an indicator that our brains do have to learn how to store long term memory, and it takes 6 whole years to fully learn. Or maybe it's a side effect of some biological process; the hippocampus maybe changes at age 6 or something. Just an interesting thought that we take memory as a de-facto feature of the brain, but perhaps it's learned like everything else and perhaps its efficacy is affected by how we learn to use it at a young age.
- landhar 10y ago
  Could you provide a reference for this "age of amnesia" ? I've heard many people claim that it's not possible to remember things before age 6. But I personally have lots of memories from ages 2 to 4 (these memories are very easy for me to accurately date, because I lived in different countries during these years). And I would definitely categorize those memories as "long term": I remember places, visits of relatives, toys I had, play dates and birthday celebrations, bike rides, etc...
  - aab0 10y ago
    https://en.wikipedia.org/wiki/Childhood_amnesia https://en.wikipedia.org/wiki/Childhood_amnesia
  - eximius 10y ago
    I think the age varies. But you'd be hard pressed to find someone who could recall pre-toddler. My earliest are at 2-5, for reference. (placed by location, but there are probably some earlier in that range)
  - thaumasiotes 10y ago
    Well, the description of the "age of amnesia" was this: > In fact, there's the age of amnesia, which is around 6, after which the majority of people forget most of their long term memories up to that point. Under that definition, I think it's safe to say that all ages are "the age of amnesia". Most of your memories, no matter when they're from, get forgotten or irretrievably buried.

41 points by shagunsodhani 10y ago

It is my understanding, from a sparse reading of these kinds of networks, that at a high level they basically just add memory units to the networking, and then the "normal" fully connected/convolution parts consume and control those memory units like any other input/output. So the "normal" parts of the network have to learn how to use the memory facilities that they've been provided. Much like how LSTMs are just fully connected networks with some storage values that they can gate, and thus learn memory by learning proper gating. It makes me wonder ... do human brains also have to learn how to use memories? I mean that merely as a curious question, not as a means to inform biology or vise versa. We have to learn how to use all our senses and all our muscles, so perhaps it makes sense that our brains also have to learn how to use long term memory facilities. In fact, there's the age of amnesia, which is around 6, after which the majority of people forget most of their long term memories up to that point. Perhaps that's an indicator that our brains do have to learn how to store long term memory, and it takes 6 whole years to fully learn. Or maybe it's a side effect of some biological process; the hippocampus maybe changes at age 6 or something. Just an interesting thought that we take memory as a de-facto feature of the brain, but perhaps it's learned like everything else and perhaps its efficacy is affected by how we learn to use it at a young age.

landhar 10y ago
Could you provide a reference for this "age of amnesia" ? I've heard many people claim that it's not possible to remember things before age 6. But I personally have lots of memories from ages 2 to 4 (these memories are very easy for me to accurately date, because I lived in different countries during these years). And I would definitely categorize those memories as "long term": I remember places, visits of relatives, toys I had, play dates and birthday celebrations, bike rides, etc...
- aab0 10y ago
  https://en.wikipedia.org/wiki/Childhood_amnesia https://en.wikipedia.org/wiki/Childhood_amnesia
- eximius 10y ago
  I think the age varies. But you'd be hard pressed to find someone who could recall pre-toddler. My earliest are at 2-5, for reference. (placed by location, but there are probably some earlier in that range)
- thaumasiotes 10y ago
  Well, the description of the "age of amnesia" was this: > In fact, there's the age of amnesia, which is around 6, after which the majority of people forget most of their long term memories up to that point. Under that definition, I think it's safe to say that all ages are "the age of amnesia". Most of your memories, no matter when they're from, get forgotten or irretrievably buried.