4 ms·

To me, it much more closely resembles a key-value retrieval system. I wonder if we could learn anything about that by training an artificial neural net to perfo

To me, it much more closely resembles a key-value retrieval system. I wonder if we could learn anything about that by training an artificial neural net to perform key-value storage?

BurningFrog 7y ago
Yeah, my brain can prolly be 80% modeled as a Redis fork...
PureParadigm 7y ago
I think it's more like a graph with edges between associated bits of information. It's easy to remember things in order, but random access is hard unless you've established links for it. For example, if you memorized how to play a song, each part links to the next but harder to ask someone what is the n-th note in a song, even if they can play the song. It would explain why making associations is a useful memory technique. Traversing this graph when you aren't near what you're looking for can be hard ("it's on the tip of my tongue").
- grawprog 7y ago
  Not only that, but sometimes as happened to me recently with my own song, you completely forget how to play it until you hear it and start playing it, then suddenly you remember every little note and nuance you do did, but only while you're playing it and good luck explaining to someone what you're doing even as you're doing it.
infradig 7y ago
Content addressable memory more like.
uomopertica 7y ago
Machine Learning phd student here! I am currently working on a branch of ML called Continual Learning that deals with this theme exactly. It turns out Artificial Neural Networks have a hard time recalling things they they see in the past [1]. One of the most effective strategies to counter this issue is storing and periodically replaying old training items (i.e. “memories”) for the network [2]. There are also approaches that set out to mimic what we understand of the brain and adopt cue-based approaches [3], but they don’t seem to have reached meaningful outbreaks just yet... [1] https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference [2] https://arxiv.org/abs/1902.10486 https://arxiv.org/abs/1902.10486 [3] https://arxiv.org/abs/1711.10563 https://arxiv.org/abs/1711.10563
rolltiide 7y ago
I’ve been saying for years that it’s like a hashtable, where the key itself is hashed in a predictable quickly retrievable pattern or namespace This article does not invalidate the idea
Deeplybrassic 7y ago
It's odd, because it very much resembles relational memory for me. I can summon no memory just by it's name, only from it's association. The space for sole base values is very limited. The plus side is that i can generally remember every caveat and detail there is to doing a thing once I've seen it once.
bra-ket 7y ago
Yes, this idea is called sparse distributed memory https://en.m.wikipedia.org/wiki/Sparse_distributed_memory https://en.m.wikipedia.org/wiki/Sparse_distributed_memory

To me, it much more closely resembles a key-value retrieval system. I wonder if we could learn anything about that by training an artificial neural net to perform key-value storage?

BurningFrog 7y ago

Yeah, my brain can prolly be 80% modeled as a Redis fork...

PureParadigm 7y ago

I think it's more like a graph with edges between associated bits of information. It's easy to remember things in order, but random access is hard unless you've established links for it. For example, if you memorized how to play a song, each part links to the next but harder to ask someone what is the n-th note in a song, even if they can play the song. It would explain why making associations is a useful memory technique. Traversing this graph when you aren't near what you're looking for can be hard ("it's on the tip of my tongue").

grawprog 7y ago
Not only that, but sometimes as happened to me recently with my own song, you completely forget how to play it until you hear it and start playing it, then suddenly you remember every little note and nuance you do did, but only while you're playing it and good luck explaining to someone what you're doing even as you're doing it.

infradig 7y ago

Content addressable memory more like.

uomopertica 7y ago

Machine Learning phd student here! I am currently working on a branch of ML called Continual Learning that deals with this theme exactly. It turns out Artificial Neural Networks have a hard time recalling things they they see in the past [1]. One of the most effective strategies to counter this issue is storing and periodically replaying old training items (i.e. “memories”) for the network [2]. There are also approaches that set out to mimic what we understand of the brain and adopt cue-based approaches [3], but they don’t seem to have reached meaningful outbreaks just yet... [1] https://en.wikipedia.org/wiki/Catastrophic_interference https://en.wikipedia.org/wiki/Catastrophic_interference [2] https://arxiv.org/abs/1902.10486 https://arxiv.org/abs/1902.10486 [3] https://arxiv.org/abs/1711.10563 https://arxiv.org/abs/1711.10563

rolltiide 7y ago

I’ve been saying for years that it’s like a hashtable, where the key itself is hashed in a predictable quickly retrievable pattern or namespace This article does not invalidate the idea

Deeplybrassic 7y ago

It's odd, because it very much resembles relational memory for me. I can summon no memory just by it's name, only from it's association. The space for sole base values is very limited. The plus side is that i can generally remember every caveat and detail there is to doing a thing once I've seen it once.

bra-ket 7y ago

Yes, this idea is called sparse distributed memory https://en.m.wikipedia.org/wiki/Sparse_distributed_memory https://en.m.wikipedia.org/wiki/Sparse_distributed_memory