5 ms·

Don’t Forget About Associative Memories

73 points by headalgorithm 6y ago

deleted 6y ago
[deleted]
bra-ket 6y ago
Great review, thanks for this! Associative memory is at the core of cognition machinery [0], I wish there was more attention to it, but it seems like memory modeling gathered quite a bit of dust since the 90s. All the resources went to learning algorithms and ANNs. In the human brain learning is just memory encoding, and that encoding is not an iterative trial and error process as conventional ANN model forces you to believe. We have made a bit of progress since Mccullough & Pitts, and Rosenblatt work. But the field is mostly stuck there in the 1940s-60s from the neuro- and cognitive modelling perspective. I've added most of the content on SDM page [1], you can find many related developments there. Also see [2] for some old and newer memory models. [0] http://www.columbia.edu/~nvg1/Wickelgren/papers/1981aWAW.pdf http://www.columbia.edu/~nvg1/Wickelgren/papers/1981aWAW.pdf [1] https://en.wikipedia.org/wiki/Sparse_distributed_memory https://en.wikipedia.org/wiki/Sparse_distributed_memory [2] https://en.wikipedia.org/wiki/Cognitive_architecture https://en.wikipedia.org/wiki/Cognitive_architecture
sideshowb 6y ago
I just lost the game
webmaven 6y ago
Was intrigued by the reference in the morphological network section to the Ritter & Sussner group having disappeared. A little digging shows that the group's site is indeed gone, but digging further it just looks like Ritter has retired (no recent papers, listed as emeritus on www.cise.ufl.edu site), but that Sussner is still active and has recent papers on the topic (as well as ones that are fuzzy math related) such as https://www.researchgate.net/publication/338045815_Extreme_learning_machine_for_a_new_hybrid_morphologicallinear_perceptron https://www.researchgate.net/publication/338045815_Extreme_l...
carapace 6y ago
See also "Holographic Reduced Representation" https://web.stanford.edu/group/cslipublications/cslipublications/site/1575864304.shtml https://web.stanford.edu/group/cslipublications/cslipublicat... http://www.cogsci.ucsd.edu/~sereno/170/readings/06-Holographic.pdf http://www.cogsci.ucsd.edu/~sereno/170/readings/06-Holograph... https://www.ijcai.org/Proceedings/91-1/Papers/006.pdf https://www.ijcai.org/Proceedings/91-1/Papers/006.pdf

73 points by headalgorithm 6y ago

deleted 6y ago

[deleted]

Great review, thanks for this! Associative memory is at the core of cognition machinery [0], I wish there was more attention to it, but it seems like memory modeling gathered quite a bit of dust since the 90s. All the resources went to learning algorithms and ANNs. In the human brain learning is just memory encoding, and that encoding is not an iterative trial and error process as conventional ANN model forces you to believe. We have made a bit of progress since Mccullough & Pitts, and Rosenblatt work. But the field is mostly stuck there in the 1940s-60s from the neuro- and cognitive modelling perspective. I've added most of the content on SDM page [1], you can find many related developments there. Also see [2] for some old and newer memory models. [0] http://www.columbia.edu/~nvg1/Wickelgren/papers/1981aWAW.pdf http://www.columbia.edu/~nvg1/Wickelgren/papers/1981aWAW.pdf [1] https://en.wikipedia.org/wiki/Sparse_distributed_memory https://en.wikipedia.org/wiki/Sparse_distributed_memory [2] https://en.wikipedia.org/wiki/Cognitive_architecture https://en.wikipedia.org/wiki/Cognitive_architecture

sideshowb 6y ago

I just lost the game

webmaven 6y ago

Was intrigued by the reference in the morphological network section to the Ritter & Sussner group having disappeared. A little digging shows that the group's site is indeed gone, but digging further it just looks like Ritter has retired (no recent papers, listed as emeritus on www.cise.ufl.edu site), but that Sussner is still active and has recent papers on the topic (as well as ones that are fuzzy math related) such as https://www.researchgate.net/publication/338045815_Extreme_learning_machine_for_a_new_hybrid_morphologicallinear_perceptron https://www.researchgate.net/publication/338045815_Extreme_l...

carapace 6y ago

See also "Holographic Reduced Representation" https://web.stanford.edu/group/cslipublications/cslipublications/site/1575864304.shtml https://web.stanford.edu/group/cslipublications/cslipublicat... http://www.cogsci.ucsd.edu/~sereno/170/readings/06-Holographic.pdf http://www.cogsci.ucsd.edu/~sereno/170/readings/06-Holograph... https://www.ijcai.org/Proceedings/91-1/Papers/006.pdf https://www.ijcai.org/Proceedings/91-1/Papers/006.pdf