3 ms·
Sounds interesting. Any links and references for "Fully Distributed Representation"? This is what I found when I Googled the quoted phrase: http://www.cap-lore.
by memexy 6y ago
Sounds interesting. Any links and references for "Fully Distributed Representation"? This is what I found when I Googled the quoted phrase: http://www.cap-lore.com/RWC97-kanerva.pdf http://www.cap-lore.com/RWC97-kanerva.pdf.
Skimming the paper this reminds me of word2vec and similar techniques for embedding words into vector spaces but instead of using XOR the embedding is accomplished with another neural network. Here's a link to a good description of the embedding process: https://towardsdatascience.com/neural-network-embeddings-explained-4d028e6f0526 https://towardsdatascience.com/neural-network-embeddings-exp....
- kragen 6y agoYup, that's the paper — I probably read it at the time from Norm's site, QEPD. More recently Pentti's been working on sparse distributed representations, which as I understand it greatly simplify the matching problem. I think the latent-spaces approach of ANNs is pretty interesting but I don't feel like I understand it well enough. It's a bit lower dimensional than Kanerva's proposal, though.
- breuderink 6y agoHere is a more recent paper [1] from Pentti Kanerva that gives a great introduction: http://www.rctn.org/vs265/kanerva09-hyperdimensional.pdf http://www.rctn.org/vs265/kanerva09-hyperdimensional.pdf. More papers can be found with the query "vector symbolic architecture". I find it utterly fascinating. [1] Kanerva, Pentti. “Hyperdimensional Computing: An Introduction to Computing in Distributed Representation with High-Dimensional Random Vectors.” Cognitive Computation 1 (2009): 139-159.