2 ms·
There was a neural net paper like this that generated a lot of discussion on HN, but that I haven't been able to find since (I probably downloaded it, but that
by pessimizer 1y ago
There was a neural net paper like this that generated a lot of discussion on HN, but that I haven't been able to find since (I probably downloaded it, but that teaches me to always remember to use Zotero because academic paper filenames are terrible.)
It was about replacing backprop with a mechanism that checked outcomes against predictions, and just adjusted parameters that deviated from the predictions rather than the entire path. It wasn't suitable for digital machines (because it isn't any more efficient on digital machines) but it worked on analog models. If anybody remembers this, I'd appreciate the link.
I might be garbling the paper because it's from memory and I'm not an expert, but hopefully it's recognizable.
- sirwhinesalot 1y agoI don't know if it is the paper you are thinking of (likely not) but this idea of checking predictions against outcomes is a very common idea in less mainstream AI research, including the so called "energy-based models" of Yann LeCun and the reference frames of the thousand brains project. A recent paper posted here also looked at Recurrent Neural Nets and how in simplifying the design to its core amounted to just having a latent prediction and repeatedly adjusting that prediction.
- pessimizer 1y agoIf it wasn't a thread on HN, it's probably not. I don't think it was LeCun. It was a long, well-illustrated paper with a web version.
- akomtu 1y agoI'm pretty sure it was a Geoffrey Hinton's paper who published it shortly before leaving Google.