3 ms·
If there is a weight update, there is a gradient, and a loss objective. You might not write them down explicitly. I can't recall exactly what the Hebbian updat
by yobbo 2y ago
If there is a weight update, there is a gradient, and a loss objective. You might not write them down explicitly.
I can't recall exactly what the Hebbian update is, but something tells me it minimises the "reconstruction loss", and effectively learns the PCA matrix.
- orbifold 2y agoNot every vector field has a potential. So not every weight update can be written as a gradient.
- yobbo 2y agoTrue.
- erikerikson 2y ago> loss objective There is no prediction or desired output, certainly explicit. I was playing with those things in my work to try and understand how our brains cause the emergence of intelligence rather than solve some classification or related problem. What I managed to replicate was the learning of XOR by some nodes and further that multidimensional XORs up to the number of inputs could be learned. Perhaps you can say that PCAish is the implicit objective/result but I still reject that there is any conceptual notion of what a node "should" output even if iteratively applying the learning rule leads us there.