3 ms·
Wait, my brain doesn't do backprop over a pile of linear algebra after having the internet rammed through it? No way that's crazy /s tl;dr: paper proposes a pr
by isaacimagine 2y ago
Wait, my brain doesn't do backprop over a pile of linear algebra after having the internet rammed through it? No way that's crazy /s
tl;dr: paper proposes a principle called 'prospective configuration' to explain how the brain does credit assignment and learns, as opposed to backprop. Backprop can lead to 'catastrophic interference' where learning new things abalates old associations, which doesn't match observed biological processes. From what I can tell, prosp. config learns by solving what the activations should have been to explain the error, and then updates the weights in accordance, which apparently somehow avoids abalating old associations. They then show how prosp. config explains observed biological processes. Cool stuff, wish I could find the code. There's some supplemental notes:
https://static-content.springer.com/esm/art%3A10.1038%2Fs41593-023-01514-1/MediaObjects/41593_2023_1514_MOESM1_ESM.pdf https://static-content.springer.com/esm/art%3A10.1038%2Fs415...
- jiggawatts 2y agoThe code: https://github.com/YuhangSong/Prospective-Configuration https://github.com/YuhangSong/Prospective-Configuration
- anon291 2y agoThis is like expressing surprise that a photon doesn't perform relativistic calculations on its mini chalkboard. A simulation of a thing is not thing itself, but it is illuminating. > pile of linear algebra The entirety of physics is -- as you say -- a 'pile of linear algebra' and 'backprop' (differential linear algebra...)
- drdeca 2y agoI don’t think “differential linear algebra” really counts as “backprop”.
- skissane 2y ago> Backprop can lead to 'catastrophic interference' where learning new things abalates old associations, which doesn't match observed biological processes. Most people find that if you move away from a topic and into a new one your knowledge of it starts to decay over time. 20+ years ago I had a job as a Perl and VB6 developer, I think most of my knowledge of those languages has been evacuated to make way for all the other technologies I've learned since (and 20 years of life experiences). Isn't that an example of "learning new things ablates old associations"?
- TheOtherHobbes 2y agoIs it replaced, or does it decay without reinforcement?
- skissane 2y agoHow can we distinguish those two possibilities? Stuff like childhood memories seems very deeply ingrained even if rarely or never reinforced. I can still remember the phone number of our house we moved out of in 1991, when I was 8 or 9. If I’m still alive in 30/40/50 years time, I expect I’ll still remember it then.