3 ms·

The back-prop learning algorithm requires information non-local to the synapse to be propagated from output of the network backwards to affect neurons deep in t

The back-prop learning algorithm requires information non-local to the synapse to be propagated from output of the network backwards to affect neurons deep in the network. There is simply no evidence for this global feedback loop, or global error correction, or delta rule training in neurophysiological data collected in the last 80 years of intensive research. [1] As for "why", biological learning it is primarily shaped by evolution driven by energy expenditures constraints and survival of the most efficient adaptation engines. One can speculate that iterative optimization akin to the one run by GPUs in ANNs is way too energy inefficient to be sustainable in a living organism. Good discussion on biological constraints of learning (from CompSci perspective) can be found in Leslie Valiant book [2]. Prof. Valiant is the author of PAC [3] one of the few theoretically sound models of modern ML, so he's worth listening to. [1] https://news.ycombinator.com/item?id=26700536 https://news.ycombinator.com/item?id=26700536 [2] https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/0195126688 https://www.amazon.com/Circuits-Mind-Leslie-G-Valiant/dp/019... [3] https://en.wikipedia.org/wiki/Probably_approximately_correct_learning https://en.wikipedia.org/wiki/Probably_approximately_correct...

hctaw 6y ago
I think there's a significant difference worth illustrating that "there is no feedback path in the brain" is not at all equivalent to "learning by feedback is not possible in the brain." It's well known in dynamics that feed-forward networks are no longer feed-forward when outputs are coupled to inputs, an example of which would be a hypothetically feed-forward network of neurons in an animal and environmental conditioning teaching it the consequences of actions. I'm very curious on the biological constraints, but I'd reiterate my point above that feedback is a mathematical or logical abstraction for analyzing the behavior of the things we call networks - which are also abstractions. There's a distinction between the physical behavior of the things we see and the mathematical models we construct to describe them, like electromechanical systems where physically no such coupling from output-to-input appears to exist, yet its existence is crucially important analytically.

hctaw 6y ago

I think there's a significant difference worth illustrating that "there is no feedback path in the brain" is not at all equivalent to "learning by feedback is not possible in the brain." It's well known in dynamics that feed-forward networks are no longer feed-forward when outputs are coupled to inputs, an example of which would be a hypothetically feed-forward network of neurons in an animal and environmental conditioning teaching it the consequences of actions. I'm very curious on the biological constraints, but I'd reiterate my point above that feedback is a mathematical or logical abstraction for analyzing the behavior of the things we call networks - which are also abstractions. There's a distinction between the physical behavior of the things we see and the mathematical models we construct to describe them, like electromechanical systems where physically no such coupling from output-to-input appears to exist, yet its existence is crucially important analytically.