3 ms·
But does predictive coding perceived as a valid theory for cortical neurons functioning? There was a paper from 2017 drawing similar conclusions about backprop
by 0lmer 5y ago
But does predictive coding perceived as a valid theory for cortical neurons functioning?
There was a paper from 2017 drawing similar conclusions about backprop approximation with Spike-Timing-Dependent Plasticity: https://arxiv.org/abs/1711.04214 https://arxiv.org/abs/1711.04214
Looks more grounded to current models of neuronal functioning. Nevertheless, it changed nothing in the field of deep learning since then.
- andyxor 5y agoas long as the model requires delta rule, or 'teacher signal' based error correction it is not biologically plausible.
- jwmullally 5y agoSome general background on STDP for the thread: Biological neurons don't just emit constant 0...1 float values, they communicate using time sensitive bursts of voltage known as "spike trains". Spiking Neural Networks (SNN) are a closer aproximation of natural networks than typical ML ANNs. [0] gives a quick overview. Spike-Timing-Dependant-Plasticity is a local learning rule experimentally observed in biological neurons. It's a form of Hebbian learning, aka "Neurons that fire together wire together." Summary from [1]. The top graph gives a clear picture of how the rule works. > With STDP, repeated presynaptic spike arrival a few milliseconds before postsynaptic action potentials leads in many synapse types to Long-Term Potentiation (LTP) of the synapses, whereas repeated spike arrival after postsynaptic spikes leads to Long-Term Depression (LTD) of the same synapse. --- [0]: https://towardsdatascience.com/deep-learning-versus-biological-neurons-floating-point-numbers-spikes-and-neurotransmitters-6eebfa3390e9 https://towardsdatascience.com/deep-learning-versus-biologic... [1]: http://www.scholarpedia.org/article/Spike-timing_dependent_plasticity http://www.scholarpedia.org/article/Spike-timing_dependent_p...