3 ms·
Honestly, the current state of neural backpropagation research is still in flux and papers will be pretty technical. My background's in neuroscience, and the ro
by bbctol 10y ago
Honestly, the current state of neural backpropagation research is still in flux and papers will be pretty technical. My background's in neuroscience, and the rough summary of what we know so far is: action potentials can travel backwards down a neuron, and they probably do so in vivo, but we don't know if this actually does anything. There's no mechanism we understand for passing information backwards through a synapse. In fact, the only reason people suspect this might be important, and not just an accidental effect, is because backpropagation has been important in developing artificial neural networks.
Getting inspiration from nature is often useful (convolutional neural networks are loosely inspired by the way information is processed down the optic nerve) but for the specifics of neuron function, the brain is almost certainly doing things that are more complex than is practical to simulate. The brain contains hundreds of billions of neurons, and each one is orders of magnitude more complex than a node in an ANN, interacting with local and distant neurons through all kinds of separate but interlocking electrical, chemical, and long-term genetic actions.
- Florin_Andrei 10y agoWould an action potential traveling backwards be able to block the neuron from properly firing, briefly? In other words, could this effectively act as a short term inhibition?
- cr0sh 10y agoThank you for this explanation - I would be curious about any papers, no matter how technical. I've never let something like that stop me before...
- derefr 10y agoRegarding "brain-like" propagation strategies: Is there any NN model which tracks a "supply" each neuron has of signalling "molecules", such that a given neuron won't be able to communicate a message of class X if it's out of X-amine, unless/until it receives some from a neighbour? Or, is there any NN model that allows each neuron to send broadcast messages, ala extracellular ionic neurotransmission, which decay with "physical distance" (requiring each node on the neural graph to have a physical position) and which remain active until "sucked up" by something? I've always thought these two things—neurons needing to "spend" chemicals for neurotransmission, and neurons being able to observe the local-in-physical-space "loudness" of certain broadcast signals—seemed to have high potential for modelling agentive systems generally, since they seem to replicate other successful systems (markets; pheromonal communication), but I've never heard of either concept being studied in an ML context.
- webmaven 10y agoThe most mainstream of the "biologically-inspired" Artificial Neural Network variants is the Spiking Neural Network model, which IBM has been implementing in their TrueNorth chip architecture[0], and APT implements in their SpiNNaker[1] design. That said, this is only from the perspective of constructing ANN's for computational purposes. There are, of course, many detailed models of various processes in the brain, but this is generally under the heading of Computational Neuroscience[2] (though the field boundaries are blurry). The main software for simulating biologically accurate neural networks is "Neuron"[3], which is used in the Blue Brain Project[4]. [0] https://en.wikipedia.org/wiki/TrueNorth https://en.wikipedia.org/wiki/TrueNorth [1] https://en.wikipedia.org/wiki/SpiNNaker https://en.wikipedia.org/wiki/SpiNNaker [2] https://en.wikipedia.org/wiki/Computational_neuroscience https://en.wikipedia.org/wiki/Computational_neuroscience [3] https://en.wikipedia.org/wiki/Neuron_(software) https://en.wikipedia.org/wiki/Neuron_(software) [4] https://en.wikipedia.org/wiki/Blue_Brain_Project https://en.wikipedia.org/wiki/Blue_Brain_Project