4 ms·
Computational Power Found in the Arms of Neurons
- zie1ony 7y agoHopefully this research will boost deep learning.
- GolDDranks 7y agoWhat I find interesting about this finding: I wonder if intra-dentrite vs. inter-dentrite vs. inter-neuronal learning has different learning rates, activation functions etc. If that's the case, maybe artifical networks could benefit such variety.
- raverbashing 7y agoMore than the computational aspects of neurons (which might be abstracted computationally), the learning process is something that seems it's still more of a mystery. The brain doesn't backpropagate. So how do you do it?
- im3w1l 7y agoWell part of it is local unsupervised hebbian learning, fire-together-wire-together, as it's succinctly put. In particular, neurons will try to fire more like upstream neurons and less like downstream neurons as determined by time-delay in correlation. Then there is also the brains reward center, the one that famously goes wrong when you get addicted to drugs. It releases chemicals which reinforce behavior. And finally some parts are "hard-coded" to be a certain way our DNA.
- im3w1l 7y agoBrains are a lot more complex than artificial networks in many ways. We know that there are many types of neurons and they use different types of neurotransmitter. and have different connectivity patterns. So yeah even without intra-dendrite learning there are biological justifications for what you propose.
- pflanze 7y agoSee earlier discussion at https://news.ycombinator.com/item?id=22061718 https://news.ycombinator.com/item?id=22061718
- axilmar 7y agoHas anyone ever tried to develop (or more appropriately, grow) an actual artificial biological brain, instead of a computer program? Perhaps trying to grow something like a brain may give us an insight on the actual algorithm used.
- cr0sh 7y agoYes - https://en.wikipedia.org/wiki/Cerebral_organoid https://en.wikipedia.org/wiki/Cerebral_organoid
- Enginerrrd 7y agoYes, and they've trained neurons to do tasks successfully. But the way they trained them is very different to how they naturally work. I don't think we know how to train a collection of neurons the "natural" way. https://www.newscientist.com/article/dn6573-brain-cells-in-a-dish-fly-fighter-plane/ https://www.newscientist.com/article/dn6573-brain-cells-in-a...
- neural_thing 7y agoI have written a short, clear, and dense book about this exact subject. It's free! Read it here: http://www.corticalcircuitry.com/ http://www.corticalcircuitry.com/
- neural_thing 7y agoA preprint that came out after I released the book that I would have mentioned: https://www.biorxiv.org/content/10.1101/613141v1.full https://www.biorxiv.org/content/10.1101/613141v1.full They model a single cortical neuron as a deep neural network with 7 hidden layers consisting of 128 hidden units each. Biological neurons are A LOT more powerful than "neurons" in neural networks. If you hear a claim about computer-brain parity being close - the people making it almost certainly don't understand the power of cortical neurons.
- cr0sh 7y agoI don't think any serious deep-learning researchers believe that their artificial models represent the reality of how an actual brain (or even the individual neurons) actually work. That's an idea, I believe, that has been perpetuated in the general public's mind by popular science articles and descriptions about deep learning and neural networks, in a (largely successful) attempt to attract and gain attention for the subject and those articles. One thing that did capture my imagination from this new research, though, as applied to deep learning, is whether this would necessarily cause any change in actual deep learning models? That is, even if the dendrites are performing more complex computations, does that change anything about the "standard" layered model of deep learning systems, or does one simply add more artificial neurons to the model to compensate (and thus, nothing is really gained)? Finally - and I know this is not the focus of this research - this new understanding doesn't seemingly do anything to advance our knowledge of how actual brains and neurons learn; as mentioned elsewhere, our current artificial models implement "back propagation" for learning purposes. While that has been a radically successful method, even its creator (Hinton) believes it to be wrong (and is working toward a different model of so-called "capsule networks" in response). As far as I am aware, no form of back propagation has been found to be performed in natural brains or neural networks, so we really don't know how this aspect works. Plenty of competing theories and ideas have sprung up regarding this. Hopefully one day we'll figure it out; I'm of the opinion that, like so many other things that have been discovered in the past, when we do figure it out, it will probably be something extremely simple and curious as to how it remained overlooked for so long. I kinda have a thought or idea that its done via "societal feedback loops" - that is, brains talking/interacting with other brains - though how you integrate this with a feedforward system isn't apparent to me. I'm also certain, though, that this idea isn't something new or unique (and it very likely is wrong, too).
- erikerikson 7y agoActually, single layer/later free networks can learn XOR under naturally occurring modulated learning conditions[1]. I am the author and was able to demonstrate 6-dimensional XOR learned in a Hopfield network using Hebbian learning. [1] https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT3DiZz/view?usp=drivesdk https://drive.google.com/file/d/1R4mS_4vzf5mycQez76O_mmpQDpT... (PDF)