5 ms·
Study reveals how, when a synapse strengthens, its neighbors weaken
- jxub 8y agoThe naive conclusion is to avoid learning in any form in order to keep all the synapses in optimal state. Rather disappointing.
- obstbraende 8y agoThat's too naive. There's next to nothing known about how activity in individual neurons and their synapses relates to mental contents, in particular "higher level" concepts and thought patterns that you're concerned with in adult learning. In other words, the network dynamics are complex and unknown, and so the suppression / deactivation of certain synapses may just as well be a normal and necessary part of learning as it may be a part of forgetting, or neither. There is currently no contender for a "neuroscience standard model" that would bridge between this kind of neural dynamics and cognitive functions. I hope to live to see one.
- soup10 8y agoAmen to that, we are starting to get some of the groundwork but are far far away from a “standard model”, neuroscientists,psychiatrists and AI developers oversell their understanding in order to keep their jobs and funding.
- SubiculumCode 8y agoI know of no psychologist or neuroscientist who even try for a 'standard model' level of understanding of brain function.
- haskellandchill 8y agoIt’s really just the AI folks.
- deleted 8y ago[deleted]
- drak0n1c 8y agoRegardless of one’s general level of intelligence, there is that age-old adage of a trade-off between book smarts and street smarts, or any other niche skill. Forgetting, and the resulting ignorance, is part of learning. Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies. Still, any society has certain forms of learning with outsized benefits that make them worthwhile.
- azeirah 8y agoI remember reading a paper a little while ago that stated that experts are better at their craft because they're better at ignoring unimportant and irrelevant input.
- lawlessone 8y ago>Perhaps that’s why people in modern societies feel unhappier than those in poorer, less developed societies. I have heard this before and would really question if it is true. Has anyone studied this?
- lars 8y agoMaximizing total synapse strength would as useful as maximizing the number of 1-bits on your hard drive. It's not what you want :)
- deleted 8y ago[deleted]
- raincom 8y agoHere is the paper: http://science.sciencemag.org/content/360/6395/1349 http://science.sciencemag.org/content/360/6395/1349
- jchook 8y agoReally curious what the biology-centric AI folks have to say about this, e.g. Jeff Hawkins.
- rytill 8y agoNormalization of a layer’s outputs kind of does this already.
- audiolion 8y agoWe are born with a ton of synapses as babies and through a process called pruning, many synapses die off, unused branches. This seems pretty normal to me
- tunnuz 8y agoI wonder if the progress in artificial neural network technology is going to be correlated to discoveries on real neural networks.
- amelius 8y agoI'm still wondering about the natural equivalent of back-propagation.
- no_identd 8y agohttps://www.ncbi.nlm.nih.gov/pmc/articles/PMC3673183/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3673183/ Citations here: https://scholar.google.com/scholar?cites=6395717759743355511 https://scholar.google.com/scholar?cites=6395717759743355511
- p1esk 8y agoI think the term "backpropagation" has a different meaning in neurobiology.
- zerostar07 8y agothose are backpropagating spikes, not errors
- SubiculumCode 8y agoIsn't that mostly a supervised learning technique? Maybe the larger question is whether learning in the brain is better described as unsupervised or supervised.
- mrep 8y agoI don't know how babies learn to initially use words but after that entails 99% supervised learning through school IMO nowadays. I would even argue that most new economic growth is the also the result of general supervised learning (aka, learning from others because the wealth of human knowledge is way more than anyone can learn in a million lifetimes and so the economic growth comes from continuously combining new ideas that combine other new ideas).
- bsenftner 8y agoIsn't that kinda the point? It's called learning.
- deleted 8y ago[deleted]
- agumonkey 8y agoself pruning mechanism ?
- pmalynin 8y agoSo basically L2 regularization.
- p1esk 8y agoNot really. This is a local inhibition, and it's not clear if there's penalty on synapse growth.
- nabla9 8y agoIn deep learning this is called local response normalization (LRN). ConvNets use LRN where most active neurons inhibit other neurons at the same location in neighboring feature maps. http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf http://www.cs.toronto.edu/~fritz/absps/imagenet.pdf
- adityab 8y agoActually the new finding is similar to weight normalisation for individual neurons. LRN would be similar to another thing called lateral inhibition.
- deleted 8y ago[deleted]
- p1esk 8y agoSo it's similar to "lateral inhibition", but for synapses.
- alexbeloi 8y agoMy naive take on this is that it makes biologically sense to keep net expected electrical impulses approximately the same before and after strengthening. At the end of the day the brain has energy constraints. This can-be/is done functionally in ANNs but achieves a different end (avoids over-fitting) but doesn't reducing energy(compute) expenditure in dense ANNs since activation and non-activation is computed in expectation and take the same number of cycles in dense networks. I'd love to see more work on massive sparse networks, where you actually get compute efficiency if you can reduce number of activation without reducing hurting your optimization target.
- im3w1l 8y agoAnd apart from speeding things up, it could also have a regularizing effect.
- techsin101 8y agoDoes this mean addiction can completely be cured
- ridgeguy 8y agoHow interesting. This looks like a lateral inhibition effect running on biochemical means that are persistent compared to the transient electrochemical phenomena of classic neuronal lateral inhibition. The latter does lots of signal processing in visual and other systems. [1] [1] https://en.wikipedia.org/wiki/Lateral_inhibition https://en.wikipedia.org/wiki/Lateral_inhibition
- zerostar07 8y ago... and a model predicting and explaining this a decade ago: https://www.ncbi.nlm.nih.gov/pubmed/18602704 https://www.ncbi.nlm.nih.gov/pubmed/18602704
- manmal 8y agoSo this is how Eg brainwashing works? Repeating a message strengthens the new pathways until the old connections are too weak to fire?
- evincarofautumn 8y agoI would think any kind of training would have this effect. In mechanism—not purpose or ethics—brainwashing doesn’t strike me as all that different from ordinary practice.
- leeeeech 8y agoArguably, a single message would only affect a single pathway.
- manmal 8y agoAFAIK a single message causes a giant electrical and chemical thunderstorm in the brain, not only one excited single pathway?
- leeeeech 8y agoYou are right of course. But because a trivial message has many "layers" of "messages", according to semantic theory, it's often a single lie in a whole message packet.