3 ms·

> OTOH, having watched the field of computational neuroscience there has not been a lot of progress since , basically the 80s. So i believe it would be best to

> OTOH, having watched the field of computational neuroscience there has not been a lot of progress since , basically the 80s. So i believe it would be best to leave each of the two fields go their own way. I wouldn't really say that. There's a lot of progress being made in using and understanding biological processes for useful tasks. Not surprisingly, the biological mechanisms are surprisingly complex and rich and vary a lot through the brain (like you mentioned with the dendrites. Dendrites also work like coincidence detection mechanisms sometimes in some layers of the cortex [1] for instance) [2] gives a very nice overview of what is needed from both machine learning and computational neuroscience to solve the entire problem of understanding the human brain. There was the liquid state computing paper [3] in 2002 which showed that random networks of spiking neurons can perform some computations and have memory even in the absence of special learning rules. There has also been quite some work on understanding e.g. the role of assemblies of neurons (groups of neurons firing), assembly sequences, plasticity rules, theory formulating neural activity in a probabilistic manner, a better understanding of the role of inhibition, dendrites etc. And lastly, there have been huge advances in neuromorphic computing, both digital and analog. In some cases, the performance on these chips (that use spiking neurons) approaches that of state of the art machine learning. e.g. [4] [1] http://www.sciencedirect.com/science/article/pii/S0166223612002032 http://www.sciencedirect.com/science/article/pii/S0166223612... [2] https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/ [3] http://www.mitpressjournals.org/doi/abs/10.1162/089976602760407955 http://www.mitpressjournals.org/doi/abs/10.1162/089976602760... [4] http://www.pnas.org/content/113/41/11441.abstract http://www.pnas.org/content/113/41/11441.abstract

Seanny123 9y ago
Nice references! I have some criticism of "Toward an Integration of Deep Learning and Neuroscience" which I've talked to the authors about. I felt like they downplayed the importance of integration between neuroscience and machine learning [1], while over-playing the biological plausibility of Deep Learning [2]. I'm also uncomfortable with the characterization of Neuromorphic computing as "Deep Learning on a Chip", as this dismisses the possibility of online learning and neural dynamic systems [3], which I think are necessary for truly intelligent systems. As for Liquid State Machines, those evolved into Reservoir Computing and Echo State Machines. If you want to read more about them, I would recommend this paper comparing them to the Neural Engineering Framework [4] (with code!) to get a good idea of the state of the field. [1] https://medium.com/@seanaubin/deep-learning-is-almost-the-brain-3aaecd924f3d https://medium.com/@seanaubin/deep-learning-is-almost-the-br... [2] https://cogsci.stackexchange.com/q/16269/4397 https://cogsci.stackexchange.com/q/16269/4397 [3] https://medium.com/@seanaubin/a-way-around-the-coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-facd4291b201 https://medium.com/@seanaubin/a-way-around-the-coming-perfor... [4] https://github.com/arvoelke/delay2017/blob/master/delay2017.pdf https://github.com/arvoelke/delay2017/blob/master/delay2017....
return0 9y ago
Definitely people are working, but we dont have an addition to our methods of something fundamental. The papers you re referencing provide interesting conceptual theories, but none of them will survive for decades the way Hodkin-huxley's or Rall's models did. A lot of work on dendrites goes back to the 90s (e.g. seminal work of Mel http://www.nature.com/neuro/journal/v7/n6/abs/nn1253.html http://www.nature.com/neuro/journal/v7/n6/abs/nn1253.html) and we still dont have a convincing theory of what they do.

Seanny123 9y ago

Nice references! I have some criticism of "Toward an Integration of Deep Learning and Neuroscience" which I've talked to the authors about. I felt like they downplayed the importance of integration between neuroscience and machine learning [1], while over-playing the biological plausibility of Deep Learning [2]. I'm also uncomfortable with the characterization of Neuromorphic computing as "Deep Learning on a Chip", as this dismisses the possibility of online learning and neural dynamic systems [3], which I think are necessary for truly intelligent systems. As for Liquid State Machines, those evolved into Reservoir Computing and Echo State Machines. If you want to read more about them, I would recommend this paper comparing them to the Neural Engineering Framework [4] (with code!) to get a good idea of the state of the field. [1] https://medium.com/@seanaubin/deep-learning-is-almost-the-brain-3aaecd924f3d https://medium.com/@seanaubin/deep-learning-is-almost-the-br... [2] https://cogsci.stackexchange.com/q/16269/4397 https://cogsci.stackexchange.com/q/16269/4397 [3] https://medium.com/@seanaubin/a-way-around-the-coming-performance-walls-neuromorphic-hardware-with-spiking-neurons-facd4291b201 https://medium.com/@seanaubin/a-way-around-the-coming-perfor... [4] https://github.com/arvoelke/delay2017/blob/master/delay2017.pdf https://github.com/arvoelke/delay2017/blob/master/delay2017....

return0 9y ago

Definitely people are working, but we dont have an addition to our methods of something fundamental. The papers you re referencing provide interesting conceptual theories, but none of them will survive for decades the way Hodkin-huxley's or Rall's models did. A lot of work on dendrites goes back to the 90s (e.g. seminal work of Mel http://www.nature.com/neuro/journal/v7/n6/abs/nn1253.html http://www.nature.com/neuro/journal/v7/n6/abs/nn1253.html) and we still dont have a convincing theory of what they do.