4 ms·

> Machine learning at the moment is an empirical science that has made great strides without consulting neuroscience for it. What is your basis for this statem

by closed 9y ago

> Machine learning at the moment is an empirical science that has made great strides without consulting neuroscience for it. What is your basis for this statement? Consider Geoff Hinton's publishing record, which contains many collaborations with notable psychologists / neuroscientists (e.g. Jay McClelland) who helped bring neural networks back into the spotlight. http://www.cs.toronto.edu/~hinton/papers.html http://www.cs.toronto.edu/~hinton/papers.html

mindcrime 9y ago
True. But, conversely, Hinton himself says in some of his videos (sorry, don't have the specific link in front of me) something to the effect of "ANN's are only very loosely modeled on real neurons and we make no pretense that this is anything like the way a real brain works". (Paraphrased, possibly poorly). The sense I get is that Machine Learning is mostly an empirical field at the moment, without a terribly solid theoretical underpinning. Which is not, of course, to suggest that ML researchers haven't consulted neuroscience at all. But there still seems to be a pretty big disconnect between neuroscience and ML. Unless I've just really missed something, which is entirely possible.
- closed 9y ago
  Yeah, that's fair. The goals of machine learning and neuroscience seem fairly different. Nobody in ML will complain if your neural network has great prediction but isn't supported biologically. OTOH, neuroscientists might take a model that isn't the best in terms of prediction, if it seems to better represents how the brain operates. There seems to be some interesting interplay, though. For example deepmind recently sponsored this ANN conference where most the speakers were neuroscientists. https://sites.google.com/site/ncpw15/ https://sites.google.com/site/ncpw15/
return0 9y ago
I think its fair to say that Hinton's justifications are post-hoc. I think a lot of neural network scientists also do that, as there are many basic components that dont have a biological analogue or even contradict biology, like backpropagation, LSTMs, RBM training layer-by-layer, semilinear activation functions. I can't think of an instance for example where the designers made a choice of connectivity "because that is how the brain does it" and they ended up having a better performing network. There have been recently attempts to match ANNs with spiking networks (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/), but only rough sketches of ideas are provided.

by closed 9y ago

mindcrime 9y ago

True. But, conversely, Hinton himself says in some of his videos (sorry, don't have the specific link in front of me) something to the effect of "ANN's are only very loosely modeled on real neurons and we make no pretense that this is anything like the way a real brain works". (Paraphrased, possibly poorly). The sense I get is that Machine Learning is mostly an empirical field at the moment, without a terribly solid theoretical underpinning. Which is not, of course, to suggest that ML researchers haven't consulted neuroscience at all. But there still seems to be a pretty big disconnect between neuroscience and ML. Unless I've just really missed something, which is entirely possible.

closed 9y ago
Yeah, that's fair. The goals of machine learning and neuroscience seem fairly different. Nobody in ML will complain if your neural network has great prediction but isn't supported biologically. OTOH, neuroscientists might take a model that isn't the best in terms of prediction, if it seems to better represents how the brain operates. There seems to be some interesting interplay, though. For example deepmind recently sponsored this ANN conference where most the speakers were neuroscientists. https://sites.google.com/site/ncpw15/ https://sites.google.com/site/ncpw15/

return0 9y ago

I think its fair to say that Hinton's justifications are post-hoc. I think a lot of neural network scientists also do that, as there are many basic components that dont have a biological analogue or even contradict biology, like backpropagation, LSTMs, RBM training layer-by-layer, semilinear activation functions. I can't think of an instance for example where the designers made a choice of connectivity "because that is how the brain does it" and they ended up having a better performing network. There have been recently attempts to match ANNs with spiking networks (e.g. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5021692/), but only rough sketches of ideas are provided.