4 ms·

Can someone explain a bit about the problems? I am not a Neuroscientist but a Machine Learning researcher, and given the recent results in Deep Learning / Artif

Can someone explain a bit about the problems? I am not a Neuroscientist but a Machine Learning researcher, and given the recent results in Deep Learning / Artificial Neural Networks which reach Human performance in certain specific tasks, it almost looks like if you would just put an ANN together which is somewhat big enough and somewhat similar wired together as the Human brain, you would yield something similar. And the point is, it doesn't really matter at all whether that is exactly like in the Human brain, and also, you don't really need to understand in detail how it will work, like you can even not really explain the current simple ANNs. It just works anyway. So, under this view, it's just a matter of scaling up, and to wait until we have enough computing power. And then, you would add more details to make it more close to the real Human brain.

sz4kerto 12y ago
As an ML practitioner, you certainly know that the ANNs you use have not much in common with biological (real) NNs. First, ANNs tend to be mostly feedforward, while rNNs are highly recurrent. ANNs are therefore not very good in tasks where memory is needed (I know about the developments involving LSTM neurons, but they are not analogous to the implicit memory of recurrent neural networks.) Second, ANNs usually don't have a time dimension, the firing is essentially a floating point value instead of action potential. Third, recurrent neural network structures do not scale: an efficient/reliable/reduntant system of 100B neurons will probably have extremely different structures than another one with 100M neurons -- because of recurrency and other stuff a rNN is a chaotic process (in the sense of sensitivity to parameters) that should be stabilized by the structure. And there are many-many other differences. Note that our task is not to solve problems but to figure out how the brain works. Also, recurrent neural network simulation cannot really be scaled right now, we don't have the hardware. It is not parallelizable with our current tools because of the huge number of connections. (Disclaimer: I was involved in a project trying to model real neural networks. It wasn't a huge success, but we learned a lot.)
- creamyhorror 12y ago
  This is a good informed summary, thank you. I asked the same question of my Alzheimer researcher friend, and he gave a pretty similar response including aspects like the huge computational requirements and the basic non-similarity of rNNs and ANNs (albeit with a disclaimer that he wasn't in the field). Nonetheless, I look forward to seeing more simple rNNs being created over time (besides the C. elegans one that was modeled recently). Who knows what strange organizational rules or structures we will discover from this strand of research?
- njloof 12y ago
  Many scientists (including my former advisor DeLiang Wang) are simulating dynamic oscillatory neural networks: http://web.cse.ohio-state.edu/~dwang/pnl/ http://web.cse.ohio-state.edu/~dwang/pnl/
skywhopper 12y ago
I would say that the problem with your proposal here is that those "more details" you mention consist of a vast horde of information we do not know anything about that far outweighs what we do know. At best, we know that certain areas of the brain concentrate certain functionality, and we can map rough levels of interconnectedness and activity traversal. That is not nearly enough information to build a brain. The neural nets that exist may sometimes achieve "Human performance" at narrow tasks but it's a big leap to infer that those neural nets are actually functional models of subsystems in the brain, or that their isolated functionality can be plugged into a larger group of such neural nets and have something coherent emerge. Our actual brains are far more interconnected and overlapping than is obvious from reading the latest fMRI study.

sz4kerto 12y ago

As an ML practitioner, you certainly know that the ANNs you use have not much in common with biological (real) NNs. First, ANNs tend to be mostly feedforward, while rNNs are highly recurrent. ANNs are therefore not very good in tasks where memory is needed (I know about the developments involving LSTM neurons, but they are not analogous to the implicit memory of recurrent neural networks.) Second, ANNs usually don't have a time dimension, the firing is essentially a floating point value instead of action potential. Third, recurrent neural network structures do not scale: an efficient/reliable/reduntant system of 100B neurons will probably have extremely different structures than another one with 100M neurons -- because of recurrency and other stuff a rNN is a chaotic process (in the sense of sensitivity to parameters) that should be stabilized by the structure. And there are many-many other differences. Note that our task is not to solve problems but to figure out how the brain works. Also, recurrent neural network simulation cannot really be scaled right now, we don't have the hardware. It is not parallelizable with our current tools because of the huge number of connections. (Disclaimer: I was involved in a project trying to model real neural networks. It wasn't a huge success, but we learned a lot.)

creamyhorror 12y ago
This is a good informed summary, thank you. I asked the same question of my Alzheimer researcher friend, and he gave a pretty similar response including aspects like the huge computational requirements and the basic non-similarity of rNNs and ANNs (albeit with a disclaimer that he wasn't in the field). Nonetheless, I look forward to seeing more simple rNNs being created over time (besides the C. elegans one that was modeled recently). Who knows what strange organizational rules or structures we will discover from this strand of research?
njloof 12y ago
Many scientists (including my former advisor DeLiang Wang) are simulating dynamic oscillatory neural networks: http://web.cse.ohio-state.edu/~dwang/pnl/ http://web.cse.ohio-state.edu/~dwang/pnl/

skywhopper 12y ago

I would say that the problem with your proposal here is that those "more details" you mention consist of a vast horde of information we do not know anything about that far outweighs what we do know. At best, we know that certain areas of the brain concentrate certain functionality, and we can map rough levels of interconnectedness and activity traversal. That is not nearly enough information to build a brain. The neural nets that exist may sometimes achieve "Human performance" at narrow tasks but it's a big leap to infer that those neural nets are actually functional models of subsystems in the brain, or that their isolated functionality can be plugged into a larger group of such neural nets and have something coherent emerge. Our actual brains are far more interconnected and overlapping than is obvious from reading the latest fMRI study.