4 ms·
> One successful deep-learning approach involves modeling behavioral tasks. Researchers build an ANN and optimize it to solve a task analogous to the one studie
by ssivark 8y ago
> One successful deep-learning approach involves modeling behavioral tasks. Researchers build an ANN and optimize it to solve a task analogous to the one studied in animals. They then compare the internals of the trained ANN with the biological neural recordings, typically smoothed spike trains of a population of neurons. If there are quantitative similarities, researchers can then attempt to reverse-engineer the ANN in order to develop a mechanistic explanation for how the ANN solves the task. The insights found in the ANN can lead to testable hypotheses for how the biological network implements the behavior.
So, IIUC, the approach involves (motivated) guesswork over the space of practically implementable artificial neural network architectures (which is a strongly biased subset of the set of all actual neural networks), to mimic behavior that is presumably fully captured by biological neural networks.
While it is tremendously exciting to try something new and find surprises, why would I expect this to be a fruitful systematic approach to neuroscience? I.E. choose a particular behavior, find an artificial network solving it, and then "understand" why it works biologically.
Basically comes down to two noob questions:
1. Why should we expect the search over architectures of artificial network architectures to be efficient/fruitful?
2. How do we know that the brain might not use a completely different architecture since computationally implementable NNs have many artificial limitations.
I would appreciate any insights and expert comments. Thanks!
- marmaduke 8y ago> How do we know that the brain might not use a completely different architecture since computationally implementable NNs have many artificial limitations. not just ANNs have limitations, but biological networks as well. Degeneracy is biology's version of "more than one way to do it" and it's present everywhere; in other words, different brain networks can implement same function. One example of this is that when you write your signature with your foot, it shows similar characteristics as when you write with your hand. Degeneracy and complexity in biological systems is a good read in this direction: http://www.pnas.org/content/pnas/98/24/13763.full.pdf http://www.pnas.org/content/pnas/98/24/13763.full.pdf
- amelius 8y ago> How do we know that the brain might not use a completely different architecture since computationally implementable NNs have many artificial limitations. I'd like to add this question: have we found the natural equivalent of "backpropagation" yet?
- matt4077 8y agoThere's a somewhat similar mechanism which underlies all learning. It's "what fires together, wires together", meaning that neural connections with correlated activity patterns tend to get strengthened over time. While that mechanism operates on what at first seems to be the forward pass, the neurosystem seems to often operate in loops, with "useful" signals continuing to go around for a while to provide the learning signal.
- ssivark 8y agoOne of the proposals for a biologically realistic alternative to backpropagation is feedback alignment: http://www.cs.toronto.edu/~tingwuwang/2546.pdf http://www.cs.toronto.edu/~tingwuwang/2546.pdf (randomly selected representative slides from Google search. See reference therein)