3 ms·

As an engineer in a systems neuroscience lab, I'm likely biased, but this is a misguided point of view: sufficiently flexible models for data have been around f

As an engineer in a systems neuroscience lab, I'm likely biased, but this is a misguided point of view: sufficiently flexible models for data have been around for a long time, but they're not useful because they provide no insight into the mechanisms at work. A DL model which might perform 100% still requires study itself to produce anything with explanatory value. edit to add, we are in fact doing projects with DL on real data, and it's extremely difficult to reverse-engineer the trained DL model to figure out how/why it does what it does: the parameters are simply uninterpretable without a good theory or mechanism, and this is precisely what is sought.

dontreact 8y ago
What if the models just can't be distilled beyond a certain point down to an explanation in English words that makes sense to you and has "explanatory power"? Clearly reducing the problem from the brain to an ANN is valuable because if we want to predict build or fix the brain, having approximations to pieces of it as an ANN let us get closer to doing that in the same way that more compact models or explanations help us get closer to doing those things.
- marmaduke 8y ago
  You can air-quote explanatory power, but it remains a useful way to refer to the relative utility of a scientific theory. An approximate model is fine, but its variables require an interpretation under some theory, which is not accomplished with a trained ANN. It’s like saying a histogram is useful: sure is but not as a theory.
  - dontreact 8y ago
    At the moment, the way they are trained is not a good theory. But that is the compact human interpretable way of thinking of these models. It seems like if we keep iterating on this then we could arrive at a compact description of the neural network which is its learning rules, architecture and environment. Why is it important to have a compact explanation of the trained resulting model if the learning rule, architecture and data are a fairly compact description? It seems like for vision there are a few simple theories of learning: having layers of nonlinearities weight sharing across space and some way of doing credit assignment on the loss from a visual task Which taken together are enough to explain a large amount of the explainable variance in the neural data. I agree that the models could get more biologically realistic in the way they learn, but I disagree that it's important to explain how the learned model functions in a compact way, since there may be no such explanation better than the one based on learning.
p1esk 8y ago
It seems like what you need is not a DL model, but something like HTM implementation: https://numenta.com/neuroscience-research/research-publications/papers/ https://numenta.com/neuroscience-research/research-publicati...

dontreact 8y ago

What if the models just can't be distilled beyond a certain point down to an explanation in English words that makes sense to you and has "explanatory power"? Clearly reducing the problem from the brain to an ANN is valuable because if we want to predict build or fix the brain, having approximations to pieces of it as an ANN let us get closer to doing that in the same way that more compact models or explanations help us get closer to doing those things.

marmaduke 8y ago
You can air-quote explanatory power, but it remains a useful way to refer to the relative utility of a scientific theory. An approximate model is fine, but its variables require an interpretation under some theory, which is not accomplished with a trained ANN. It’s like saying a histogram is useful: sure is but not as a theory.
- dontreact 8y ago
  At the moment, the way they are trained is not a good theory. But that is the compact human interpretable way of thinking of these models. It seems like if we keep iterating on this then we could arrive at a compact description of the neural network which is its learning rules, architecture and environment. Why is it important to have a compact explanation of the trained resulting model if the learning rule, architecture and data are a fairly compact description? It seems like for vision there are a few simple theories of learning: having layers of nonlinearities weight sharing across space and some way of doing credit assignment on the loss from a visual task Which taken together are enough to explain a large amount of the explainable variance in the neural data. I agree that the models could get more biologically realistic in the way they learn, but I disagree that it's important to explain how the learned model functions in a compact way, since there may be no such explanation better than the one based on learning.

p1esk 8y ago

It seems like what you need is not a DL model, but something like HTM implementation: https://numenta.com/neuroscience-research/research-publications/papers/ https://numenta.com/neuroscience-research/research-publicati...