3 ms·

What if the models just can't be distilled beyond a certain point down to an explanation in English words that makes sense to you and has "explanatory power"?

What if the models just can't be distilled beyond a certain point down to an explanation in English words that makes sense to you and has "explanatory power"? Clearly reducing the problem from the brain to an ANN is valuable because if we want to predict build or fix the brain, having approximations to pieces of it as an ANN let us get closer to doing that in the same way that more compact models or explanations help us get closer to doing those things.

marmaduke 8y ago
You can air-quote explanatory power, but it remains a useful way to refer to the relative utility of a scientific theory. An approximate model is fine, but its variables require an interpretation under some theory, which is not accomplished with a trained ANN. It’s like saying a histogram is useful: sure is but not as a theory.
- dontreact 8y ago
  At the moment, the way they are trained is not a good theory. But that is the compact human interpretable way of thinking of these models. It seems like if we keep iterating on this then we could arrive at a compact description of the neural network which is its learning rules, architecture and environment. Why is it important to have a compact explanation of the trained resulting model if the learning rule, architecture and data are a fairly compact description? It seems like for vision there are a few simple theories of learning: having layers of nonlinearities weight sharing across space and some way of doing credit assignment on the loss from a visual task Which taken together are enough to explain a large amount of the explainable variance in the neural data. I agree that the models could get more biologically realistic in the way they learn, but I disagree that it's important to explain how the learned model functions in a compact way, since there may be no such explanation better than the one based on learning.

marmaduke 8y ago

You can air-quote explanatory power, but it remains a useful way to refer to the relative utility of a scientific theory. An approximate model is fine, but its variables require an interpretation under some theory, which is not accomplished with a trained ANN. It’s like saying a histogram is useful: sure is but not as a theory.

dontreact 8y ago
At the moment, the way they are trained is not a good theory. But that is the compact human interpretable way of thinking of these models. It seems like if we keep iterating on this then we could arrive at a compact description of the neural network which is its learning rules, architecture and environment. Why is it important to have a compact explanation of the trained resulting model if the learning rule, architecture and data are a fairly compact description? It seems like for vision there are a few simple theories of learning: having layers of nonlinearities weight sharing across space and some way of doing credit assignment on the loss from a visual task Which taken together are enough to explain a large amount of the explainable variance in the neural data. I agree that the models could get more biologically realistic in the way they learn, but I disagree that it's important to explain how the learned model functions in a compact way, since there may be no such explanation better than the one based on learning.