4 ms·

>But then in a practical setting, the model doesn’t correspond to something physical or to the inference goal of a practitioner. How does this invalidate the n

>But then in a practical setting, the model doesn’t correspond to something physical or to the inference goal of a practitioner. How does this invalidate the notion of interpretability? A model can be both wrong/misapplied and fully interpretable.

mlthoughts2018 8y ago
> “How does this invalidate the notion of interpretability?” Who said that it did? I’m saying the development of a model that yields scientific progress doesn’t have to have a connection to the parent comment’s proposed definition of interpretability, and may have competing interpretability concerns driven by specific inference goals that have nothing to do with proofs about the model under assumptions.
- romaniv 8y ago
  >I’m saying the development of a model that yields scientific progress doesn’t have to have a connection to the parent comment’s proposed definition of interpretability Let's say someone used a neural network to predict planetary positions as a function of time, mass and so on. They made a network with several hundred parameters, trained until it stopped converging and published their final state vector. Would that ever lead to gravitational theory?
  - mlthoughts2018 8y ago
    What do you mean by “gravitational theory” apart from “correctly accounts for predicting the gravitational effects that we observe”? If a neural net can do that, then yes, a perfectly “elegant” theory could be the giant enumeration of a big ugly bunch of parameters. Of course we might suspect that’s overfitting and whittle down to a compact description, and that’s great, but doesn’t make the ugly enumeration of a big model any less scientific. Another point of view would be to say how does the Wiles proof of Fermat’s Last Theorem give us “understanding” (or substitute virtually any non-constructive existence proof, or proofs by contradiction, such as for the Halting Problem). Compact math descriptions are good, don’t get me wrong. When the mechanism of some data generating process is actually oriented in such a way that we can get a concise math description then we absolutely should. But that is largely unrelated to advancing understanding or scientific progress, which can perfectly be some ugly enumerated garble of parameters, or some extremely piecewise solution space. Not every data generating process is required to admit a concise mathematical description.
    - romaniv 8y ago
      >What do you mean by “gravitational theory” apart from “correctly accounts for predicting the gravitational effects that we observe”? F = G (m1 * m2) / r^2 Newton's theory of gravitation identified the relevant variables and their relationships. It had no junk variables or parameters. You can do algebraic transformations on it and use the formula outside of the original context. It was not fully correct, but it made exact predictions. People were able to notice the discrepancies it had with orbits of some celestial bodies, which helped in identifying general theory of relativity. All of this would have been lost of the formula was a state vector of a large neural network.
      - mlthoughts2018 8y ago
        > “Newton's theory of gravitation identified the relevant variables and their relationships.” No, it introduced a useful low-dimensional fiction that approximately works, with high accuracy at coarse scales, and doesn’t account for relativistic or quantum effects (and so, in an absolute sense, is only an approximation, not an interpretation of what is real). Besides, what you glibly type out as a few characters of a formula unpackages into a deeply hard-to-explain theory in natural language. It’s vastly less interpretable than simpler (and less correct) models because it introduces much more complexity of the meaning and physical concepts behind the formula terms. Just as quantum or relativistic extensions add even more complexity and counter-intuitive concepts and become even harder to interpret. > “All of this would have been lost of the formula was a state vector of a large neural network.” Why? I see no reason why that would be true. If there is a very low-dimensional manifold in parameter space that accounts for all the predictive power, that would be a very easy thing to diagnose from a neural network model. Lots of things have been discovered that way, like graph centrality metrics coming from eigen decompositions, style transfer, glove vectors, eigenfaces. Getting the high-dimensional representation that yields accurate predictions is often step 1, great scientific progress and often a huge leap forward. Then step 2 is often asking if there is some natural low-dimensional compression of the parameters that is approximately as good, and if so, does it correspond to any known quantities. Though sometimes this isn’t possible or even desirable, such as with non-parametric models, and yet such models can still represent “interpretable” scientific progress in the sense of allowing us to predict how states of affairs will develop when we previously couldn’t and solving inference goals.

mlthoughts2018 8y ago

> “How does this invalidate the notion of interpretability?” Who said that it did? I’m saying the development of a model that yields scientific progress doesn’t have to have a connection to the parent comment’s proposed definition of interpretability, and may have competing interpretability concerns driven by specific inference goals that have nothing to do with proofs about the model under assumptions.

romaniv 8y ago
>I’m saying the development of a model that yields scientific progress doesn’t have to have a connection to the parent comment’s proposed definition of interpretability Let's say someone used a neural network to predict planetary positions as a function of time, mass and so on. They made a network with several hundred parameters, trained until it stopped converging and published their final state vector. Would that ever lead to gravitational theory?
- mlthoughts2018 8y ago
  What do you mean by “gravitational theory” apart from “correctly accounts for predicting the gravitational effects that we observe”? If a neural net can do that, then yes, a perfectly “elegant” theory could be the giant enumeration of a big ugly bunch of parameters. Of course we might suspect that’s overfitting and whittle down to a compact description, and that’s great, but doesn’t make the ugly enumeration of a big model any less scientific. Another point of view would be to say how does the Wiles proof of Fermat’s Last Theorem give us “understanding” (or substitute virtually any non-constructive existence proof, or proofs by contradiction, such as for the Halting Problem). Compact math descriptions are good, don’t get me wrong. When the mechanism of some data generating process is actually oriented in such a way that we can get a concise math description then we absolutely should. But that is largely unrelated to advancing understanding or scientific progress, which can perfectly be some ugly enumerated garble of parameters, or some extremely piecewise solution space. Not every data generating process is required to admit a concise mathematical description.
  - romaniv 8y ago
    >What do you mean by “gravitational theory” apart from “correctly accounts for predicting the gravitational effects that we observe”? F = G (m1 * m2) / r^2 Newton's theory of gravitation identified the relevant variables and their relationships. It had no junk variables or parameters. You can do algebraic transformations on it and use the formula outside of the original context. It was not fully correct, but it made exact predictions. People were able to notice the discrepancies it had with orbits of some celestial bodies, which helped in identifying general theory of relativity. All of this would have been lost of the formula was a state vector of a large neural network.
    - mlthoughts2018 8y ago
      > “Newton's theory of gravitation identified the relevant variables and their relationships.” No, it introduced a useful low-dimensional fiction that approximately works, with high accuracy at coarse scales, and doesn’t account for relativistic or quantum effects (and so, in an absolute sense, is only an approximation, not an interpretation of what is real). Besides, what you glibly type out as a few characters of a formula unpackages into a deeply hard-to-explain theory in natural language. It’s vastly less interpretable than simpler (and less correct) models because it introduces much more complexity of the meaning and physical concepts behind the formula terms. Just as quantum or relativistic extensions add even more complexity and counter-intuitive concepts and become even harder to interpret. > “All of this would have been lost of the formula was a state vector of a large neural network.” Why? I see no reason why that would be true. If there is a very low-dimensional manifold in parameter space that accounts for all the predictive power, that would be a very easy thing to diagnose from a neural network model. Lots of things have been discovered that way, like graph centrality metrics coming from eigen decompositions, style transfer, glove vectors, eigenfaces. Getting the high-dimensional representation that yields accurate predictions is often step 1, great scientific progress and often a huge leap forward. Then step 2 is often asking if there is some natural low-dimensional compression of the parameters that is approximately as good, and if so, does it correspond to any known quantities. Though sometimes this isn’t possible or even desirable, such as with non-parametric models, and yet such models can still represent “interpretable” scientific progress in the sense of allowing us to predict how states of affairs will develop when we previously couldn’t and solving inference goals.