3 ms·

i see what you mean but it 's kind of circular: the causal model is assumed, then observations are made and a law is formed which can be used to make prediction

i see what you mean but it 's kind of circular: the causal model is assumed, then observations are made and a law is formed which can be used to make predictions. The law itself is symmetric in time though and can make predictions in reverse, so the causal model is not baked in it. Regardless, this causal model works for physics which has well defined hypotheses and well defined deterministic systems. In biology , establishing causation (and thus explanability) is tricky because , even though hypotheses are well defined, the systems are not very deterministic. In ANNs OTOH, even though the systems can be very deterministic, there is very little to make in terms of hypotheses. An explanation of the sort "you have cancer because neurons 10, 18, and 19 fired" is not satisfactory enough to pass the human test. It may be that for some complicated problems, searching for patterns in the neurons in order to explain them may prove to be futile. Not that people should give up on that, but not everything has a neat closed form explanation. Lecun mentioned above that you may have recurrent relationships (which also occur in quantum systems), and these muddy the waters a lot, making it difficult to establish cause and effect. It is also a major pain in neuroscience, when real neurons are seen as an evolving dynamical system.

yellowapple 7y ago
> An explanation of the sort "you have cancer because neurons 10, 18, and 19 fired" is not satisfactory enough to pass the human test. Sure, but that doesn't mean we should be okay with not understanding why those neurons fired. That is: if an ANN can reliably figure out if someone has cancer based on various inputs, then it should be possible to isolate the ANN's rationale for that determination. That probably ain't an easy task by any means, but unless ANNs are magic spells not subject to our physical laws, it is possible nonetheless.
6gvONxR4sf7o 7y ago
I think you're limiting yourself too strictly to the equations and missing their application. Suppose, not a causal model, but a causal question. My question is if I put a mass weighing X on a spring with spring constant k, what will the displacement be (after oscillations stop, under standard gravity g, blah blah)? That's a causal question, answered by hooke's law. You can construct analogous questions for all sorts of scenarios. I'd say a model can be considered causal if it can answer all (relevant) causal questions about what it describes. Take something more dynamic, like a ball being thrown on a level surface (uniform gravity, ideal vacuum, etc). If I throw it with some force, it'll follow a perfect symmetric parabola and land with that exact same force. The force it lands with (y) is exactly identical to the force I threw it with (x). It's basically time symmetry, but simpler. The equation here is the symmetric y = x, so it won't help you define causation. But clearly the usual version of causation says that my cause is x and the effect is y. Maybe where we differ is that you say it's "the causal model is assumed, then observations are made and a law is formed which can be used to make predictions," while I'd describe it as "interventions are made, then observations are made and a law is formed which can be used to make predictions (and the law itself doesn't depend on the intervention taken)." That's what makes the y = x scenario not symmetric. X is my intervention.
- buboard 7y ago
  no physical law implies causation. most laws describe relationships between measurable quantities. there is no causation implied e.g. in e=mc2 or e=hv or maxwell's equations etc. the causality is imposed post-hoc depending on your application, but it's not explained by the law itself. alternatively, causality is a prerequisite in order to have physical laws at all.

yellowapple 7y ago

> An explanation of the sort "you have cancer because neurons 10, 18, and 19 fired" is not satisfactory enough to pass the human test. Sure, but that doesn't mean we should be okay with not understanding why those neurons fired. That is: if an ANN can reliably figure out if someone has cancer based on various inputs, then it should be possible to isolate the ANN's rationale for that determination. That probably ain't an easy task by any means, but unless ANNs are magic spells not subject to our physical laws, it is possible nonetheless.

6gvONxR4sf7o 7y ago

I think you're limiting yourself too strictly to the equations and missing their application. Suppose, not a causal model, but a causal question. My question is if I put a mass weighing X on a spring with spring constant k, what will the displacement be (after oscillations stop, under standard gravity g, blah blah)? That's a causal question, answered by hooke's law. You can construct analogous questions for all sorts of scenarios. I'd say a model can be considered causal if it can answer all (relevant) causal questions about what it describes. Take something more dynamic, like a ball being thrown on a level surface (uniform gravity, ideal vacuum, etc). If I throw it with some force, it'll follow a perfect symmetric parabola and land with that exact same force. The force it lands with (y) is exactly identical to the force I threw it with (x). It's basically time symmetry, but simpler. The equation here is the symmetric y = x, so it won't help you define causation. But clearly the usual version of causation says that my cause is x and the effect is y. Maybe where we differ is that you say it's "the causal model is assumed, then observations are made and a law is formed which can be used to make predictions," while I'd describe it as "interventions are made, then observations are made and a law is formed which can be used to make predictions (and the law itself doesn't depend on the intervention taken)." That's what makes the y = x scenario not symmetric. X is my intervention.

buboard 7y ago
no physical law implies causation. most laws describe relationships between measurable quantities. there is no causation implied e.g. in e=mc2 or e=hv or maxwell's equations etc. the causality is imposed post-hoc depending on your application, but it's not explained by the law itself. alternatively, causality is a prerequisite in order to have physical laws at all.