5 ms·

Okay, I feel like a crazy person arguing against a luminary of the field, but this is so misleading as to be disingenuous and make me trust the guy less. He's t

Okay, I feel like a crazy person arguing against a luminary of the field, but this is so misleading as to be disingenuous and make me trust the guy less. He's throwing causality and explainability into the same bucket and arguing against the need for the latter. >We often hear that AI systems must provide explanations and establish causal relationships, particularly for life-critical applications. >Yes, that can be useful. Or at least reassuring. >But sometimes people have accurate models of a phenomenon without any intuitive explanation or causation that provides an accurate picture of the situation. It goes on to argue mostly against the need for intuitive explanations, not the establishing of causal relationships. >Now, if there ever was a life-critical physical phenomenon, it is lift production by an airliner wing. >But we don't actually have a "causal" explanation for it, though we do have an accurate mathematical model and decades of experimental evidence. The physical models we have are causal ones. The intuitive abstractions like bernoulli's principle may not work, but analysis based on navier stokes sure does. You plug your shape (cause) into the equations and see what forces (effect) occur. That's causation. >You know what other life-critical phenomena we don't have good causal explanations for? > The mechanism of action of many drugs (if not most of them). Using an industry that's nearly synonymous with a randomized controlled trial as a refutation for the need of a causal relationship is crazy talk. The mechanism may be missing, but the causal explanation is that via a series of RCTs it's established that the drug causes the effects. I get that half of this is trying to go against a percieved need for intuitive explanations, but it weirdly lumps causation in there.

nabla9 7y ago
> The physical models we have are causal ones. The question is do you need a detailed causal explanation you can understand. In the case of lift, you can simulate aircraft model and calculate the change of vertical momentum in particles going above the wing and below the wing. Usually 80% of the lift comes from above the wing. If you are modelling a brick, it's the other way around. But do aircraft designers need to know that or are they just satisfied with less?
- 6gvONxR4sf7o 7y ago
  No I don't think that is the question. The questions (plural) are whether you need detailed explanations or intuitively understandable explanations or explanations at all, as well as whether you have to build a detailed causal model or an intuitive causal model or establish causation at all. Aircraft designers are happy to test something in a wind tunnel and establish that the change in shape caused a change in performance. But they knew to try that particular new shape that because of all the physical (causal) models at various levels of intuitive understandability.
  - joe_the_user 7y ago
    The thing is the article is starting out with the sort of network of semi-causal explanations that aircraft designers have and pointing out that this isn't simple causality, isn't just a line of causal events. But then the article indeed jumps from that observation to the claim that causation isn't something we should worry about - which feels just wrong to me (and I think a lot of people). I mean, all sort of intuitive explanations feel like causal explanation but, of course, are much more sketchy. But this feeling like causality matters imo, the whole network of explanations that satisfy a human together seems to make things robust, not fragile, whereas AI predictions and models are renown for being fragile. But still, I think a big part of the situation is we haven't characterized fully what humans do here.
    - taeric 7y ago
      Then would it be better to say that these models allow you to make causal changes? That is, sure, they don't tell you why the specific lift properties are there. However, they do enable a practitioner to make a change and know what impact it will have on the lift.
    - 6gvONxR4sf7o 7y ago
      >The thing is the article is starting out with the sort of network of semi-causal explanations that aircraft designers have and pointing out that this isn't simple causality, isn't just a line of causal events. I'd argue that it absolutely is simple causality. Sure we have to flail around in the dark a little to get to a better plane, but they physically test things during development. That's simple causality right there. I guess I'm arguing that engaging in active (not watching someone else do it) trial and error is a case of causal reasoning.
uoaei 7y ago
Explainability -> gradients "How much does this input seem to confuse this output? What is the pattern across inputs for how this model is systematically confused?" Causality -> counterfactuals "How would the outcome be different if x was different? If I acted differently, would I get a more favorable outcome?" You're right to say these are two different things. They are. And they're different still from interpretability, i.e., "What are the explicit patterns that this model is seeking in the data?" DL practitioners routinely mix up explainability and interpretability but I would never in a million years have seen LeCun be so intellectually dishonest as to lump causality in there with them.
- joe_the_user 7y ago
  The thing is, I would claim that causality, explainability and interpretability are all mixed together in human informal discussions of various phenomena. As others on the thread have pointed out, Pearl's causality isn't everyone's causality. A tension structure can disrupt our "common sense" idea of what's holding up what but tensions structure doesn't seem at all like a black-box, unexplainable item. The way the article mixes the range of these issues seems definitely wrong but I don't think that means the line between all things is normally crystal clear.
- abdullahkhalids 7y ago
  Do you have a some reading material that discusses the differences and similarities between these three concepts?
buboard 7y ago
> The physical models we have are causal ones. The physical laws are correlational, not causal. F=ma doesn't tell you if acceleration causes force or vice versa. Causation is a humanly imposed concept . That's why we need to invent things like the Chronology protection conjecture. https://en.wikipedia.org/wiki/Causality#Physics https://en.wikipedia.org/wiki/Causality#Physics
- srean 7y ago
  > The physical laws are correlational You mean if I extend the spring the weight stretching it will increase ?
  - ChrisLomont 7y ago
    You can do it by adding more mass in a fixed inertial frame, or by leaving the mass along and changing the reference frame. You cannot decide which happens, which is an insight from relativity. So, which was the cause? You cannot tell. You simply can do the math and see that it matches the experiment. Only later do humans try to call it cause and effect, because the math and the experiment match so often and so well. But this all can change if we find an experiment breaking the models. Before relativity, most people thought time was constant. They were wrong. So the above poster is correct in what we usually call cause and effect to simply be stronger correlations that match (current) models. That can all be changed in future understanding of Nature.
    - 6gvONxR4sf7o 7y ago
      Causation: if I do X, what happens to Y? For any well understood physical situation describing X and Y, the laws of physics tell you what happens to Y when you do X. If I swap out a mass on a spring for a heavier mass, physics will predict how much lower it hangs. The cause is me swapping one mass for another. The effect is where it now hangs. That's the whole point. If it wasn't this way, we'd say our physical understanding is wrong.
      - ChrisLomont 7y ago
kelnos 7y ago
> It goes on to argue mostly against the need for intuitive explanations, not the establishing of causal relationships. And then fails to acknowledge that we don't even have intuitive explanations for a lot of AI models! And for a layperson, it's just a complete black box; for cases where we do have intuitive explanations, I'm not sure experts are at all effective at translating them into something a layperson can understand. I'm not a physicist or aeronautical engineer, but I can grok intuitive explanations for how airplanes work without much trouble.

nabla9 7y ago

> The physical models we have are causal ones. The question is do you need a detailed causal explanation you can understand. In the case of lift, you can simulate aircraft model and calculate the change of vertical momentum in particles going above the wing and below the wing. Usually 80% of the lift comes from above the wing. If you are modelling a brick, it's the other way around. But do aircraft designers need to know that or are they just satisfied with less?

6gvONxR4sf7o 7y ago
No I don't think that is the question. The questions (plural) are whether you need detailed explanations or intuitively understandable explanations or explanations at all, as well as whether you have to build a detailed causal model or an intuitive causal model or establish causation at all. Aircraft designers are happy to test something in a wind tunnel and establish that the change in shape caused a change in performance. But they knew to try that particular new shape that because of all the physical (causal) models at various levels of intuitive understandability.
- joe_the_user 7y ago
  The thing is the article is starting out with the sort of network of semi-causal explanations that aircraft designers have and pointing out that this isn't simple causality, isn't just a line of causal events. But then the article indeed jumps from that observation to the claim that causation isn't something we should worry about - which feels just wrong to me (and I think a lot of people). I mean, all sort of intuitive explanations feel like causal explanation but, of course, are much more sketchy. But this feeling like causality matters imo, the whole network of explanations that satisfy a human together seems to make things robust, not fragile, whereas AI predictions and models are renown for being fragile. But still, I think a big part of the situation is we haven't characterized fully what humans do here.
  - taeric 7y ago
    Then would it be better to say that these models allow you to make causal changes? That is, sure, they don't tell you why the specific lift properties are there. However, they do enable a practitioner to make a change and know what impact it will have on the lift.
  - 6gvONxR4sf7o 7y ago
    >The thing is the article is starting out with the sort of network of semi-causal explanations that aircraft designers have and pointing out that this isn't simple causality, isn't just a line of causal events. I'd argue that it absolutely is simple causality. Sure we have to flail around in the dark a little to get to a better plane, but they physically test things during development. That's simple causality right there. I guess I'm arguing that engaging in active (not watching someone else do it) trial and error is a case of causal reasoning.

uoaei 7y ago

Explainability -> gradients "How much does this input seem to confuse this output? What is the pattern across inputs for how this model is systematically confused?" Causality -> counterfactuals "How would the outcome be different if x was different? If I acted differently, would I get a more favorable outcome?" You're right to say these are two different things. They are. And they're different still from interpretability, i.e., "What are the explicit patterns that this model is seeking in the data?" DL practitioners routinely mix up explainability and interpretability but I would never in a million years have seen LeCun be so intellectually dishonest as to lump causality in there with them.

joe_the_user 7y ago
The thing is, I would claim that causality, explainability and interpretability are all mixed together in human informal discussions of various phenomena. As others on the thread have pointed out, Pearl's causality isn't everyone's causality. A tension structure can disrupt our "common sense" idea of what's holding up what but tensions structure doesn't seem at all like a black-box, unexplainable item. The way the article mixes the range of these issues seems definitely wrong but I don't think that means the line between all things is normally crystal clear.
abdullahkhalids 7y ago
Do you have a some reading material that discusses the differences and similarities between these three concepts?

buboard 7y ago

> The physical models we have are causal ones. The physical laws are correlational, not causal. F=ma doesn't tell you if acceleration causes force or vice versa. Causation is a humanly imposed concept . That's why we need to invent things like the Chronology protection conjecture. https://en.wikipedia.org/wiki/Causality#Physics https://en.wikipedia.org/wiki/Causality#Physics

srean 7y ago
> The physical laws are correlational You mean if I extend the spring the weight stretching it will increase ?
- ChrisLomont 7y ago
  You can do it by adding more mass in a fixed inertial frame, or by leaving the mass along and changing the reference frame. You cannot decide which happens, which is an insight from relativity. So, which was the cause? You cannot tell. You simply can do the math and see that it matches the experiment. Only later do humans try to call it cause and effect, because the math and the experiment match so often and so well. But this all can change if we find an experiment breaking the models. Before relativity, most people thought time was constant. They were wrong. So the above poster is correct in what we usually call cause and effect to simply be stronger correlations that match (current) models. That can all be changed in future understanding of Nature.
  - 6gvONxR4sf7o 7y ago
    Causation: if I do X, what happens to Y? For any well understood physical situation describing X and Y, the laws of physics tell you what happens to Y when you do X. If I swap out a mass on a spring for a heavier mass, physics will predict how much lower it hangs. The cause is me swapping one mass for another. The effect is where it now hangs. That's the whole point. If it wasn't this way, we'd say our physical understanding is wrong.
    - ChrisLomont 7y ago

kelnos 7y ago

> It goes on to argue mostly against the need for intuitive explanations, not the establishing of causal relationships. And then fails to acknowledge that we don't even have intuitive explanations for a lot of AI models! And for a layperson, it's just a complete black box; for cases where we do have intuitive explanations, I'm not sure experts are at all effective at translating them into something a layperson can understand. I'm not a physicist or aeronautical engineer, but I can grok intuitive explanations for how airplanes work without much trouble.