5 ms·

The article is missing an important point: you cannot learn causality from observational data alone. It's not about shortcomings of this or that model, it's a t

The article is missing an important point: you cannot learn causality from observational data alone. It's not about shortcomings of this or that model, it's a theoretical impossibility. Reinforcement learning is uniquely positioned to build machines that understand cause-and-effect on their own because the algorithm is allowed to interact with the world, observe the results, gather more data, rule out hypotheses, and so on.

dfmooreqqq 6y ago
Agreed. But even RL is often difficult to transfer to other domains of knowledge. Though I agree that this is an important next challenge (and causality has been the "next challenge" for at least the last 5 years), it's often more easily solved these days with mixing in human expert knowledge to the equation (that is, using ML alongside of human expertise).
jasonwatkinspdx 6y ago
While CRTs are the gold standard for establishing causality, that doesn't mean all is lost in observational data alone. Causal inference is a rich field, with a lot of work in recent decades. There's so much more to causal inference than the old dismissive chesnut about correlation.
jbay808 6y ago
Can you elaborate on this? If agent A interacts with a system and can learn causal relationships, how could agent B who observes all of agent A's experiments not be capable of drawing the same conclusions? It seems that any theorem that rules out learning causality from observational data alone would also rule out learning causality from any kind of interactions. Unless you're assuming that agent A "knows" it has free will so its own actions have no cause, while agent B can't tell whether the environment caused agent A's actions or vice-versa. But if that's what the proof hinges on, it's pretty shallow, because agent A has no such guarantee that its own choices have no root cause.
- blackbear_ 6y ago
  Sorry, I was quite sloppy in my previous comment. Agent B certainly can learn everything that A learns from the same observations. What I meant is that you cannot learn from "general" observational data, unless it is structured in a certain way (the randomized controlled trial mentioned in a sibling). RL is able to gather data on its own, while other ML methods must do with what they are given. This means that RL could eventually discover the causal relationships, while non-RL cannot (except if the data comes from a RCT).
  - jbay808 6y ago
    Thank you for the follow-up! That's much clearer.
  - zwaps 6y ago
    Do I understand you correctly that you mean: "Learn automatically without further human input?" Because in statistics, causal inference is certainly possible without RCTs.
xenocyon 6y ago
That's not correct. 1. First let's get through the easy part: reinforcement learning (RL) is not unique in its ability to identify cause-and-effect - this was achieved long ago through the use of randomized controlled trials. RL merely streamlines the task of reacting to such information (as well as optimizing experiments w.r.t. a desired goal). 2. Now the trickier part: you can learn causality from observational data alone if you combine this with understanding of a mechanism. Indeed, the whole field of causal inference is an attempt to formalize and extend such methods. There is a massive space of problems where experimentation (whether old-fashioned A/B testing or more sophisticated online learning) is simply not possible, whether because of ethical reasons, cost, or other reasons behind non-destructive study. These problems are common in medicine, economics, physics. In such problems the only data is observational. Causal inference is very valuable here.
- blackbear_ 6y ago
  You are completely right, I jumped through too many hops to reach that conclusion, and I was very imprecise in stating it. Let me try again: the most interesting and frequent setting is where you cannot control the experimental conditions and you have no idea about the causal mechanism at play. And that is where traditional ML cannot help, but RL can.
  - zwaps 6y ago
    What you are describing seems to be essentially the process of science: Develop, step by step, the causal mechanisms at play until you solve the problem. It should not be underestimated that our collective knowledge of causal inference with statistical methods is good, but still improving. I mean, in that sense, ML is helpful. Heck, it is already used in causal inference in two-step estimators and the like.
benlivengood 6y ago
What of astronomy, then? I think it is possible, but certainly more difficult, to infer causality from observation.
6gvONxR4sf7o 6y ago
That's not true. Look up doubly robust estimators for a neat counterexample. Or IPTW for an extension of a dataset that can help with this (X, t, y, and propensities, instead of just X, t, y). Causal inference has a super rich literature that's quickly growing. Or better yet (stretching and sidestepping how you meant it), give your friendly neighborhood statistician/econometrician just a dataset and they can't do causal inference. Give them propensities and column names/descriptions and a writeup of the experiment/where the data came from, and suddenly they might be able to do causal inference. It points to a need to augment our observations with more structured metadata, if we want to do causal inference with data that's just lying around.
axiom92 6y ago
>you cannot learn causality from observational data alone. As others have pointed out, this is kind of the point of much of Pearl's work on causality. Specifically, do-calculus provides a set of primitive operations that can be used to convert queries in interventional/counterfactual (causal) distributions to estimands in a purely observational distribution.

dfmooreqqq 6y ago

Agreed. But even RL is often difficult to transfer to other domains of knowledge. Though I agree that this is an important next challenge (and causality has been the "next challenge" for at least the last 5 years), it's often more easily solved these days with mixing in human expert knowledge to the equation (that is, using ML alongside of human expertise).

jasonwatkinspdx 6y ago

While CRTs are the gold standard for establishing causality, that doesn't mean all is lost in observational data alone. Causal inference is a rich field, with a lot of work in recent decades. There's so much more to causal inference than the old dismissive chesnut about correlation.

jbay808 6y ago

Can you elaborate on this? If agent A interacts with a system and can learn causal relationships, how could agent B who observes all of agent A's experiments not be capable of drawing the same conclusions? It seems that any theorem that rules out learning causality from observational data alone would also rule out learning causality from any kind of interactions. Unless you're assuming that agent A "knows" it has free will so its own actions have no cause, while agent B can't tell whether the environment caused agent A's actions or vice-versa. But if that's what the proof hinges on, it's pretty shallow, because agent A has no such guarantee that its own choices have no root cause.

blackbear_ 6y ago
Sorry, I was quite sloppy in my previous comment. Agent B certainly can learn everything that A learns from the same observations. What I meant is that you cannot learn from "general" observational data, unless it is structured in a certain way (the randomized controlled trial mentioned in a sibling). RL is able to gather data on its own, while other ML methods must do with what they are given. This means that RL could eventually discover the causal relationships, while non-RL cannot (except if the data comes from a RCT).
- jbay808 6y ago
  Thank you for the follow-up! That's much clearer.
- zwaps 6y ago
  Do I understand you correctly that you mean: "Learn automatically without further human input?" Because in statistics, causal inference is certainly possible without RCTs.

xenocyon 6y ago

That's not correct. 1. First let's get through the easy part: reinforcement learning (RL) is not unique in its ability to identify cause-and-effect - this was achieved long ago through the use of randomized controlled trials. RL merely streamlines the task of reacting to such information (as well as optimizing experiments w.r.t. a desired goal). 2. Now the trickier part: you can learn causality from observational data alone if you combine this with understanding of a mechanism. Indeed, the whole field of causal inference is an attempt to formalize and extend such methods. There is a massive space of problems where experimentation (whether old-fashioned A/B testing or more sophisticated online learning) is simply not possible, whether because of ethical reasons, cost, or other reasons behind non-destructive study. These problems are common in medicine, economics, physics. In such problems the only data is observational. Causal inference is very valuable here.

blackbear_ 6y ago
You are completely right, I jumped through too many hops to reach that conclusion, and I was very imprecise in stating it. Let me try again: the most interesting and frequent setting is where you cannot control the experimental conditions and you have no idea about the causal mechanism at play. And that is where traditional ML cannot help, but RL can.
- zwaps 6y ago
  What you are describing seems to be essentially the process of science: Develop, step by step, the causal mechanisms at play until you solve the problem. It should not be underestimated that our collective knowledge of causal inference with statistical methods is good, but still improving. I mean, in that sense, ML is helpful. Heck, it is already used in causal inference in two-step estimators and the like.

benlivengood 6y ago

What of astronomy, then? I think it is possible, but certainly more difficult, to infer causality from observation.

6gvONxR4sf7o 6y ago

That's not true. Look up doubly robust estimators for a neat counterexample. Or IPTW for an extension of a dataset that can help with this (X, t, y, and propensities, instead of just X, t, y). Causal inference has a super rich literature that's quickly growing. Or better yet (stretching and sidestepping how you meant it), give your friendly neighborhood statistician/econometrician just a dataset and they can't do causal inference. Give them propensities and column names/descriptions and a writeup of the experiment/where the data came from, and suddenly they might be able to do causal inference. It points to a need to augment our observations with more structured metadata, if we want to do causal inference with data that's just lying around.

axiom92 6y ago

>you cannot learn causality from observational data alone. As others have pointed out, this is kind of the point of much of Pearl's work on causality. Specifically, do-calculus provides a set of primitive operations that can be used to convert queries in interventional/counterfactual (causal) distributions to estimands in a purely observational distribution.