4 ms·
That's not correct. 1. First let's get through the easy part: reinforcement learning (RL) is not unique in its ability to identify cause-and-effect - this was
by xenocyon 6y ago
That's not correct.
1. First let's get through the easy part: reinforcement learning (RL) is not unique in its ability to identify cause-and-effect - this was achieved long ago through the use of randomized controlled trials. RL merely streamlines the task of reacting to such information (as well as optimizing experiments w.r.t. a desired goal).
2. Now the trickier part: you can learn causality from observational data alone if you combine this with understanding of a mechanism. Indeed, the whole field of causal inference is an attempt to formalize and extend such methods.
There is a massive space of problems where experimentation (whether old-fashioned A/B testing or more sophisticated online learning) is simply not possible, whether because of ethical reasons, cost, or other reasons behind non-destructive study. These problems are common in medicine, economics, physics. In such problems the only data is observational. Causal inference is very valuable here.
- blackbear_ 6y agoYou are completely right, I jumped through too many hops to reach that conclusion, and I was very imprecise in stating it. Let me try again: the most interesting and frequent setting is where you cannot control the experimental conditions and you have no idea about the causal mechanism at play. And that is where traditional ML cannot help, but RL can.
- zwaps 6y agoWhat you are describing seems to be essentially the process of science: Develop, step by step, the causal mechanisms at play until you solve the problem. It should not be underestimated that our collective knowledge of causal inference with statistical methods is good, but still improving. I mean, in that sense, ML is helpful. Heck, it is already used in causal inference in two-step estimators and the like.