if you’re interested in causal inference, offline RL in particular is actually a problem of causal inference. unlike online RL, where you can essentially run ex

if you’re interested in causal inference, offline RL in particular is actually a problem of causal inference. unlike online RL, where you can essentially run experiments by acting on the environment and seeing what happens, in offline RL you must infer good actions from observational data alone. lots of interesting work looking at RL more broadly from this perspective — e.g. the classic “actor critic” RL algorithm is actually equivalent to the “doubly robust” estimator from causal inference.