Yeah, I work on on-device audio, so definitely agree on the expense problem. In audio we've now got a few different approaches that work really well for sequenc

Yeah, I work on on-device audio, so definitely agree on the expense problem. In audio we've now got a few different approaches that work really well for sequence modeling, and we're constantly finding cool ways to make inference run faster. Reframing reinforcement learning as sequence modeling /should/ make it possible to reuse (m)any of the approaches we use for audio, including GANs. Generative audio is nicely analogous to RL problems : There's complex state and interesting predictive distributions, which shift subtly over time, and you need to combine short-term good behavior (good individual samples) in a reasonable way to get good long-term behavior (matching melspectra).