This is basically a re-implementation of the recent work from OpenAI[1], where you first collect high-quality observation-action-observation transition tuples,

This is basically a re-implementation of the recent work from OpenAI[1], where you first collect high-quality observation-action-observation transition tuples, train a model to predict the actions between observations, then use that trained model to label a much larger observation-only observation dataset (e.g. youtube videos of people playing a game). Finally, you just do dead-simple behavior cloning (predict "expert" action from current observation) on the large labeled dataset. It's still surprising to me how well this works! Behavior cloning has some theoretical issues since it doesn't address the sequential nature of decision problems at all, but apparently with enough scale it can still do fairly well - super cool. [1]: https://openai.com/blog/vpt/ https://openai.com/blog/vpt/