Reinforcement learning has an issue currently with modeling complicated aspects of an agent’s given simulation (their “world model”) over long time horizons. T

Reinforcement learning has an issue currently with modeling complicated aspects of an agent’s given simulation (their “world model”) over long time horizons. This is probably because there is a vast amount of information in any given “frame” of this simulation and the agent has no baked-in priors about its environment (or if they do, it’s because they’ve been hand-programmed, which isn’t ideal in a field premised on the automation of such things). If you instead pretrain _another_ model using the same tech as GPT3 on the agent’s environment, the agents can use this model during their simulations to model the world they see, essentially giving them something that looks kind of like common sense (although I certainly wouldn’t call it that to another researcher).