I don't see any mention of DIAMOND (https://diamond-wm.github.io/ https://diamond-wm.github.io/) which does something pretty similar, training a model to predic

I don't see any mention of DIAMOND (https://diamond-wm.github.io/ https://diamond-wm.github.io/) which does something pretty similar, training a model to predict a game or otherwise 3D world based on videos of gameplay plus corresponding user inputs. It's fascinating how much understanding of the world is being extracted and learned by these models in order to do this. (For the 'that's not really understanding' crowd, what definition of 'understanding' are you using?)