For simulations like games, it's a trivial matter to feed the neural game engine pixel-perfect metadata. Instead of rendering the final shaded and textured pix

For simulations like games, it's a trivial matter to feed the neural game engine pixel-perfect metadata. Instead of rendering the final shaded and textured pixels, the engine would output just the material IDs, motion vectors, and similar "meta" data that would normally be the inputs into a real-time shader. The AI can use this as inputs to render a photorealistic output. It can be trained using offline-rendered "ground-truth" raytraced scenes. Potentially, video labelled in a similar way could be used to give it a flair of realism. This is already what NVIDIA DLSS and similar AI upscaling tech uses. The obvious next step is not just to upscale rendered scenes, but to do the rendering itself.