This is a good question - it certainly is possible to run a NN on the artwork or pre-g-buffer, and you could imagine having something bake the changes. To do t

This is a good question - it certainly is possible to run a NN on the artwork or pre-g-buffer, and you could imagine having something bake the changes. To do that, you’d need access to the game’s source code and assets, and you would need to rewrite portions of the rendering engine. Training might be incredibly slow if it needs to iterate on builds of the game, or that problem could be worked around but might need considerable engineering to support the training iterations (e.g. data representation and a pipeline with the ability to render changes to the artwork and renderer on the fly without needing a game rebuild). So I’d speculate the answer to why not is that it’s just not what the authors set out to do, that they don’t have access to the game source nor the time or resources needed to refactor the renderer. Part of the magic of neural networks is their black box properties - they can do what they do without needing to understand 3d geometry or integrate with a render engine. Throw an image in, and a new image pops out, without the neural network needing to understand what it’s doing or why. That said, I would guess that coming down the road is examples of exactly what you’re suggesting - games that will use neural networks to drive realism in the assets and shaders and renderer. To some degree there are already tools starting to do this.