3 ms·

The astute among you will notice that the outputs look more or less indistinguishable from modern video games -- which is to say, they don't look real. It's no

The astute among you will notice that the outputs look more or less indistinguishable from modern video games -- which is to say, they don't look real. It's not to downplay the achievement. It's to say that I hope one of you takes up the task of making truly real, indistinguishable-from-TV videos generated in real time. It may seem like an impossible dream. But so was realtime raytracing, at the start of my career. And there are a few promising signs that it's coming within reach. ML trained on video, applied to gamedev, seems like a matter of time. You could do it right now. Since the talking point of "We don't want real-looking outputs!" always comes up, I suggest ignoring that detail. It's worth doing, even if nobody wanted it, just to show biology who's boss. And incidentally, you'll want to become familiar with how to run blinded experiments, since you'll realize that your own judgement isn't any good when deciding how "real" something looks. The sole test is whether human observers can distinguish your outputs from real outputs no better than random chance.

carl_dr 5y ago
> ML trained on video, applied to gamedev, seems like a matter of time. You could do it right now. Indeed! Photorealistic GTA V using ML : https://www.theverge.com/platform/amp/2021/5/12/22432945/intel-gta-v-realistic-machine-learning-cityscapes-dataset https://www.theverge.com/platform/amp/2021/5/12/22432945/int...
- vinkelhake 5y ago
  The "photorealism" of that ML demo was vastly overstated. A lot of post processing effects in games are there to (somewhat ironically) emulate defects of cameras: vignetting, chromatic aberration, bloom, flares etc. I think this ML demo got there by effectively emulating a low-res, poorly white balanced, low framerate dash cam. I think those "defects" make people see the image looking more like the output of a real (albeit shitty) camera system. It would be more interesting to see what kind of output we could get at 4K 60fps and properly white balanced.
  - deleted 5y ago
    [deleted]
- ajmurmann 5y ago
  not to diminish their work, but it just changes the look of the game too much to look more dreary. I have the suspicion that this happened because the training set seems to be based on Northern European cities whereas the game is set in LA. I wish there was a version trained the engine on LA or at least a city more similar in feel. It certainly would make for a more intuitive comparison.
  - mhh__ 5y ago
    They basically made it look like GTA 4
alkonaut 5y ago
Realism is only one of the benefits of global illumination, the other (perhaps more important) is the massive increase in productivity you can get in creating environments. Having dynamically lit environments in engines with pre baked lightning requires carefully placing emitters, triggering them on geometry changes and so on. See when people argue “I wan’t dynamic and destructible environments” the problem is to a large extent the lighting. If you can blow a hole in the roof but the sun doesn’t shine in - you can tell something is wrong. So you can’t make a hole in the roof. With GI you can crash a part of the roof and the interior lighting “just works”.

carl_dr 5y ago

> ML trained on video, applied to gamedev, seems like a matter of time. You could do it right now. Indeed! Photorealistic GTA V using ML : https://www.theverge.com/platform/amp/2021/5/12/22432945/intel-gta-v-realistic-machine-learning-cityscapes-dataset https://www.theverge.com/platform/amp/2021/5/12/22432945/int...

vinkelhake 5y ago
The "photorealism" of that ML demo was vastly overstated. A lot of post processing effects in games are there to (somewhat ironically) emulate defects of cameras: vignetting, chromatic aberration, bloom, flares etc. I think this ML demo got there by effectively emulating a low-res, poorly white balanced, low framerate dash cam. I think those "defects" make people see the image looking more like the output of a real (albeit shitty) camera system. It would be more interesting to see what kind of output we could get at 4K 60fps and properly white balanced.
- deleted 5y ago
  [deleted]
ajmurmann 5y ago
not to diminish their work, but it just changes the look of the game too much to look more dreary. I have the suspicion that this happened because the training set seems to be based on Northern European cities whereas the game is set in LA. I wish there was a version trained the engine on LA or at least a city more similar in feel. It certainly would make for a more intuitive comparison.
- mhh__ 5y ago
  They basically made it look like GTA 4

alkonaut 5y ago

Realism is only one of the benefits of global illumination, the other (perhaps more important) is the massive increase in productivity you can get in creating environments. Having dynamically lit environments in engines with pre baked lightning requires carefully placing emitters, triggering them on geometry changes and so on. See when people argue “I wan’t dynamic and destructible environments” the problem is to a large extent the lighting. If you can blow a hole in the roof but the sun doesn’t shine in - you can tell something is wrong. So you can’t make a hole in the roof. With GI you can crash a part of the roof and the interior lighting “just works”.