3 ms·

Question for the experts: Is it possible that GANs will be used for rendering video game environments in the near-ish future? This has been one of my private pr

Question for the experts: Is it possible that GANs will be used for rendering video game environments in the near-ish future? This has been one of my private predictions with respect to upcoming tech, but I'd love to know if people are already thinking about this, or alternatively, why it won't happen.

Tenoke 5y ago
Non real-time as in generating levels for your game to release it's doable today. Real-time it will probably be doable soon, especially for shared-world games where a server can generate for multiple people at a time rather than single-player. Even real-time today it should be doable if you create your game with that in mind. You really don't need to generate all the textures, just a compact representation of the level which is to be rendered normally after the fact.
malka 5y ago
It has already started with dlss. I am not sure about Nvidia implementation but super resolution can have some adverserial training. If you mean asset / level generation. Then yes. It is the next step in procedural generation imo.
- Agentlien 5y ago
  DLSS is just image upscaling using a neural network. It's a very different problem from what is shown here or what I believe GP is talking about.
aspaviento 5y ago
If the result is also a 3d environment, it could save a lot of time designing scenarios.
mcbuilder 5y ago
Artistically, developers could do some trippy dream sequences with GANs, where the glitchyness and training artifacts add to the immersions. Because one can sample GANs or mix in latent dimensions, the experience can be tailored individually based on the characters decisions for instance.
jayd16 5y ago
It depends on how exactly you frame the question. If you ignore the implementation, its basically a procedural texturing technique? Those are widely used now. If you're talking about a real time post effect, it would probably be a bit too slow for a few more years. If you count SLAM techniques that label camera feeds for AR games, those are very close but I dont think most run at a full framerate.
- _Microft 5y ago
  "SLAM" is this: https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping https://en.wikipedia.org/wiki/Simultaneous_localization_and_...
  - jayd16 5y ago
    Yep, sorry about that.
    - _Microft 5y ago
      No problem, I thought I could help with that :)
- Aeolun 5y ago
  > If you're talking about a real time post effect, it would probably be a bit too slow for a few more years. The paper in question can apparently render at 2K and 30fps though. Or at least that’s what the videos claim.
rcv 5y ago
I'm not sure if this is what you're asking about, but here's a Two Minute Papers (dear fellow scholars!) video about a deep learning paper for super sampling with some applications for games: https://www.youtube.com/watch?v=OzHenjHBBds https://www.youtube.com/watch?v=OzHenjHBBds
m463 5y ago
That seems like an obvious use case. We've had procedurally generated worlds for a long time, but this would take it from roguelike top-down or isometric to immersive fps.

Tenoke 5y ago

Non real-time as in generating levels for your game to release it's doable today. Real-time it will probably be doable soon, especially for shared-world games where a server can generate for multiple people at a time rather than single-player. Even real-time today it should be doable if you create your game with that in mind. You really don't need to generate all the textures, just a compact representation of the level which is to be rendered normally after the fact.

malka 5y ago

It has already started with dlss. I am not sure about Nvidia implementation but super resolution can have some adverserial training. If you mean asset / level generation. Then yes. It is the next step in procedural generation imo.

Agentlien 5y ago
DLSS is just image upscaling using a neural network. It's a very different problem from what is shown here or what I believe GP is talking about.

aspaviento 5y ago

If the result is also a 3d environment, it could save a lot of time designing scenarios.

mcbuilder 5y ago

Artistically, developers could do some trippy dream sequences with GANs, where the glitchyness and training artifacts add to the immersions. Because one can sample GANs or mix in latent dimensions, the experience can be tailored individually based on the characters decisions for instance.

jayd16 5y ago

It depends on how exactly you frame the question. If you ignore the implementation, its basically a procedural texturing technique? Those are widely used now. If you're talking about a real time post effect, it would probably be a bit too slow for a few more years. If you count SLAM techniques that label camera feeds for AR games, those are very close but I dont think most run at a full framerate.

_Microft 5y ago
"SLAM" is this: https://en.wikipedia.org/wiki/Simultaneous_localization_and_mapping https://en.wikipedia.org/wiki/Simultaneous_localization_and_...
- jayd16 5y ago
  Yep, sorry about that.
  - _Microft 5y ago
    No problem, I thought I could help with that :)
Aeolun 5y ago
> If you're talking about a real time post effect, it would probably be a bit too slow for a few more years. The paper in question can apparently render at 2K and 30fps though. Or at least that’s what the videos claim.

rcv 5y ago

I'm not sure if this is what you're asking about, but here's a Two Minute Papers (dear fellow scholars!) video about a deep learning paper for super sampling with some applications for games: https://www.youtube.com/watch?v=OzHenjHBBds https://www.youtube.com/watch?v=OzHenjHBBds

m463 5y ago

That seems like an obvious use case. We've had procedurally generated worlds for a long time, but this would take it from roguelike top-down or isometric to immersive fps.