3 ms·

I'm still waiting for video games to adopt stable diffusion, GPT, and other GenAI models. the tech is there, but I guess the inertia in the industry doesn't all

I'm still waiting for video games to adopt stable diffusion, GPT, and other GenAI models. the tech is there, but I guess the inertia in the industry doesn't allow us to have nice things yet.

a_wild_dandan 3y ago
You'll see those things! We already have DLSS, for instance. But unfortunately we can't simply glue expensive black boxes onto games and ship them. Wrangling performant, richly interactive media is difficult enough without these models. This modern ML + gaming fusion space is barely in its infancy. We need to explore what's practical, and discover patterns to do it. Even without further breakthroughs, the next 5 to 10 years will be incredible. I'm so excited.
wayfinder 3y ago
Wouldn’t say tech is there yet. It still needs a lot of human input and direction so slapping it into a video game would just be immersion breaking when it generates something out of character randomly. There’s less impactful ways to implement it like generating art paintings in a museum dynamically but that is in the “a little gimmicky” territory.
- sva_ 3y ago
  I think quite a bit of stuff like texture generation has already been used for some time - just on the developers machines pre-release. I suppose using these technologies client-side in real time will be something else entirely. Consumer hardware is lagging behind for that though, even with optimizations such as quantization tweaks etc.
- orbital-decay 3y ago
  Temporal stability has been solved for a while [1], there's just nothing of that grade for diffusion models currently. From above, tagged geometry guiding a temporally stable neural renderer seems to be the way to go for games, but this needs to be confirmed in the trenches. The industry has a lot to digest, and of course the hardware should improve quite a bit. [1] https://isl-org.github.io/PhotorealismEnhancement/ https://isl-org.github.io/PhotorealismEnhancement/

I'm still waiting for video games to adopt stable diffusion, GPT, and other GenAI models. the tech is there, but I guess the inertia in the industry doesn't allow us to have nice things yet.

a_wild_dandan 3y ago

You'll see those things! We already have DLSS, for instance. But unfortunately we can't simply glue expensive black boxes onto games and ship them. Wrangling performant, richly interactive media is difficult enough without these models. This modern ML + gaming fusion space is barely in its infancy. We need to explore what's practical, and discover patterns to do it. Even without further breakthroughs, the next 5 to 10 years will be incredible. I'm so excited.

wayfinder 3y ago

Wouldn’t say tech is there yet. It still needs a lot of human input and direction so slapping it into a video game would just be immersion breaking when it generates something out of character randomly. There’s less impactful ways to implement it like generating art paintings in a museum dynamically but that is in the “a little gimmicky” territory.

sva_ 3y ago
I think quite a bit of stuff like texture generation has already been used for some time - just on the developers machines pre-release. I suppose using these technologies client-side in real time will be something else entirely. Consumer hardware is lagging behind for that though, even with optimizations such as quantization tweaks etc.
orbital-decay 3y ago
Temporal stability has been solved for a while [1], there's just nothing of that grade for diffusion models currently. From above, tagged geometry guiding a temporally stable neural renderer seems to be the way to go for games, but this needs to be confirmed in the trenches. The industry has a lot to digest, and of course the hardware should improve quite a bit. [1] https://isl-org.github.io/PhotorealismEnhancement/ https://isl-org.github.io/PhotorealismEnhancement/