4 ms·

I'm almost speechless. I've been keeping an eye on the text-to-video models, and if these example videos are truly indicative of the model, this is an order of

I'm almost speechless. I've been keeping an eye on the text-to-video models, and if these example videos are truly indicative of the model, this is an order of magnitude better than anything currently available. In particular, looking at the video titled "Borneo wildlife on the Kinabatangan River" (number 7 in the third group), the accurate parallax of the tree stood out to me. I'm so curious to learn how this is working. [Direct link to the video: https://player.vimeo.com/video/913130937?h=469b1c8a45 https://player.vimeo.com/video/913130937?h=469b1c8a45]

calgoo 3y ago
The video of the gold rush town just makes me think of what games like Red Dead and GTA could look like.
- 93po 3y ago
  holy cow, is that the future of gaming? instead of 3D renders it's real-time video generation, complete with audio and music and dialog and intelligent AI conversations and it's a unique experience no one else has ever played. gameplay mechanics could even change on the fly
  - monlockandkey 3y ago
    Shove all the tech you mentioned into a VR headset and it is literally game over for humans
    - Xirgil 3y ago
      Digital Westworld
      - slowmovintarget 3y ago
        I'll take one holodeck, please.
    - rightbyte 3y ago
      You'd still get a headache after 20 minutes. No matter how addictive, it wont be bad until you can wear VR headsets for hours.
      - krapp 3y ago
        Even otherwise, and no matter how good the screen and speakers are, a screen and speakers can only be so immersive. People oversell the potential for VR when they describe it as being as good as or better than reality. Nothing less than the Matrix is going to work in that regard.
        trafficante 3y ago
        Yep, once your brain gets over the immediate novelty of VR, it’s very difficult to get back that “Ready Player One” feeling due to the absence of sensory feedback. If/once they get it working though, society will shift fast. There’s an XR app called Brink Traveler that’s full of handcrafted photogrammetry recreations of scenic landmarks. On especially gloomy PNW winter days, I’ll lug a heat lamp to my kitchen and let it warm up the tiled stone a bit, put a floor fan on random oscillation, toss on some good headphones, load up a sunny desert location in VR, and just lounge on the warm stone floor for an hour. My conscious brain “knows” this isn’t real and just visuals alone can’t fool it anymore, but after about 15 minutes of visuals + sensory input matching, it stops caring entirely. I’ve caught myself reflexively squinting at the virtual sun even though my headset doesn’t have HDR.
      - theshackleford 3y ago
        Many people can. I can and have been since the DK1. I’ve done 12 hour plus stints in it.
        arathis 3y ago
        lol horseshit
        theshackleford 3y ago
  - gdubs 3y ago
    That's why NVIDIA's CEO said recently that in the future every pixel will be generated — not rendered.
    - Keyframe 3y ago
      five years ago: https://www.youtube.com/watch?v=ayPqjPekn7g https://www.youtube.com/watch?v=ayPqjPekn7g I'm eager to see an updated version.
      - 7734128 3y ago
        DLSS is essentially this, isn't it? It uses a low quality render from the game and then increases the fidelity with something very similar to a diffusion model.
  - dartos 3y ago
    Sometimes, but for specific or unique art styles, statistical models like this may not work well. For games like call of duty or other hyper realistic games it very likely will be.
    - arvinsim 3y ago
      For games like 2D/3D fighting games where you don't to generate a lot of terrain, the possibilities of randomly generating stages with unique terrain and obstacles is interesting.
      - dartos 3y ago
        That’s also true, but those stages would need to fit in a specific art style. A large part of fighting games is the style. The cost difference of just making bespoke art and tuning an AI system to generate it for you may not be worth it (at least right now.)
  - yard2010 3y ago
    The answer is most definitely YES. Computer games, and of course, porn, the stuff the internet is made up for.
  - joegibbs 3y ago
    I think for the near future we’ll see something like this: https://youtube.com/watch?v=P1IcaBn3ej0 https://youtube.com/watch?v=P1IcaBn3ej0 From a few years ago, where the game is rendered traditionally and used as a ground truth, with a model on top of it that enhances the graphics. After maybe 10-15 years we will be past the point where the entire game can be generated without obvious mistakes in consistency. Realtime AI dialogue is already possible but still a bit primitive, I wrote a blog post about it here: https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine
  - notpachet 3y ago
    Lucid Dreaming as a Service. See also: https://en.wikipedia.org/wiki/Vanilla_Sky https://en.wikipedia.org/wiki/Vanilla_Sky
QuadmasterXLII 3y ago
The diffusion is almost certainly taking place over some sort of compressed latent, from the visual quirks of the output I suspect that the process of turning that latent into images goes latent -> nerf / splat -> image, not latent -> convolutional decoder -> image
deleted 3y ago
[deleted]

calgoo 3y ago

The video of the gold rush town just makes me think of what games like Red Dead and GTA could look like.

93po 3y ago
holy cow, is that the future of gaming? instead of 3D renders it's real-time video generation, complete with audio and music and dialog and intelligent AI conversations and it's a unique experience no one else has ever played. gameplay mechanics could even change on the fly
- monlockandkey 3y ago
  Shove all the tech you mentioned into a VR headset and it is literally game over for humans
  - Xirgil 3y ago
    Digital Westworld
    - slowmovintarget 3y ago
      I'll take one holodeck, please.
  - rightbyte 3y ago
    You'd still get a headache after 20 minutes. No matter how addictive, it wont be bad until you can wear VR headsets for hours.
    - krapp 3y ago
      Even otherwise, and no matter how good the screen and speakers are, a screen and speakers can only be so immersive. People oversell the potential for VR when they describe it as being as good as or better than reality. Nothing less than the Matrix is going to work in that regard.
      - trafficante 3y ago
        Yep, once your brain gets over the immediate novelty of VR, it’s very difficult to get back that “Ready Player One” feeling due to the absence of sensory feedback. If/once they get it working though, society will shift fast. There’s an XR app called Brink Traveler that’s full of handcrafted photogrammetry recreations of scenic landmarks. On especially gloomy PNW winter days, I’ll lug a heat lamp to my kitchen and let it warm up the tiled stone a bit, put a floor fan on random oscillation, toss on some good headphones, load up a sunny desert location in VR, and just lounge on the warm stone floor for an hour. My conscious brain “knows” this isn’t real and just visuals alone can’t fool it anymore, but after about 15 minutes of visuals + sensory input matching, it stops caring entirely. I’ve caught myself reflexively squinting at the virtual sun even though my headset doesn’t have HDR.
    - theshackleford 3y ago
      Many people can. I can and have been since the DK1. I’ve done 12 hour plus stints in it.
      - arathis 3y ago
        lol horseshit
        theshackleford 3y ago
- gdubs 3y ago
  That's why NVIDIA's CEO said recently that in the future every pixel will be generated — not rendered.
  - Keyframe 3y ago
    five years ago: https://www.youtube.com/watch?v=ayPqjPekn7g https://www.youtube.com/watch?v=ayPqjPekn7g I'm eager to see an updated version.
    - 7734128 3y ago
      DLSS is essentially this, isn't it? It uses a low quality render from the game and then increases the fidelity with something very similar to a diffusion model.
- dartos 3y ago
  Sometimes, but for specific or unique art styles, statistical models like this may not work well. For games like call of duty or other hyper realistic games it very likely will be.
  - arvinsim 3y ago
    For games like 2D/3D fighting games where you don't to generate a lot of terrain, the possibilities of randomly generating stages with unique terrain and obstacles is interesting.
    - dartos 3y ago
      That’s also true, but those stages would need to fit in a specific art style. A large part of fighting games is the style. The cost difference of just making bespoke art and tuning an AI system to generate it for you may not be worth it (at least right now.)
- yard2010 3y ago
  The answer is most definitely YES. Computer games, and of course, porn, the stuff the internet is made up for.
- joegibbs 3y ago
  I think for the near future we’ll see something like this: https://youtube.com/watch?v=P1IcaBn3ej0 https://youtube.com/watch?v=P1IcaBn3ej0 From a few years ago, where the game is rendered traditionally and used as a ground truth, with a model on top of it that enhances the graphics. After maybe 10-15 years we will be past the point where the entire game can be generated without obvious mistakes in consistency. Realtime AI dialogue is already possible but still a bit primitive, I wrote a blog post about it here: https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine https://jgibbs.dev/blogs/local-llm-npcs-in-unreal-engine
- notpachet 3y ago
  Lucid Dreaming as a Service. See also: https://en.wikipedia.org/wiki/Vanilla_Sky https://en.wikipedia.org/wiki/Vanilla_Sky

QuadmasterXLII 3y ago

The diffusion is almost certainly taking place over some sort of compressed latent, from the visual quirks of the output I suspect that the process of turning that latent into images goes latent -> nerf / splat -> image, not latent -> convolutional decoder -> image

deleted 3y ago

[deleted]