6 ms·

What does "world" mean here? How does the spatiality fit into some latent space? Or what constitutes the "world"? If the answer is, there is none, the world is

What does "world" mean here? How does the spatiality fit into some latent space? Or what constitutes the "world"? If the answer is, there is none, the world is just frames of video and any consistency quickly blurs out after a few seconds. That's not a world generation, that's just generation of video frames following frames. Not that it isn't cool, but it has almost zero usability for generating a "world" simulation. The key to a realistic world is that you can reliably navigate it. Visit and revisit places. If you modify anything, those modifications are persisted. If you leave a room and re-enter it hours later, the base expectation is that the same objects are in that room. Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world? It wouldn't be infinite, but on the other hand no one has a need for an infinite world either. A spherical world solves the border issue pretty handily. As I understood it, there was some element of that in the new FS2024 (discussed yesterday on HN).

QuantumG 2y ago
What you really want is a story telling engine informing the world model and the humans need to generate power.
scotty79 2y ago
It's basically a dream. Maybe that's what dreams are? World prediction engine running with no sensor input and memory in read only mode?
- withinboredom 2y ago
  When you go to sleep, your brain stem disconnects from your body and your brain enters a feedback loop. The sensors are very much connected, just to whatever. Hence why you can grow dragon wings in your dreams and feel them. Memories can be made as well. I used to be really into lucid dreaming and time compression. My longest dream was nine years compressed into a 12-hour sleep period.
  - Helonomoto 2y ago
    Where did you get the 'brain stem disconnects from your body'? Because thats not how it works in the brain. We have the part which controls your muscles and we have another part which simulates the movement. Not executing on it has nothing to do with the brain stem disconnecting from the 'body'. Its the same mechanism as you thinking about a movement but not doing it.
    - lxgr 2y ago
      I believe it's a much more fundamental difference than just the distinction between ideating and acting. Many people occasionally experience the transition between "conencted" and "disconnected" states as a sudden jerk or loud noise just at the moment of falling asleep. Sleep paralysis is another "failure mode" of this mechanism that reveals what's going on. (I'm not sure if there is a reverse to it, i.e. whether sleepwalking could be explained as a drastic fail-open of the same mechanism).
      - Helonomoto 2y ago
        I read a book, just a few weeks ago regarding this specific topic and my explanaition is directly from that book. We do have specific brain areas for this. This sudden jerk you can read about it here: https://en.wikipedia.org/wiki/Hypnic_jerk https://en.wikipedia.org/wiki/Hypnic_jerk and its not scienctificly clear why and how it works, the best assumption is that its a reflex. You are not disconnecting your brain from anything.
        withinboredom 2y ago
        Your brain disconnects: https://article.imrpress.com/bri/Landmark/articles/pdf/Landmark1043.pdf https://article.imrpress.com/bri/Landmark/articles/pdf/Landm... is probably the best paper I know of that describes the actual process going on in detail.
  - jebarker 2y ago
    > My longest dream was nine years compressed into a 12-hour sleep period. Does the brain change in response to that sleep period? Or is there no change because there's no new information input?
roboboffin 2y ago
Not an expert, but I think procedurally generated terrain is generally fractal in nature, and is reproducible in that sense from a seed that is used in the generation. It is therefore recursive, as fractals are recursive. A traditional neural network is a universal function approximator, however it is not recursive in nature, unless it is some sort of RNN. The transformer architecture, which this seems fairly similar to this one, is also not recursive in nature; although I believe, limited recursion can come about through CoT. Therefore, I don't believe this could match the reproducibility, in an infinite sense, of a traditional procedural generator.
- jrvieira 2y ago
  "Fractals are not self-similar" https://m.youtube.com/watch?v=gB9n2gHsHN4 https://m.youtube.com/watch?v=gB9n2gHsHN4 If you want to learn more about the fascinating world of fractals.
  - falcor84 2y ago
    The parent comment didn't mention self-similarity; did you maybe intend to reply to another comment?
- jerf 2y ago
  Procedurally generated terrain is whatever it wants to be. It isn't necessary for it to be fractal in any particular sense. Or even arguably all that useful, at least at the present time.
- astrobe_ 2y ago
  Luanti [1] (Minetest) has a fractal map generator ("mapgen"). You can test it for yourself. It's funny at first but becomes eventually boring. Its other mapgens massively some kind of Perlin noise in various ways, so that you can have "realistic" landscapes (e.g. the Carpethian mapgen) or landscapes with impossible mountains and floating rocks sometimes (e.g. the V7 mapgen) that are good for fantasy/sci-fi worlds. Noise is a pretty efficient way to fake complexity, and it's not a coincidence [2]. [1] https://www.luanti.org/ https://www.luanti.org/ [2] https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity
jerf 2y ago
"Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world?" That's Minecraft. It does work. And when you turn around, then turn back to where you were facing, you don't feel like you're in a fever dream, because the landscape hasn't completely shifted. The tech is neat and I'm sure worth publishing. The rhetoric accompanying it is terribly overblown. Of course that's as likely to be their University PR department as the authors.
astrobe_ 2y ago
> Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world? If you go "ultra-low" resolution, that's basically Minecraft or Luanti (formerly Minetest). There some mods for Luanti that let one generate a less "squary" world by adding slopes, but that's still heavily polygonal. > It wouldn't be infinite, but on the other hand no one has a need for an infinite world either. Minecraft has an infinite world (with glitches when you go very far from origin due to floating point errors). Luanti's world is finite (around 64000 m^3 because 16 bits coordinates; players move at 2-4m/s usually), some people are working to make it infinite. In my opinion, you are right that nobody needs an infinite world; the argument for an infinite world is that a finite world can be a problem on popular servers but in my eyes servers where this is a problem don't use enough the available space (in particular the huge vertical space that can be used to create multiple worlds) or shoot themselves in the foot by providing players with ways to move fast (mainly transports); increasing the player's speed actually shrinks the world. I think there is a conundrum from the players and gameplay perspective. On one hand, you want a lot of space, but on the other hand you want to be close to other players (for commerce, play together, etc.). It's not enough to have an infinite world, you also have to have the gameplay that goes with it. Another thing to consider are the interaction of mobiles (NPCs, enemies) with the terrain. Because nobody likes an infinitely empty world either. I haven't played an AAA game recently, but a decade ago even with precomputed terrain, some mobiles would look silly sometimes with steep terrains.

QuantumG 2y ago

What you really want is a story telling engine informing the world model and the humans need to generate power.

scotty79 2y ago

It's basically a dream. Maybe that's what dreams are? World prediction engine running with no sensor input and memory in read only mode?

withinboredom 2y ago
When you go to sleep, your brain stem disconnects from your body and your brain enters a feedback loop. The sensors are very much connected, just to whatever. Hence why you can grow dragon wings in your dreams and feel them. Memories can be made as well. I used to be really into lucid dreaming and time compression. My longest dream was nine years compressed into a 12-hour sleep period.
- Helonomoto 2y ago
  Where did you get the 'brain stem disconnects from your body'? Because thats not how it works in the brain. We have the part which controls your muscles and we have another part which simulates the movement. Not executing on it has nothing to do with the brain stem disconnecting from the 'body'. Its the same mechanism as you thinking about a movement but not doing it.
  - lxgr 2y ago
    I believe it's a much more fundamental difference than just the distinction between ideating and acting. Many people occasionally experience the transition between "conencted" and "disconnected" states as a sudden jerk or loud noise just at the moment of falling asleep. Sleep paralysis is another "failure mode" of this mechanism that reveals what's going on. (I'm not sure if there is a reverse to it, i.e. whether sleepwalking could be explained as a drastic fail-open of the same mechanism).
    - Helonomoto 2y ago
      I read a book, just a few weeks ago regarding this specific topic and my explanaition is directly from that book. We do have specific brain areas for this. This sudden jerk you can read about it here: https://en.wikipedia.org/wiki/Hypnic_jerk https://en.wikipedia.org/wiki/Hypnic_jerk and its not scienctificly clear why and how it works, the best assumption is that its a reflex. You are not disconnecting your brain from anything.
      - withinboredom 2y ago
        Your brain disconnects: https://article.imrpress.com/bri/Landmark/articles/pdf/Landmark1043.pdf https://article.imrpress.com/bri/Landmark/articles/pdf/Landm... is probably the best paper I know of that describes the actual process going on in detail.
- jebarker 2y ago
  > My longest dream was nine years compressed into a 12-hour sleep period. Does the brain change in response to that sleep period? Or is there no change because there's no new information input?

roboboffin 2y ago

Not an expert, but I think procedurally generated terrain is generally fractal in nature, and is reproducible in that sense from a seed that is used in the generation. It is therefore recursive, as fractals are recursive. A traditional neural network is a universal function approximator, however it is not recursive in nature, unless it is some sort of RNN. The transformer architecture, which this seems fairly similar to this one, is also not recursive in nature; although I believe, limited recursion can come about through CoT. Therefore, I don't believe this could match the reproducibility, in an infinite sense, of a traditional procedural generator.

jrvieira 2y ago
"Fractals are not self-similar" https://m.youtube.com/watch?v=gB9n2gHsHN4 https://m.youtube.com/watch?v=gB9n2gHsHN4 If you want to learn more about the fascinating world of fractals.
- falcor84 2y ago
  The parent comment didn't mention self-similarity; did you maybe intend to reply to another comment?
jerf 2y ago
Procedurally generated terrain is whatever it wants to be. It isn't necessary for it to be fractal in any particular sense. Or even arguably all that useful, at least at the present time.
astrobe_ 2y ago
Luanti [1] (Minetest) has a fractal map generator ("mapgen"). You can test it for yourself. It's funny at first but becomes eventually boring. Its other mapgens massively some kind of Perlin noise in various ways, so that you can have "realistic" landscapes (e.g. the Carpethian mapgen) or landscapes with impossible mountains and floating rocks sometimes (e.g. the V7 mapgen) that are good for fantasy/sci-fi worlds. Noise is a pretty efficient way to fake complexity, and it's not a coincidence [2]. [1] https://www.luanti.org/ https://www.luanti.org/ [2] https://en.wikipedia.org/wiki/Kolmogorov_complexity https://en.wikipedia.org/wiki/Kolmogorov_complexity

jerf 2y ago

"Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world?" That's Minecraft. It does work. And when you turn around, then turn back to where you were facing, you don't feel like you're in a fever dream, because the landscape hasn't completely shifted. The tech is neat and I'm sure worth publishing. The rhetoric accompanying it is terribly overblown. Of course that's as likely to be their University PR department as the authors.

astrobe_ 2y ago

> Wouldn't a working approach be to just create a really low resolution 3D world in the traditional "3D game world" sense to get the spatial consistency. Then this crude map with attributes is fed into frame generation to create the resulting world? If you go "ultra-low" resolution, that's basically Minecraft or Luanti (formerly Minetest). There some mods for Luanti that let one generate a less "squary" world by adding slopes, but that's still heavily polygonal. > It wouldn't be infinite, but on the other hand no one has a need for an infinite world either. Minecraft has an infinite world (with glitches when you go very far from origin due to floating point errors). Luanti's world is finite (around 64000 m^3 because 16 bits coordinates; players move at 2-4m/s usually), some people are working to make it infinite. In my opinion, you are right that nobody needs an infinite world; the argument for an infinite world is that a finite world can be a problem on popular servers but in my eyes servers where this is a problem don't use enough the available space (in particular the huge vertical space that can be used to create multiple worlds) or shoot themselves in the foot by providing players with ways to move fast (mainly transports); increasing the player's speed actually shrinks the world. I think there is a conundrum from the players and gameplay perspective. On one hand, you want a lot of space, but on the other hand you want to be close to other players (for commerce, play together, etc.). It's not enough to have an infinite world, you also have to have the gameplay that goes with it. Another thing to consider are the interaction of mobiles (NPCs, enemies) with the terrain. Because nobody likes an infinitely empty world either. I haven't played an AAA game recently, but a decade ago even with precomputed terrain, some mobiles would look silly sometimes with steep terrains.