6 ms·
WorldGrow: Generating Infinite 3D World
- deleted 11mo ago[deleted]
- jackdoe 11mo agocant wait for the new diablo :)
- speedgoose 11mo agoIt looks more like the Stanley parable.
- pjmlp 11mo agoWith a quarter the size of the development team, 'cause productivity!
- embedding-shape 11mo agoIt is only a paper as of now: > The code is being prepared for public release; pretrained weights and full training/inference pipelines are planned. Any ideas of how it would different and better compared to "traditional" PCG? Seems like it'd give you more resource consumption, worse results and less control, neither of which seem like a benefit.
- tantalor 11mo agoAn unpublished paper.
- glenneroo 11mo agoThe description in the linked YouTube video for some reason has more info than the github repo: > We tackle the challenge of generating the infinitely extendable 3D world — large, continuous environments with coherent geometry and realistic appearance. Existing methods face key challenges: 2D-lifting approaches suffer from geometric and appearance inconsistencies across views, 3D implicit representations are hard to scale up, and current 3D foundation models are mostly object-centric, limiting their applicability to scene-level generation. Our key insight is leveraging strong generation priors from pre-trained 3D models for structured scene block generation. To this end, we propose WorldGrow, a hierarchical framework for unbounded 3D scene synthesis. Our method features three core components: (1) a data curation pipeline that extracts high-quality scene blocks for training, making the 3D structured latent representations suitable for scene generation; (2) a 3D block inpainting mechanism that enables context-aware scene extension; and (3) a coarse-to-fine generation strategy that ensures both global layout plausibility and local geometric/textural fidelity. Evaluated on the large-scale 3D-FRONT dataset, WorldGrow achieves SOTA performance in geometry reconstruction, while uniquely supporting infinite scene generation with photorealistic and structurally consistent outputs. These results highlight its capability for constructing large-scale virtual environments and potential for building future world models.
- embedding-shape 11mo agoThat seems to compare to other similar "generating infinite 3D worlds" approaches, but not to traditional PCG, which would give you all of that except higher quality, better performance and more/better control.
- Garlef 11mo agoI don't think generating virtual space is the issue. It's about generating interesting virtual space!
- james-bcn 11mo agoYep. People have been doing this kind of stuff for computer games for decades. It's actually not that difficult. It's not clear what novel problem is being solved here.
- jsheard 11mo agoYeah but those traditional procgen techniques don't use AI, and this one does use AI. They solved the problem of them not being AI enough for the AI era. AI!
- agravier 11mo agoDo you have some particular piece of software or tech demo or game in mind with interesting very large generated 3D worlds?
- sirtaj 11mo agoValheim and No Man's Sky are ones I've played recently.
- SiempreViernes 11mo agoIn Mario 64 there is a staircase you can run up forever, granted it looks the same no matter how long you have Mario run up the stairs, but that certainly fits "big but uninteresting 3d world."
- bogwog 11mo ago> big but uninteresting 3d world. I know 'interesting' is subjective, but your comment is demonstrably false. Just type "mario 64 staircase" into youtube, and look at the hundreds (thousands? millions?) of videos and many millions of views.
- rana762 11mo ago[dead]
- gcr 11mo agoThis could be a great way to make backrooms horror environments! I've dreamed of a NeRF-powered backrooms walking simulator for quite a while now. This approach is "worse" because the mesh seems explicit rather than just the world becoming what you look at, but that's arguably better for real-world use cases of course.
- grumbelbart2 11mo ago> backrooms horror environments True, it sounds (and looks) a lot like https://scp-wiki.wikidot.com/scp-3008 https://scp-wiki.wikidot.com/scp-3008
- gpderetta 11mo agoThat SCP was literally the first thing that came to my mind when looking at the intro video!
- endymion-light 11mo agoi'm thinking a new version of LSD dream emulator could be really interesting.
- fjfaase 11mo agoI wonder if they also have a strategy for deleting generate tiles, otherwise the infinite is limited to the size of available memory. I also wonder if with their method can exactly recreate tiles that have been deleted. Or in other words, that they have a method for generating unique seeds for all tiles. The paper does not give much technical details. If the seed has a limited size and there is a method for generating seeds for each 2D coordinate, I wonder if it is possible to make a non-repeating infinite world. I think it is not possible with a limited size seed.
- keyle 11mo agoThis is cool. And could be fun in games. Not sure I get the point otherwise... The thought that came to mind was "Architectural slop".
- flohofwoe 11mo agoGames have used procedural world generation since at least the 1980s, and on 8-bit home computers. Glancing through the video and webpage, the results don't look much different from what's possible with traditional Wave Function Collapse tbh.
- splintercell 11mo agoIs it just me, or some of the places it generates are just not realistic? Like a small area of some kind which is a dead space, and there is a giant window into it.
- oniony 11mo agoNot only not realistic but also not explicit: not so much as a peachy bottom in sight.
- icoder 11mo agoIt's not just you. The generated stuff - in my opinion - doesn't make any sense at all, with regard to structure or meaning. Unless, perhaps, the aim was to generate some kind of badly designed Ikea store.
- hobofan 11mo agoYeah, I think either the method doesn't work well, or there is something off with their tuning. Their block-by-block generation method seems to be too local in its considerations, where each 3x3 section (= the ones generated based on the immediate neighbors) looks a lot more coherent than the 4x4 sections and above. I think it might need to be extended to be less local and might also in general need to be paired with some sort of guidance systems (e.g. in the office example would generate the overall floor layout).
- theknarf 11mo agoDid they reinvent "wave function collapse" (https://github.com/mxgmn/WaveFunctionCollapse https://github.com/mxgmn/WaveFunctionCollapse)?
- fallat 11mo agoNo. WFC is fundamentally different from this.
- ivanjermakov 11mo agoWFC?
- nikolay 11mo agoWFC = Wave Function Collapse
- swiftcoder 11mo agoIndeed, but it does serve more or less the same purpose in procgen pipelines (and folks have tweaked WFC for infinite worlds before[1]). [1]: https://www.youtube.com/watch?v=7ffT_8wViBA https://www.youtube.com/watch?v=7ffT_8wViBA
- mock-possum 11mo agoWhoa this is awesome, thanks for the link!
- kittikitti 11mo agoThis is great and really cool! Thank you for sharing.
- tantalor 11mo agoOh great, it's the Severance simulator.
- deterministic 11mo agoThe problem is not generating worlds. The problem is generating interesting worlds.