8 ms·
Infinite Photorealistic Worlds Using Procedural Generation
- dmarchand90 3y agohttps://infinigen.org/ https://infinigen.org/
- dmarchand90 3y agoDemo video is very impressive. I thought it was yet another GAN, but, no. It's something "mathy". I'll need to read the paper for sure!
- mnky9800n 3y ago> Zero ai I gave a standing ovation when that popped up in the video.
- nine_k 3y ago"Infinigen is free and open source". What a gift. Gotta love it.
- schemescape 3y agoNow we just need to make a No Man’s Sky-style—but open source—universe with this.
- addandsubtract 3y agoNo Man's License
- prox 3y agoHow about a planet that runs on a server and is part of a federated universe basically. Each server has a solar system to play with. Then add spaceships so you can actually travel between them. Godot runs in the browser these days, so that could work.
- elif 3y agoI think no man's sky proved conclusively that we don't need more no man's sky. One was already too much no man's sky.
- malermeister 3y agoWell there's a second one coming soon with Starfield
- dgellow 3y agoWhat do you mean? The release didn’t go too well (mainly due to unrealistic expectations) but the game is still updated and has a healthy community.
- schemescape 3y agoAs in the universe was too big or as in the game was no good? I think one could make a case for the former, but the latter is subjective (I liked the game, although it is a bit shallow).
- dopidopHN 3y agoIt’s random, sorry. But I looked at it briefly and I saw jargon that I had difficulty picking up. What’s baseline is needed to operate that stuff with meaningful control. ( I’m confident any dev can run it and have fun )
- madacol 3y agohttps://www.youtube.com/watch?v=6tgspeI-GHY https://www.youtube.com/watch?v=6tgspeI-GHY
- johnea 3y ago[flagged]
- Xen9 3y agoProcedural generation, not transformers—truly infinite! Of course AI input would work well with this...
- FpUser 3y agoThis is very nice. Thinking of playing with it
- jumpkick 3y agoLove the "Matrix" zoom-in on the green terminal text.
- ilrwbwrkhv 3y agoCan't wait for my vision pro
- vsskanth 3y agoSomething feels odd about the fire animation. Water flowing also feels a bit odd but I am not able to say how.
- Animats 3y agoThis is great. Terrain generators have been around for decades, but this is a nice open-source one. " The wall time to produce a pair of 1080p images is 3.5 hours." Ouch. It's in Python, but still...
- justinclift 3y agoYeah, it's not clear if the wall time needed is due to Python being slow or not. eg could it be redone in a compiled language for a better result I'd kind of expect that most of the core stuff is being done in C++ or similar behind the scenes though. Maybe (haven't looked). :)
- Animats 3y agoConsidering that part of the code is in CUDA and part is in C/C++, there's already been some optimization.
- oersted 3y agoThey Python part just generates Blender procgen specifications (compute graphs), that is probably relatively fast. But apparently the output graphs are huge and unoptimized, so generating the geometry takes a lot of resources (CPU bound). Rendering might also take a while if they don't have good ways to limit detail (something like Nanite would come in very handy here).
- Animats 3y agoWith Nanite, if you have a sidewalk with occasional cracks, and all the cracks are the same, those get combined into a single submesh. This is recursive; if you have multiple instances of that sidewalk, those get combined at a higher level. It's a form of compression for mesh data with redundancy. This only works if the data contains such redundancy. That depends on how it's generated. Not clear if this type of generator creates such redundancy. Something using random processes for content generation won't do that. Nanite is optimized for large areas of content generated by a team of people working together, as in a major game project. There's a lot of asset reuse. Nanite exploits that. It's not a good fit for randomly generated assets, or for assets generated by a large community, as in a serious metaverse. The redundancy isn't there to be exploited.
- avaer 3y agoI ran this for most of today in the background and I have some thoughts: The quality is good and it's giving you all of the maps (as well as the .blends!). It seems great for its stated goal of generating ground truth for training. However, it's very slow/CPU bound (go get lunch) so probably doesn't make sense for applications with users behind the computer in the current state. Additionally, the .blend files are so unoptimized that you can't even edit them on a laptop with texturing on. The larger generations will OOM a single run on a reasonably beefy server. To be fair, these warnings are in the documentation. With some optimization (of the output) you could probably do some cool things with the resulting assets, but I would agree with the authors the best use case is where you need a full image set (diffuse, depth, segmentation) for training, where you can run this for a week on a cluster. To hype this up as No Man's Sky is a stretch (NMS is a marvel in its own right, but has a completely different set of tradeoffs). EDIT: Although there are configuration files you can use to create your own "biomes", there is no easy way to control this with an LLM. Maybe you might be able to hack GPT-4 functions to get the right format for it to be accepted, but I wouldn't expect great results from that technique.
- justinclift 3y ago> ... on a reasonably beefy server. Out of curiosity, what kind of cpu / ram are you meaning here? Asking because I have some spare hardware just sitting around, so am thinking... :)
- p1esk 3y agoA typical server would be 8xA100 with a dual CPU (128 cores total), 2TB of RAM. I doubt you have something like this sitting around.
- justinclift 3y agoThat's only a "typical server" for some companies with specialised needs. Lots of places have servers with 128-256GB of ram around though.
- terrycody 3y agoThink of a real world scale sandbox gaming with Pokémon elements...must be a crazy game.
- ukuina 3y ago"Zero AI" will likely become a selling point.
- omoikane 3y agoI worked on a thing many years ago that involved machine learning, which usually produced reasonable results but all users hated it nonetheless, because machine learning made it completely opaque. The correct predictions it made were mostly acceptable, but the incorrect predictions it made were hilariously bad, and in both cases nobody could explain why it generated those outputs. Eventually we concluded that machine learning wasn't a good fit for our problem, and our users were very keen to maintain that conclusion.
- noduerme 3y agoI'm thinking of a very complex logistics system I wrote, that had to trace millions of possible paths to optimize a route. Even when the range of choices is too extensive to present to the user directly, and you need to resort to a list of best choices, it's indispensable to show somehow how the logic was arrived at and present ways of disabling portions of what went into the deductive process. That's something machine learning simply isn't geared towards, because the reasoning doesn't rest on reproducible sets of hierarchical rules.
- acutesoftware 3y agoThis is really impressive, even the trees look pretty good which appears to be tricky to do with procgen stuff. A nice next step or addition would be to take the results and remesh them to lower poly models so it can be used in a game engine to walk around in.
- peter_d_sherman 3y ago"Introducing Infinigen": https://www.youtube.com/watch?v=6tgspeI-GHY https://www.youtube.com/watch?v=6tgspeI-GHY
- userbinator 3y agoProcedural generation is something the demoscene has essentially specialised in for several decades, but I see no mention of it in the article. The demoscene has also done so using several orders of magnitude less computing resources than mentioned here, so I think everyone else has a lot to learn from them. Here's a memorable intro, from 2009: https://news.ycombinator.com/item?id=31636482 https://news.ycombinator.com/item?id=31636482 Earlier than that, another famous 64k from 2000: http://www.theproduct.de/index.html http://www.theproduct.de/index.html
- elif 3y agoThis is an academic paper not an article. Generously 0.01% of demos generate landscape scenes, photorealistic or not.. so it is clear why the paper would not rely upon them for inspiration.
- laurentlb 3y agoWhat they are doing has clear overlap with what we've seen in the demoscene. On a technical point of view, they generate all meshes and textures from formulas, using the same kinds of algorithms. See for example some 64kB demos that have landscapes / nature: - Paradise by Rgba, 2004 https://www.pouet.net/prod.php?which=12821 https://www.pouet.net/prod.php?which=12821 - Gaia Machina by Approximate, 2012 (https://www.pouet.net/prod.php?which=59107 https://www.pouet.net/prod.php?which=59107) - Turtles all the way down by Brain Control, 2013 (https://www.pouet.net/prod.php?which=61204 https://www.pouet.net/prod.php?which=61204) Of course, it's not exactly the same, the demoscene tends to focus on a few carefully crafted scenes and do everything in real-time (with up to 30s of precalc), but I think there's a lot of similarity.
- virtualritz 3y agoProcedural generation in computer graphics starts in the 70s. The demo scene was already standing on the shoulders of giants.
- nickdothutton 3y ago
- taneq 3y agoThe name’s reminiscent of Terragen, I wonder if the author is involved in any way?
- HellDunkel 3y agoNone of this looks "photorealistic". The creatures look hilarious and the paper is not well written either. "Each part generator is either a transpiled node-graph, or a non-uniform rational basis spline (NURBS). NURBS parameter-space is high-dimensional, so we randomize NURBS parameters under a factorization inspired by lofting, composed of deviations from a center curve. To tune the random distribution, we modelled 30 example heads and bodies, and ensured that our distribution supports them." This strikes me as a fairly random approach. No wonder why those creatures look the way they do. I fail to see why this is worth a scientific paper as it appears to be no more than a student project with a number of contributors across different fields. Building a (somewhat) procedually based asset library has been done countless times before by game dev studios big and small.
- prox 3y agoSpore, No Man’s Sky, Elite Dangerous all do galaxy generation to good effect. Elite is probably the most realistic. No Man’s Sky has creatures like Spore.
- grumbel 3y agoSpace Engine[1] is another popular one [1] https://spaceengine.org/ https://spaceengine.org/
- thumbuddy 3y agoComplete opposite reaction for me here. Interesting to see how two people can see the same thing and think polar opposites.
- beaker52 3y agoThe infinite virtual reality escape comes nearer
- ChrisMarshallNY 3y agoReminds me of the Mac Aquarium SereneScreen screensaver. That was around for decades. https://m.youtube.com/watch?v=Ws1zpk9GWkk https://m.youtube.com/watch?v=Ws1zpk9GWkk Much, MUCH more limited, but pretty cool.
- laurentlb 3y agoInteresting - they've just changed the license from GPL to BSD: https://github.com/princeton-vl/infinigen/commit/258cf38e8601dc1d010dd4789eb322a0e40d26d3 https://github.com/princeton-vl/infinigen/commit/258cf38e860... This makes the repo much more useful to me.
- anthk 3y agoI'd love this generating images for a Don Woods' Adventure version. Or for Nethack/Slashem.
- pk-protect-ai 3y agoAbsolutely amazing. The holly grail of shaders.
- deleted 3y ago[deleted]
- jcelerier 3y agoI can't help but find this so much more interesting and fun to use than anything AI-based