5 ms·
Infinite Nature: Perpetual View Generation of Natural Scenes from a Single Image
- crispyambulance 6y agoWow, these images are very dreamlike and not just from the flying POV. There's something fascinating about the familiar but nonsensical details. Who says a CPU can't go on an acid trip?
- h0l0cube 6y ago> familiar and nonsensical You might like this analysis of the phenomena https://www.youtube.com/watch?v=0F7XBwFwA-M&vl=en https://www.youtube.com/watch?v=0F7XBwFwA-M&vl=en
- mistercow 6y agoI think the fact that it has short-term but not long-term temporal coherence is part of what gives it that surreal quality. A range of any five frames looks like a plausible view shift, but over longer timeframes, it looks like details are appearing out of the horizon. Still really cool though.
- gamer9 6y agolooks amazing, can be very useful for procedure generated game world.
- nmca 6y agoLots of issues with long-term consistency in these, but I expect now the dataset is out those shouldn't take too long to resolve.
- dhsysusbsjsi 6y agoAgree RE long term consistency. After 10 frames I suspect what I’m seeing from that point is a journey through the trained model internals. It doesn’t seem to reference back to the original image and very quickly changes to vastly different terrain. Still impressive!
- aniijbod 6y ago"very quickly changes to vastly different terrain" I anticipate potential use-cases ('level/scene generation' in game content creation) where this actually turns out to be the most valuable aspect of the output. I can't imagine that it would be too much of a stretch to 'start with a photo' as a 'seed' and use it to create a 3D world. My only concern would be 'sameyness tendency', in other words, the problem would be that a variety of seed photos of similar kinds of scenes would end up creating worlds that were too similar to one another to make 'still photo to fly-through movie to navigable detailed 3D world' a sufficiently 'creative' source of sufficiently differentiated new game worlds (or of sufficiently differentiated new fly-through movies). I suspect this could all be fixed by 'differentiator algorithms' interfering with the key variables of the 'refinement' process, as well as broadening and continually adding to the range of movies in the training data.
- andybak 6y agoThis is the most impressive attempt to tackle temporal consistency that I've come across: https://arxiv.org/pdf/2007.08509.pdf https://arxiv.org/pdf/2007.08509.pdf https://www.youtube.com/watch?v=rlCh6-2NfSg https://www.youtube.com/watch?v=rlCh6-2NfSg
- nmca 6y agoThanks, that's excellent!
- kingkawn 6y agoEnhance.
- yowlingcat 6y agoWow. Every day, we get closer to the eponymous electric dreams of androids. Personally, I can't wait to use something like this in a music video.
- 127 6y agoThere's already a service for that https://wzrd.ai/ https://wzrd.ai/
- yowlingcat 6y agoWell, I'll be. Combine that with green-screen technology, and I don't really have any excuse to not record myself and set myself to video for my favorite compositions. Thanks for putting this on my radar, friend.
- dwighttk 6y agoWow. Huge step forward from competitors and also wow there’s a lot more to do.
- xwdv 6y agoGod damn, John Carmack was right, someday we’ll be able to take footage from TV shoes and just walk around in scenes in VR.
- TimTheTinker 6y agoReminds me of weird dreams I’ve had.
- behnamoh 6y agoThis has huge applications in video games! I can't wait for game developers to implement a truly infinite world with this.
- pbhjpbhj 6y agoFor what sort of games? There's a few ways I see procedural generation as worse than limited [human] curated modelling - you lose the very careful balance, the iterated architecting to enhance game play, you also lose the commonality of experience across players, and if the game is often played you lose an aspect of the player's ability to become expert at the game. It's natural that game companies will use lots of procedural/ML generation of game aspects; but it's probably going to be "if you have a hammer, everything looks like a nail". Television had a certain manner of aiding societal cohesion when we only had a few channels (3 when I was little). That meant lots of commonality of experience. It seems [some] games could go the same way - players getting such vast differences of experience that we lose some of that shared experience that binds people together. Ultimately, a future of us all living in our own entirely unique VR realty seems like some version of hell.
- techer 6y agoFlashbacks to VistaPro on my Amiga. (Not mine) https://m.youtube.com/watch?v=48KtJtDqkaM https://m.youtube.com/watch?v=48KtJtDqkaM
- rajveermalviya 6y agoWow. I am guessing this will be used in Google Maps Street View?
- lupire 6y agoWhy? Maps has actual data to show. Fake data isn't needed or helpful.
- fab1an 6y agoReally cool. One thing the most boring part of my mind is wondering about when seeing these is related to the legality of stuff like that: many of the drone videos this is trained on are obviously copyrighted, and the system has used them to learn how to come up with novel ones. The same applies to GPT-n, which is trained on massive amounts of copyrighted material, no? Does anyone know anything about this; there must be a couple of dissertations on this topic by now?
- mpoteat 6y agoA much more salient and important point is generative models relationship to music. Right now, covers are legally protected as derivative works. I predict commercial usage of generative music to avoid royalties within the next decade. Philosophically, I think that the process of learning - whether by a human or simulacrom - should be protected. That is, if I write a book, I don't need to seek the permission or pay the authors of every book I've ever read. To me, this is the only sane result for a non-dualist materialist who believes neural networks are in principle isomorphic to human minds. As a matter of law, I believe it will have to be determined on the case law level some point. I'm not a lawyer and this is not legal advice.