5 ms·
Lots of issues with long-term consistency in these, but I expect now the dataset is out those shouldn't take too long to resolve.
by nmca 6y ago
Lots of issues with long-term consistency in these, but I expect now the dataset is out those shouldn't take too long to resolve.
- dhsysusbsjsi 6y agoAgree RE long term consistency. After 10 frames I suspect what I’m seeing from that point is a journey through the trained model internals. It doesn’t seem to reference back to the original image and very quickly changes to vastly different terrain. Still impressive!
- aniijbod 6y ago"very quickly changes to vastly different terrain" I anticipate potential use-cases ('level/scene generation' in game content creation) where this actually turns out to be the most valuable aspect of the output. I can't imagine that it would be too much of a stretch to 'start with a photo' as a 'seed' and use it to create a 3D world. My only concern would be 'sameyness tendency', in other words, the problem would be that a variety of seed photos of similar kinds of scenes would end up creating worlds that were too similar to one another to make 'still photo to fly-through movie to navigable detailed 3D world' a sufficiently 'creative' source of sufficiently differentiated new game worlds (or of sufficiently differentiated new fly-through movies). I suspect this could all be fixed by 'differentiator algorithms' interfering with the key variables of the 'refinement' process, as well as broadening and continually adding to the range of movies in the training data.
- andybak 6y agoThis is the most impressive attempt to tackle temporal consistency that I've come across: https://arxiv.org/pdf/2007.08509.pdf https://arxiv.org/pdf/2007.08509.pdf https://www.youtube.com/watch?v=rlCh6-2NfSg https://www.youtube.com/watch?v=rlCh6-2NfSg
- nmca 6y agoThanks, that's excellent!