5 ms·
To quote the paper: > Our model is supervised with paired input and target views of a scene (along with their camera poses)... The model then needs just a sing
by btown 6y ago
To quote the paper:
> Our model is supervised with paired input and target
views of a scene (along with their camera poses)... The model then needs just a single image at test time.
Correct me if I’m wrong, but: Given a novel scene, it seems the model must be retrained on multiple images of that scene? It seems disingenuous, then, to say it works from a single image. No doubt the interpolation is state of the art, but the title seems misleadingly magical.
- kfarr 6y agoVery curious to understand this as well. Does that imply that images with similar attributes (like other hotel rooms in one of their examples with slightly different dimensions or furniture) will work without additional training? Or did each example require multiple image training of that scene before a 3d mesh can be computed from arbitrary angle? In the latter case it can still be useful as compared to photogrammetry which would require hundreds of photos to achieve similar results, so maybe this could work with a dozen or less?
- hijklmno 6y agoUnfortunately that's most(all) of academia!
- steerablesafe 6y agoI don't see how that would work with the paintings.
- svantana 6y agoI think you're misunderstanding the paper. They are using multiple images in the learning phase to infer the 3d structure. At test time, it works on novel scenes with just a single image.
- btown 6y agoI believe from a second reading this is correct - after training on the Matterport real estate image dataset, it appears they can transform novel images (which happen to be from that dataset, so they are still home interiors but can handle completely distinct scenes with distinct furniture etc.). Which is actually really fascinating and leverages decades of innovation. This is actually really cool and fully justifies the hype in the title.