4 ms·

This is amazing. Does anyone have any insight on what makes the back view possible?

dogma1138 6y ago
I’m guessing this was trained on images and their 3D scanned models so it kinda fits the unseen parts of the body. I can nearly guarantee you that if you try to to use an image with something on the back it won’t be able to fit it, backpacks don’t work and probably something like the Victoria’s Secret angel wings won’t either. You can also see where it really fudges things like the shoes on some of their examples, their examples are also the best case scenario (similar to the StyleGAN2 results) and not necessarily representative of what the model achieves regularly. I’ve tried using some pictures of my partner in order to 3D print her but it didn’t really work well I’m guessing you need a really clean background.
- dorkwood 6y ago
  Make sure you use a long focal length, too. A normal wide angle photo from your camera phone won't produce very desirable results. This guy on Twitter did some of his own tests and they came out pretty good. https://mobile.twitter.com/Yokohara_h/status/1272396712594702342 https://mobile.twitter.com/Yokohara_h/status/127239671259470...
swframe2 6y ago
It requires a video and the person needs to rotate. It doesn't imagine the back view. It appears to be doing optical flow and photogrammetry.
- makeworld 6y ago
  The site clearly shows nice 3D outputs created from a single static image. Even the video models are created from a single frame alone.
- TaylorAlexander 6y ago
  It is not doing photogrammetry and it works on a single image. It has learned what people look like (and training may have used photogrammetry in some way) and it can imagine a good 3D model from even a single image. The last line of the abstract: “We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images.”

I’m guessing this was trained on images and their 3D scanned models so it kinda fits the unseen parts of the body. I can nearly guarantee you that if you try to to use an image with something on the back it won’t be able to fit it, backpacks don’t work and probably something like the Victoria’s Secret angel wings won’t either. You can also see where it really fudges things like the shoes on some of their examples, their examples are also the best case scenario (similar to the StyleGAN2 results) and not necessarily representative of what the model achieves regularly. I’ve tried using some pictures of my partner in order to 3D print her but it didn’t really work well I’m guessing you need a really clean background.

dorkwood 6y ago
Make sure you use a long focal length, too. A normal wide angle photo from your camera phone won't produce very desirable results. This guy on Twitter did some of his own tests and they came out pretty good. https://mobile.twitter.com/Yokohara_h/status/1272396712594702342 https://mobile.twitter.com/Yokohara_h/status/127239671259470...

swframe2 6y ago

It requires a video and the person needs to rotate. It doesn't imagine the back view. It appears to be doing optical flow and photogrammetry.

makeworld 6y ago
The site clearly shows nice 3D outputs created from a single static image. Even the video models are created from a single frame alone.
TaylorAlexander 6y ago
It is not doing photogrammetry and it works on a single image. It has learned what people look like (and training may have used photogrammetry in some way) and it can imagine a good 3D model from even a single image. The last line of the abstract: “We demonstrate that our approach significantly outperforms existing state-of-the-art techniques on single image human shape reconstruction by fully leveraging 1k-resolution input images.”