4 ms·
You'll notice the effect isn't as good at about 7 seconds into the movie where the green grass between the bride and her maids moves with the bride. Why does t
by anon1m0us 7y ago
You'll notice the effect isn't as good at about 7 seconds into the movie where the green grass between the bride and her maids moves with the bride.
Why does the algorithm think that is part of her, rather than the background?
- sniklaus 7y agoEstimating the scene geometry from a single image is highly challenging and remains an unsolved problem. As such, you can find subtle artifacts like this in most of the results. The artifact that you are referring to stems from a geometry adjustment step in which we make sure that salient objects like humans or animals are on a plane in the 3D space. This requires image segmentation which is also highly challenging and may lead to parts of the background being assigned to the segmented foreground object and vice versa.
- anon1m0us 7y agoCould you not use something like the "magic wand" selection tool in image editing programs? Doesn't the dramatic color difference indicate it's not part of the same object? I thought maybe it grabbed it because it was the same color as the bouquet she is holding, but that is entirely within the white of her dress.
- sniklaus 7y agoThere definitely is room for improvement. Our proposed framework uses Mask R-CNN for object segmentation. However, Mask R-CNN only as a resolution of 28x28 pixels for the object masks if I remember correctly (which is then resized to the actual resolution of the object in the image). So there are a lot of limitations from that alone.
- pedrocx486 7y agoThe two examples involving humans look terrible to me, the others look really nice. Probably some bias on my perception.
- sniklaus 7y agoYeah, we just are good at seeing artifacts in human renderings, it can easily look wrong or uncanny. In contrast, if a boulder is rendered geometrically wrong we usually do not notice it.
- gbrown 7y agoIt's people moving up or down on the screen relative to one another in a way which their relative distances wouldn't dictate. Agreed that it gives me an uncanny-valley feeling (still super cool though).