3 ms·

Why not? I see just two deceptions: 1. You can only choose from pre-build characters (the poster-style thumbnails of the characters implies something different

Why not? I see just two deceptions: 1. You can only choose from pre-build characters (the poster-style thumbnails of the characters implies something different) 2. I don't think you can actually move the camera (or just wiggle it a bit) They just carefully chained a bunch of things: - segmentation to isolate character - in-paint for character removal and a 360 HDR photo sphere for lighting - pose estimation (and something equivalent for the face, but that was a bit weak I felt) - estimate camera position from tracking points on the floor - render hand built 3D model from poses and with light sphere So the shortcomings for now are: - you can only use pre-build humanoid 3D models - you still need to film it with real actors - face tracking doesn't seem that good yet - lighting is semi-convincing - no interaction with the environment (not even shadows) - some visual artifacts from the character removal (I think mostly segmentation failing and hairs getting out of bounds, you can see it in the last shot)

eps 4y ago
At 00:37 robot's hand is way thinner than the original actor's, but the missing parts of the background are perfectly reconstituted. I realize that Photoshop could do this few years ago, but here it looks a bit too flawless.
- anotheryou 4y ago
  why is it "too flawless" if photoshop can do it for years :) ? You can see flaws if you look at the head around 1:56 to 1:58
  - appstorelottery 4y ago
    If you look closely at the "Clean Plate" demo right at the end, there is a shimmering where the character was removed. This isn't a criticism of the product, because I certainly can't do better myself - and they might be pushing the realms of what is possible right now... so kudos, it's amazing stuff. But it's not perfect, and perhaps it just has to be "good enough" for the target market (which based on previous comments is not big VFX houses with the resources to create their own internal tools).
- pbhjpbhj 4y ago
  Perhaps I've misunderstood, but it looks to me like the actors are green-screened into a [cg] scene. I don't think the scene is real actors in a real setting, then modified to replace actors with characters. Thus, there's no problem with the in-painting, the whole actor is on a different layer? I'm seeing it in my phone. Like I said, maybe I misunderstood, but the scenes didn't seem like real places (eg shot cropped so foot-shadows aren't included, as this is hard to get right in the composition).

eps 4y ago

At 00:37 robot's hand is way thinner than the original actor's, but the missing parts of the background are perfectly reconstituted. I realize that Photoshop could do this few years ago, but here it looks a bit too flawless.

anotheryou 4y ago
why is it "too flawless" if photoshop can do it for years :) ? You can see flaws if you look at the head around 1:56 to 1:58
- appstorelottery 4y ago
  If you look closely at the "Clean Plate" demo right at the end, there is a shimmering where the character was removed. This isn't a criticism of the product, because I certainly can't do better myself - and they might be pushing the realms of what is possible right now... so kudos, it's amazing stuff. But it's not perfect, and perhaps it just has to be "good enough" for the target market (which based on previous comments is not big VFX houses with the resources to create their own internal tools).
pbhjpbhj 4y ago
Perhaps I've misunderstood, but it looks to me like the actors are green-screened into a [cg] scene. I don't think the scene is real actors in a real setting, then modified to replace actors with characters. Thus, there's no problem with the in-painting, the whole actor is on a different layer? I'm seeing it in my phone. Like I said, maybe I misunderstood, but the scenes didn't seem like real places (eg shot cropped so foot-shadows aren't included, as this is hard to get right in the composition).