2 ms·
But it's much trickier to find the point in a GAN's representation space for "the exact same fluffy pink earrings that the actress was wearing in the last scene
by londons_explore 5y ago
But it's much trickier to find the point in a GAN's representation space for "the exact same fluffy pink earrings that the actress was wearing in the last scene".
Mostly because one can't write code to judge equal-ness of pink earrings. Or the fact the trash can in the background had a lid in the last scene and is now missing the lid. Or the fact the brick wall has turned into a painted wall. etc.
Having said that, I'm sure some filmmakers will go for it anyway - after all, human-made films have some consistency issues, and many viewers simply don't care.
- notahacker 5y agoI think the main thing is it becomes a programming problem rather than an AI limitation. From what I've seen it's now possible for the AI to generate EarringsAsset57 and KitchenWall51 and then to ask the AI to draw the actor wearing EaringsAsset57 in front of Kitchenwall51 and for stills at least, it can produce acceptable results
- dougabug 5y agoYou can decompose an entire scene into a collection of 3D GANs, each of which is fit by GAN Inversion to the object instance which best explains what is seen in the images. Also, we absolutely can tell the difference between trash cans with and without lids, as well as brick walls vs painted walls. This falls under many areas of ML-based Computer Vision, such as metric, contrastive, and zero shot machine learning, also open world object detection / recognition. These frame level semantic discrepancies can be detected by discriminative DNNs, which we can use to provide a temporal consistency loss to guide the generator to produce temporally consistent videos.
- webmaven 5y ago> But it's much trickier to find the point in a GAN's representation space for "the exact same fluffy pink earrings that the actress was wearing in the last scene". Not so much, since you can supply an image of the desired item and get back a bunch of vectors to reuse.