I think that's a really good question. IMO this is an essential part of any pipeline aiming to do that, but there's a bunch of hurdles with neural networks that

I think that's a really good question. IMO this is an essential part of any pipeline aiming to do that, but there's a bunch of hurdles with neural networks that's haven't been crossed yet. The first one that comes to mind is that I'm not aware of any work which would try to teach neural networks to imagine what someone would look like from different angles (aside from relatively restricted settings like facial frontalization). Second, you talk of videos. That means the person needs to look 'consistent' from frame to frame. This isn't a solved problem either, especially when it comes to GANs. I'm sure there are a few other areas that need to be solved.