6 ms·

Lucasfilm's subsidiary Industrial Lights and Magic is known for leading visual effects on actors's appearances to help make films that cannot be realized withou

Lucasfilm's subsidiary Industrial Lights and Magic is known for leading visual effects on actors's appearances to help make films that cannot be realized without such technology. For instance, Martin Scorsese entrusted them to "youthen" the leading actors in "The Irishman" so that Robert Deniro et al can play their characters at a younger age without wearing a red-ball "clown-nose" tracker. "Star Wars: Rouge One" practically reanimates Peter Cushing to continue and expand the involvement of Grand Moff Tarkin in the Star Wars saga. These processes are painstaking, and artists sweat over details on a frame-by-frame basis to negate the "uncanny valley" of artificial human likenesses. Obviously, the labor-intensive nature of today's CGI techniques drive up production costs. Meanwhile, the deepfakes on YouTube provide a convincing enough rendition of likenesses without actual actors, all produced on consumer-level GPUs. This presents a huge potential to save costs and the benefits are clearly enticing to film productions. As Hollywood gravitates towards blockbuster franchises, productions will want to bring the same ensemble of actors (or at least their likenesses) as long as possible. While moviegoers may be unsettled by seeing "reanimated" dead actors like in Rouge One, they still may hope to see franchise actors to look consistently youthful or attractive on screen. Deepfakes may be more relied upon to provide that effect.

shadowtree 5y ago
The Irishman is the perfect example of where the Deepfake is MILES BETTER than then classic CGI de-aging: https://www.youtube.com/watch?v=dHSTWepkp_M https://www.youtube.com/watch?v=dHSTWepkp_M Just look at it, the CGI DeNiro looks like from the Polar Express.
- dualboot 5y ago
  This is deceptive, though. The "deep fake" is essentially building on the CGI de-aged product. If they'd started with deepfake it likely wouldn't have yielded this level of result.
  - burnte 5y ago
    I disagree, deepfaking completely replaces the face. It doesn't ahve to be close to start with. Check this one out: https://www.youtube.com/watch?v=861gfPVmgdc https://www.youtube.com/watch?v=861gfPVmgdc
    - ad404b8a372f2b9 5y ago
      It doesn't have to be close but it helps a lot both the quality of the results and the convergence speed if both faces are similar. In movies you don't usually have the luxury of choosing the original face you want to replace, but for people who make memes (or porn), you commonly choose a source video featuring someone that resembles the person you want to put in.
      - seph-reed 5y ago
        > In movies you don't usually have the luxury of choosing the original face you want to replace Hiring an actor that roughly resembles the person you're trying to deep fake seems doable.
  - 411111111111111 5y ago
    Uuuh, why? I can't imagine a reason why the deaged video would've improved the deep fake version. If anything, it should've reduced it's quality by adding incorrect data which could confuse the model.
    - ad404b8a372f2b9 5y ago
      The default model for deepfakes is an autoencoder. The encoder and decoder will converge faster and to simpler solutions if the distributions of original faces and fake faces are closer. It's an intuitive result even ignoring the specific model. It takes less information to go to and from similar faces than it does two completely dissimilar ones.
  - vernie 5y ago
    I'm inclined to agree and I'd be interested to see this process applied to some of the raw footage featured in the the award season campaign. It's also worth noting that in addition to de-aging they also changed De Niro's eye color.
- kilroy123 5y ago
  Wow the deep fake version is WAY better.
- tjoff 5y ago
  It is? The deepfake just looks out of focus to me, just something blurry that as a side-effect removes some wrinkles but also takes away lightning and everything else. Not to say the deepfake isn't seriously impressive. But I very much prefer the netflix version.
- pen2l 5y ago
  Wait what, Deepfake is better? To me, not at all, deepfake seems blurry. It looks like, well, a deepfake, there is something distinct about deepfakes which just stands out, I think it's the soft blur around individual parts. It's taking the easy route by blurring and darkening a lot of things, look at the eyelids for example in modeled example, there is incredible detail there. Deepfake definitely has a place in this space, in that it can do things with 1/100th the effort in a scalable way, but it has a lot more to catch up on with traditional modeling than what some folks appear to be thinking. rendering with goodies likes SSS, AO, etc. gives magical results which are hard to achieve any other way. And as soon as you get a little bit complicated in what you're trying to create, at least the currently existing neural network models fall apart are are just not applicable. Take this video for example, which was very manually modeled: https://www.youtube.com/watch?v=BC2dRkm8ATU https://www.youtube.com/watch?v=BC2dRkm8ATU Deepfake is a long, long way to taking a stab at things like this.
  - castlecrasher2 5y ago
    >Wait what, Deepfake is better? To me, not at all, deepfake seems blurry. It looks like, well, a deepfake, there is something distinct about deepfakes which just stands out, I think it's the soft blur around individual parts. It's taking the easy route by blurring and darkening a lot of things, look at the eyelids for example in modeled example, there is incredible detail there. I agree with your complaints about DeepFake but imo it did a far better job de-aging DeNiro. To me, the release version had a lot of "old man" cues that the DeepFake one didn't, such as the jowls and heavy wrinkles.
  - jjeaff 5y ago
    On mobile, I don't see the blur on the deep fake. It looks really good. I suspect it would be more obvious on a large screen.
- deleted 5y ago
  [deleted]
- lttlrck 5y ago
  That does look amazing. But I'm not sure I should be seeing myriad facsimile of younger deniro... If Deniro appears as Scorsese wanted him to appear then the deep fake (that I do find subjectively better) is actually aiming at the wrong target. It seems like a flawed comparison.
ksec 5y ago
My question is, how would the cost structure works. Can I now hire an actor that looks 90% like someone I have in mind, and then Deepfake it to look 99.9%, and save on actor's cost? It works when you are trying to do that on actor that are no longer with us, but what about actors that are still alive?
- seanicus 5y ago
  Not a lawyer but you need likeness rights to reproduce an actor's face. I.e. Peter Cushing's estate gave permission for him to "appear" in Rogue One. But maybe it doesn't apply to parody(?) https://youtu.be/9WfZuNceFDM https://youtu.be/9WfZuNceFDM

shadowtree 5y ago

The Irishman is the perfect example of where the Deepfake is MILES BETTER than then classic CGI de-aging: https://www.youtube.com/watch?v=dHSTWepkp_M https://www.youtube.com/watch?v=dHSTWepkp_M Just look at it, the CGI DeNiro looks like from the Polar Express.

dualboot 5y ago
This is deceptive, though. The "deep fake" is essentially building on the CGI de-aged product. If they'd started with deepfake it likely wouldn't have yielded this level of result.
- burnte 5y ago
  I disagree, deepfaking completely replaces the face. It doesn't ahve to be close to start with. Check this one out: https://www.youtube.com/watch?v=861gfPVmgdc https://www.youtube.com/watch?v=861gfPVmgdc
  - ad404b8a372f2b9 5y ago
    It doesn't have to be close but it helps a lot both the quality of the results and the convergence speed if both faces are similar. In movies you don't usually have the luxury of choosing the original face you want to replace, but for people who make memes (or porn), you commonly choose a source video featuring someone that resembles the person you want to put in.
    - seph-reed 5y ago
      > In movies you don't usually have the luxury of choosing the original face you want to replace Hiring an actor that roughly resembles the person you're trying to deep fake seems doable.
- 411111111111111 5y ago
  Uuuh, why? I can't imagine a reason why the deaged video would've improved the deep fake version. If anything, it should've reduced it's quality by adding incorrect data which could confuse the model.
  - ad404b8a372f2b9 5y ago
    The default model for deepfakes is an autoencoder. The encoder and decoder will converge faster and to simpler solutions if the distributions of original faces and fake faces are closer. It's an intuitive result even ignoring the specific model. It takes less information to go to and from similar faces than it does two completely dissimilar ones.
- vernie 5y ago
  I'm inclined to agree and I'd be interested to see this process applied to some of the raw footage featured in the the award season campaign. It's also worth noting that in addition to de-aging they also changed De Niro's eye color.
kilroy123 5y ago
Wow the deep fake version is WAY better.
tjoff 5y ago
It is? The deepfake just looks out of focus to me, just something blurry that as a side-effect removes some wrinkles but also takes away lightning and everything else. Not to say the deepfake isn't seriously impressive. But I very much prefer the netflix version.
pen2l 5y ago
Wait what, Deepfake is better? To me, not at all, deepfake seems blurry. It looks like, well, a deepfake, there is something distinct about deepfakes which just stands out, I think it's the soft blur around individual parts. It's taking the easy route by blurring and darkening a lot of things, look at the eyelids for example in modeled example, there is incredible detail there. Deepfake definitely has a place in this space, in that it can do things with 1/100th the effort in a scalable way, but it has a lot more to catch up on with traditional modeling than what some folks appear to be thinking. rendering with goodies likes SSS, AO, etc. gives magical results which are hard to achieve any other way. And as soon as you get a little bit complicated in what you're trying to create, at least the currently existing neural network models fall apart are are just not applicable. Take this video for example, which was very manually modeled: https://www.youtube.com/watch?v=BC2dRkm8ATU https://www.youtube.com/watch?v=BC2dRkm8ATU Deepfake is a long, long way to taking a stab at things like this.
- castlecrasher2 5y ago
  >Wait what, Deepfake is better? To me, not at all, deepfake seems blurry. It looks like, well, a deepfake, there is something distinct about deepfakes which just stands out, I think it's the soft blur around individual parts. It's taking the easy route by blurring and darkening a lot of things, look at the eyelids for example in modeled example, there is incredible detail there. I agree with your complaints about DeepFake but imo it did a far better job de-aging DeNiro. To me, the release version had a lot of "old man" cues that the DeepFake one didn't, such as the jowls and heavy wrinkles.
- jjeaff 5y ago
  On mobile, I don't see the blur on the deep fake. It looks really good. I suspect it would be more obvious on a large screen.
deleted 5y ago
[deleted]
lttlrck 5y ago
That does look amazing. But I'm not sure I should be seeing myriad facsimile of younger deniro... If Deniro appears as Scorsese wanted him to appear then the deep fake (that I do find subjectively better) is actually aiming at the wrong target. It seems like a flawed comparison.

ksec 5y ago

My question is, how would the cost structure works. Can I now hire an actor that looks 90% like someone I have in mind, and then Deepfake it to look 99.9%, and save on actor's cost? It works when you are trying to do that on actor that are no longer with us, but what about actors that are still alive?

seanicus 5y ago
Not a lawyer but you need likeness rights to reproduce an actor's face. I.e. Peter Cushing's estate gave permission for him to "appear" in Rogue One. But maybe it doesn't apply to parody(?) https://youtu.be/9WfZuNceFDM https://youtu.be/9WfZuNceFDM