8 ms·

Pinscreen Deepfake Live Prototype [video]

142 points by freediver 7y ago

lqet 7y ago
Before anyone is panicking, please remind yourself that whatever medium was used by humans as a form of communication in the last few thousand years, was also used for fakes. You only need a half-decent voice actor to pretend you are someone else on the phone. Fake photographs are as old as photography. Even the probably most immediate form of interpersonal communication, seeing another person, can be faked, which is demonstrated by the many Doppelgängers used by politicians in the past. The power of the house Habsburg basically had its roots in several imperial documents and bulls faked by Rudolf IV. in 1358 (which was only discovered in the 19th century). This isn't a new phenomenon that now means that we cannot trust anything anymore - it's only that the fakes are finally catching up with live video.
- prox 7y ago
  One rule there was in old times you always had 2 witnesses with you for any important event to vouch for. Reasoning was that it is a lot more difficult to do any trickery when more people are involved. Just because we scaled a lot of services in a digital network means our trust services need to adapt as well.
- numlock86 7y ago
  > You only need a half-decent voice actor to pretend you are someone else on the phone. You can even digitalize this part already. All you need are 5 seconds of recording and a pre-trained model in the desired language/accent. https://www.youtube.com/watch?v=0sR1rU3gLzQ https://www.youtube.com/watch?v=0sR1rU3gLzQ
  - saurik 7y ago
    Right... but the point is that you _don't_ need fancy neural networks, you _only_ really need someone with an actually not that uncommon skill.
    - numlock86 7y ago
      Yes. But I think there is another way to look at this: You _don't_ need fancy people with a not so uncommon skill, you _only_ need a bit of computing power. Point is: I totally agree! The entry level to fake something is really low. No matter the technical value or skills you bring to the table.
      - saurik 7y ago
        But the point of the comment being responded to--which I also think is a very important point that it feels like you missed--is this isn't some new issue, caused by computers. Zeitgeist: panic, "the concept of truth is being destroyed by advancements in computers!!!" OP: sigh, "no... you have always been able to trick people, such as with audio, using a trained voice actor" You: "but, even with audio, you can also trick people with fancy new computers!" Me: ?!?
        feanaro 7y ago
        The OP seems as if he understood that, but his argument is that this new technology makes the costs significantly lower. The lowering of cost has a strong impact in itself. In other words, almost no one would have bothered seeking out and then paying a talented individual with a somewhat uncommon skill in the vast majority of cases. Once this kind of software is available for free and packaged in such a package that almost zero effort is needed in order to achieve the same effect, a significantly higher number of people will do it. So something important has changed. I agree that the alarmist stance is still unwarranted, though.
    - _Microft 7y ago
      Using voice actors and doppelgängers does not scale. It is similar to the argument that automated surveillance is not bad because you could as well observe someone yourself. Sure that is possible but in reality it is limited to the niche of police and private detectors because it does not scale well.
  - scarejunba 7y ago
    Very cool. Thanks for sharing. I wonder why the warbling is easier to detect in the male voices. They sound like a computer. But very cool advancement.
- mikorym 7y ago
  I'm glad to see this point being made fairly often. I see the "base reality" debate in a similar situation. Base reality could be the same thing as a religion, where the "god" is the system operator [1]. That way, you are "only" as scared of multiple levels of reality as you are of religion. But perhaps more importantly, that way you have 2000+ years of debate and reflection on the topic. [1] Not my idea.
- philbarr 7y ago
  Reminds me of this story from WW2 [0]. I'm sure I saw a documentary about this but that's the only reference I can find. "In late December 1942 Grimson and Allan Morris walked out of the camp with a group of German servicemen but were recaptured 2 days later, when they showed more courtesy than was expected from a German Officer. The prisoner forgery department had produced the documents necessary for the escape and the German uniforms to go over their civilian clothing. Both Grimson and Morris had again noticed that they had near doubles serving as officers in the German guard company who they impersonated and both had lain in wait for 3 nights fully disguised in their German uniforms waiting for the right moment when the two clones attended the show. Outside the wire they changed into civilian clothes disguised as foreign workers, walking to Sagan station and catching a train to the outskirts of Leipzig before being caught." [0] https://en.wikipedia.org/wiki/George_Grimson https://en.wikipedia.org/wiki/George_Grimson
- ramraj07 7y ago
  In the past, you needed to put effort to fake things. Of course, people still exist that believe the moonlanding videos were faked, but their hypothesis hinges on the government spending extraordinary amounts of money to produce the fake. Deepfakes are deeply troubling, because anyone with a computer (a requirement that's getting optional as well quite fast) can fake any video almost instantly. Consider just one sinister motive that's already making women in many countries cower in fear: if you reject a dude he can just make a deepfake porn video with you and release it as revenge. What's the equivalent of this in the distant past? All a guy could have done is "allege" the woman is so-and-so, which to be sure, probably did ruin some women's lives, but it's almost as if our society has devolved back to that same Stone age at best then. And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth. To not take this technology seriously, to brush it off because "sensible" people will still pursue to seek the truth, is the kind of smug arrogance about questionable tech that has led us to the current overall shit:sandwich state of affairs to begin with. Let's hope we learn some lessons at least.
  - keiferski 7y ago
    What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. I think this is actually a good thing if it encourages people to dig deeper into the sources of information. > If you reject a dude he can just make a deepfake porn video with you and release it as revenge. This will be troubling at first, but again in the long run it will likely just lead to people not taking these sort of videos seriously and interpreting them as a joke / weird stuff on the Internet. This already essentially happens with Photoshop and celebrities. > And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth This is how human beings have been since the beginning of time. There is no "Golden Age of Rationality" that we are exiting.
    - BiteCode_dev 7y ago
      > What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. This is doubtful. People still believe in pictures despite it's so easy to forge. People believe obvious oral lies. Hell, people believe tweets. The video medium is incredibly powerful, and the damage it can do is huge. We are going to suffer for this, and most people won't even understand why.
- keiferski 7y ago
  Similar things happened in Russia and the Ottoman Empire - people claiming to be so-and-so, heir to the throne. As with anything else, if you are/aren't validated by society + the "gatekeepers of power", it doesn't really matter how good/bad your fake evidence is. https://en.wikipedia.org/wiki/False_Dmitry https://en.wikipedia.org/wiki/False_Dmitry
- giarc 7y ago
  In 1358 you could fake your identity and communicate with a handful of people. In 2020 you can fake your identity and communicate with 7.5 billion people at once. I think there is a difference.
kavapebumazh 7y ago
It scares ma a lot!
FailMore 7y ago
Can someone explain the difference of this and a typical Instagram / Snapchat filter? Sorry I'm a bit of a deepfake n00b
- speedgoose 7y ago
  This looks slightly better. It's also a bit more in the uncanny valley in my humble opinion.
  - mkagenius 7y ago
    Another technical difference is deepfake is end-to-end and nobody knows the exact maths applied to each pixels, while filters are more or less hand crafted functions. This particular article is not that impressive, looks like animation to me.
jokoon 7y ago
The human brain is very skilled at recognizing faces, it's the result of evolution. While deepfake are high quality, I think most people can notice that it's not entirely authentic. A deepfake cannot really account for the shape of the skull, face mimics, face muscles, how the head and jaw move, etc, there are many things that the human brain works on to identify a face. I'm not really sure that deepfakes measure and render all those things reliably. So I'm not really convinced by deepfakes in general. It's mostly a glorified silicon mask, nothing more: it will trick people if they're far away enough. Those might be able to fool face recognition softwares, but not humans, even those who are aware of deepfakes.
- mbel 7y ago
  While it's true that there are some artifacts, I think that they can be mitigated with some tricks like: avoiding sudden head movements by the actor or artificially decreasing the quality of the video. Low quality photos of big foot entertained people for years. I generally agree though that with time we will learn to see red flags like this and treat video as probably fake.
- giarc 7y ago
  I think people are mostly able to tell because they are told "this is a deepfake video of arnold schwarzenegger" (and the technology is early). Now imagine someone deepfakes a video of a gathering of people taken from across the room. It purportely shows a politician talking to some constituents and espousing some whacky ideas. It gets posted and before the politician can address it, it's already gone viral and there are pitchforks at their door. The video quality issues are hidden by the fact that it is a grainy cell phone video in a dimmly lit room with digital zoom enabled.
- danlugo92 7y ago
  Actors with similar skull shapes might be used for more convincing deep fakes.
kavalg 7y ago
Hmm, I thought deep fakes consume lots of computational power both at training and inference time. Is there some tech/science breakthrough here or they have just scaled it up to more hardware?
- mavhc 7y ago
  I think it's the training of the model that requires most of the time, once that's done it just needs to decide what to swap with swap. So I'd expect it to only work on the 1 person in the demo, with the 4 famous people depicted.
- jorgemf 7y ago
  Training yes, but for inference a commercial GPU can do that at 14fps with a small latency (very noticeable in the video)
darwinreally 7y ago
How can we achieve this with what's available in open source world? Or is it only available as a proprietary system?
- Erlich_Bachman 7y ago
  Some recent links and models and code, updated early 2020 Q1 https://github.com/iperov/DeepFaceLab https://github.com/iperov/DeepFaceLab ("More than 95% of deepfake videos are created with DeepFaceLab. DeepFaceLab is used by such popular youtube channels as Ctrl Shift Face, Sham00k, Collider videos, VFXChris Ume") https://github.com/deepfakes/faceswap https://github.com/deepfakes/faceswap https://faceswap.dev/ https://faceswap.dev/ https://forum.faceswap.dev/ https://forum.faceswap.dev/ https://dfblue.com/ https://dfblue.com/ https://pub.dfblue.com/pub/2019-10-25-deepfacelab-tutorial https://pub.dfblue.com/pub/2019-10-25-deepfacelab-tutorial
rusticpenn 7y ago
The George Bush example was not impressive. I havent seen many DeepFakes, but is hair generally not part of the equation?
- actimia 7y ago
  I don't think hair is usually included in the faked part. Generally for a convincing result you would use a base video of someone who has at least some semblance to the target (including hair). It is also possible that hair was disabled to get real-time performance for this (quite impressive) demo.
z3t4 7y ago
Damnit he skipped the female models. Which I think is the most funny. Thinking about monetization - what about animal models? Who wouldn't want to look like a panther...
darepublic 7y ago
Most of the debate here is around evil use cases for this but I just keep thinking how it will aid in the creation of memes
- etiam 7y ago
  I don't necessarily see the contradiction...
owl57 7y ago
Why did he choose celebrities of different races? Did more similar faces fall even deeper into the uncanny valley?
- speedgoose 7y ago
  Ethnicity.
huling0 7y ago
though I have to admit this looks like the best in class, there is a public real time deep fake service already available: https://www.aphrodite.ai/ https://www.aphrodite.ai/
cpr 7y ago
Funny how all this deep fake scare is happening just as important people like Epstein & cohorts start being brought down...
durpleDrank 7y ago
OH NO! PEOPLE WILL HAVE TO START USING SKEPTICISM AND CRITICAL THINKING WHEN WATCHING VIDEOS FROM SHADY PARTS OF THE INTERNET!
deleted 7y ago
[deleted]

142 points by freediver 7y ago

lqet 7y ago

Before anyone is panicking, please remind yourself that whatever medium was used by humans as a form of communication in the last few thousand years, was also used for fakes. You only need a half-decent voice actor to pretend you are someone else on the phone. Fake photographs are as old as photography. Even the probably most immediate form of interpersonal communication, seeing another person, can be faked, which is demonstrated by the many Doppelgängers used by politicians in the past. The power of the house Habsburg basically had its roots in several imperial documents and bulls faked by Rudolf IV. in 1358 (which was only discovered in the 19th century). This isn't a new phenomenon that now means that we cannot trust anything anymore - it's only that the fakes are finally catching up with live video.

prox 7y ago
One rule there was in old times you always had 2 witnesses with you for any important event to vouch for. Reasoning was that it is a lot more difficult to do any trickery when more people are involved. Just because we scaled a lot of services in a digital network means our trust services need to adapt as well.
numlock86 7y ago
> You only need a half-decent voice actor to pretend you are someone else on the phone. You can even digitalize this part already. All you need are 5 seconds of recording and a pre-trained model in the desired language/accent. https://www.youtube.com/watch?v=0sR1rU3gLzQ https://www.youtube.com/watch?v=0sR1rU3gLzQ
- saurik 7y ago
  Right... but the point is that you _don't_ need fancy neural networks, you _only_ really need someone with an actually not that uncommon skill.
  - numlock86 7y ago
    Yes. But I think there is another way to look at this: You _don't_ need fancy people with a not so uncommon skill, you _only_ need a bit of computing power. Point is: I totally agree! The entry level to fake something is really low. No matter the technical value or skills you bring to the table.
    - saurik 7y ago
      But the point of the comment being responded to--which I also think is a very important point that it feels like you missed--is this isn't some new issue, caused by computers. Zeitgeist: panic, "the concept of truth is being destroyed by advancements in computers!!!" OP: sigh, "no... you have always been able to trick people, such as with audio, using a trained voice actor" You: "but, even with audio, you can also trick people with fancy new computers!" Me: ?!?
      - feanaro 7y ago
        The OP seems as if he understood that, but his argument is that this new technology makes the costs significantly lower. The lowering of cost has a strong impact in itself. In other words, almost no one would have bothered seeking out and then paying a talented individual with a somewhat uncommon skill in the vast majority of cases. Once this kind of software is available for free and packaged in such a package that almost zero effort is needed in order to achieve the same effect, a significantly higher number of people will do it. So something important has changed. I agree that the alarmist stance is still unwarranted, though.
  - _Microft 7y ago
    Using voice actors and doppelgängers does not scale. It is similar to the argument that automated surveillance is not bad because you could as well observe someone yourself. Sure that is possible but in reality it is limited to the niche of police and private detectors because it does not scale well.
- scarejunba 7y ago
  Very cool. Thanks for sharing. I wonder why the warbling is easier to detect in the male voices. They sound like a computer. But very cool advancement.
mikorym 7y ago
I'm glad to see this point being made fairly often. I see the "base reality" debate in a similar situation. Base reality could be the same thing as a religion, where the "god" is the system operator [1]. That way, you are "only" as scared of multiple levels of reality as you are of religion. But perhaps more importantly, that way you have 2000+ years of debate and reflection on the topic. [1] Not my idea.
philbarr 7y ago
Reminds me of this story from WW2 [0]. I'm sure I saw a documentary about this but that's the only reference I can find. "In late December 1942 Grimson and Allan Morris walked out of the camp with a group of German servicemen but were recaptured 2 days later, when they showed more courtesy than was expected from a German Officer. The prisoner forgery department had produced the documents necessary for the escape and the German uniforms to go over their civilian clothing. Both Grimson and Morris had again noticed that they had near doubles serving as officers in the German guard company who they impersonated and both had lain in wait for 3 nights fully disguised in their German uniforms waiting for the right moment when the two clones attended the show. Outside the wire they changed into civilian clothes disguised as foreign workers, walking to Sagan station and catching a train to the outskirts of Leipzig before being caught." [0] https://en.wikipedia.org/wiki/George_Grimson https://en.wikipedia.org/wiki/George_Grimson
ramraj07 7y ago
In the past, you needed to put effort to fake things. Of course, people still exist that believe the moonlanding videos were faked, but their hypothesis hinges on the government spending extraordinary amounts of money to produce the fake. Deepfakes are deeply troubling, because anyone with a computer (a requirement that's getting optional as well quite fast) can fake any video almost instantly. Consider just one sinister motive that's already making women in many countries cower in fear: if you reject a dude he can just make a deepfake porn video with you and release it as revenge. What's the equivalent of this in the distant past? All a guy could have done is "allege" the woman is so-and-so, which to be sure, probably did ruin some women's lives, but it's almost as if our society has devolved back to that same Stone age at best then. And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth. To not take this technology seriously, to brush it off because "sensible" people will still pursue to seek the truth, is the kind of smug arrogance about questionable tech that has led us to the current overall shit:sandwich state of affairs to begin with. Let's hope we learn some lessons at least.
- keiferski 7y ago
  What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. I think this is actually a good thing if it encourages people to dig deeper into the sources of information. > If you reject a dude he can just make a deepfake porn video with you and release it as revenge. This will be troubling at first, but again in the long run it will likely just lead to people not taking these sort of videos seriously and interpreting them as a joke / weird stuff on the Internet. This already essentially happens with Photoshop and celebrities. > And we live in a time when "facts" already don't matter much anymore and people are just looking to find evidence that supports their claim alone rather than the truth This is how human beings have been since the beginning of time. There is no "Golden Age of Rationality" that we are exiting.
  - BiteCode_dev 7y ago
    > What the original commenter is saying, and what I will agree will happen, is that the behavior of automatically trusting video as evidence will go away. This is doubtful. People still believe in pictures despite it's so easy to forge. People believe obvious oral lies. Hell, people believe tweets. The video medium is incredibly powerful, and the damage it can do is huge. We are going to suffer for this, and most people won't even understand why.
keiferski 7y ago
Similar things happened in Russia and the Ottoman Empire - people claiming to be so-and-so, heir to the throne. As with anything else, if you are/aren't validated by society + the "gatekeepers of power", it doesn't really matter how good/bad your fake evidence is. https://en.wikipedia.org/wiki/False_Dmitry https://en.wikipedia.org/wiki/False_Dmitry
giarc 7y ago
In 1358 you could fake your identity and communicate with a handful of people. In 2020 you can fake your identity and communicate with 7.5 billion people at once. I think there is a difference.

kavapebumazh 7y ago

It scares ma a lot!

FailMore 7y ago

Can someone explain the difference of this and a typical Instagram / Snapchat filter? Sorry I'm a bit of a deepfake n00b

speedgoose 7y ago
This looks slightly better. It's also a bit more in the uncanny valley in my humble opinion.
- mkagenius 7y ago
  Another technical difference is deepfake is end-to-end and nobody knows the exact maths applied to each pixels, while filters are more or less hand crafted functions. This particular article is not that impressive, looks like animation to me.

jokoon 7y ago

The human brain is very skilled at recognizing faces, it's the result of evolution. While deepfake are high quality, I think most people can notice that it's not entirely authentic. A deepfake cannot really account for the shape of the skull, face mimics, face muscles, how the head and jaw move, etc, there are many things that the human brain works on to identify a face. I'm not really sure that deepfakes measure and render all those things reliably. So I'm not really convinced by deepfakes in general. It's mostly a glorified silicon mask, nothing more: it will trick people if they're far away enough. Those might be able to fool face recognition softwares, but not humans, even those who are aware of deepfakes.

mbel 7y ago
While it's true that there are some artifacts, I think that they can be mitigated with some tricks like: avoiding sudden head movements by the actor or artificially decreasing the quality of the video. Low quality photos of big foot entertained people for years. I generally agree though that with time we will learn to see red flags like this and treat video as probably fake.
giarc 7y ago
I think people are mostly able to tell because they are told "this is a deepfake video of arnold schwarzenegger" (and the technology is early). Now imagine someone deepfakes a video of a gathering of people taken from across the room. It purportely shows a politician talking to some constituents and espousing some whacky ideas. It gets posted and before the politician can address it, it's already gone viral and there are pitchforks at their door. The video quality issues are hidden by the fact that it is a grainy cell phone video in a dimmly lit room with digital zoom enabled.
danlugo92 7y ago
Actors with similar skull shapes might be used for more convincing deep fakes.

kavalg 7y ago

Hmm, I thought deep fakes consume lots of computational power both at training and inference time. Is there some tech/science breakthrough here or they have just scaled it up to more hardware?

mavhc 7y ago
I think it's the training of the model that requires most of the time, once that's done it just needs to decide what to swap with swap. So I'd expect it to only work on the 1 person in the demo, with the 4 famous people depicted.
jorgemf 7y ago
Training yes, but for inference a commercial GPU can do that at 14fps with a small latency (very noticeable in the video)

darwinreally 7y ago

How can we achieve this with what's available in open source world? Or is it only available as a proprietary system?

Erlich_Bachman 7y ago
Some recent links and models and code, updated early 2020 Q1 https://github.com/iperov/DeepFaceLab https://github.com/iperov/DeepFaceLab ("More than 95% of deepfake videos are created with DeepFaceLab. DeepFaceLab is used by such popular youtube channels as Ctrl Shift Face, Sham00k, Collider videos, VFXChris Ume") https://github.com/deepfakes/faceswap https://github.com/deepfakes/faceswap https://faceswap.dev/ https://faceswap.dev/ https://forum.faceswap.dev/ https://forum.faceswap.dev/ https://dfblue.com/ https://dfblue.com/ https://pub.dfblue.com/pub/2019-10-25-deepfacelab-tutorial https://pub.dfblue.com/pub/2019-10-25-deepfacelab-tutorial

rusticpenn 7y ago

The George Bush example was not impressive. I havent seen many DeepFakes, but is hair generally not part of the equation?

actimia 7y ago
I don't think hair is usually included in the faked part. Generally for a convincing result you would use a base video of someone who has at least some semblance to the target (including hair). It is also possible that hair was disabled to get real-time performance for this (quite impressive) demo.

z3t4 7y ago

Damnit he skipped the female models. Which I think is the most funny. Thinking about monetization - what about animal models? Who wouldn't want to look like a panther...

darepublic 7y ago

Most of the debate here is around evil use cases for this but I just keep thinking how it will aid in the creation of memes

etiam 7y ago
I don't necessarily see the contradiction...

owl57 7y ago

Why did he choose celebrities of different races? Did more similar faces fall even deeper into the uncanny valley?

speedgoose 7y ago
Ethnicity.

huling0 7y ago

though I have to admit this looks like the best in class, there is a public real time deep fake service already available: https://www.aphrodite.ai/ https://www.aphrodite.ai/

cpr 7y ago

Funny how all this deep fake scare is happening just as important people like Epstein & cohorts start being brought down...

durpleDrank 7y ago

OH NO! PEOPLE WILL HAVE TO START USING SKEPTICISM AND CRITICAL THINKING WHEN WATCHING VIDEOS FROM SHADY PARTS OF THE INTERNET!

deleted 7y ago

[deleted]