5 ms·

Memers are making deepfakes, and things are getting weird

68 points by jpindar 6y ago

RcouF1uZ4gsC 6y ago
I think this is actually a good thing. The more the average person is familiar with deepfake tech, the less likely they are to get misled.
- SQueeeeeL 6y ago
  I like the philosophy, but tricking people isn't very hard. Apparently, some shaking camera footage of a computer monitor was all it took to discredit a political dissident https://www.nytimes.com/2020/08/23/world/asia/cambodia-facebook-disinformation.html https://www.nytimes.com/2020/08/23/world/asia/cambodia-faceb...
  - mythobit 6y ago
    It seems like as soon as it confirms their bias they are willing to ignore if something is real or not.
    - Spivak 6y ago
      You say "they" like this isn't something that every single person in the world does constantly -- nobody is immune to it. Confirmation bias is just a derogatory term for people having priors.
      - makomk 6y ago
        Yeah, and it really is everyone. For example, remember when the White House Press Secretary tweeted a video that had apparently been doctored to make it look like a journalist had attacked one of their staffers? There was a really clear, convincing comparison that overlaid a semi-transparent version of that video on the original, showing that it leapt ahead in the part where his hand went near her because it had been sped up to make the interaction look more violent. Many people spread it, including YouTube debunker Captain Disillusion with this message: "Getting lots of requests on the topic, but there's nothing for me to examine. This person has examined it, very simply and clearly. And I concur." https://twitter.com/cdisillusion/status/1060564297103917056 https://twitter.com/cdisillusion/status/1060564297103917056 Now, having compared the two videos myself and not found anything like this, obviously I had a look at it - and worked out the trick almost immediately. The comparison had the overlay ahead of the original by the same number of frames the whole way through from frame 1, but it was so faint that during normal viewing the difference was only visible in fast-moving parts, creating the illusion it leapt ahead in those parts. Easiest trick imaginable, could probably have been done trivially with film a century ago, and fooled someone who'd built a career and reputation around debunking faked videos along with many others. (As for the White House video, that's a long story but the short version is that it seemed to be pretty much exactly what it purported to be, and the only actual "doctoring" was probably the obvious, intentional stuff like repeating parts of the video zoomed in.)
- Spivak 6y ago
  This assumes you even need to go that far to successfully mislead people. The most authoritative sources of information in the world right now today are the written and spoken word. I mean people wouldn't just lie right? For the world you're describing to make sense people would actually have to demand video evidence to believe something.
- tdeck 6y ago
  That still leaves the other problem - people will eventually stop believing real video evidence. Cops will claim videos of their abuse are deepfakes, and people will believe them. Politicians will claim that recordings of them soliciting bribes are deepfakes, and they'll be off the hook. Concrete evidence of misdeeds is one of our best ways to hold powerful people accountable, but with deepfakes they can simply flood the web with fake incriminating evidence and we'll become so accustomed to ignoring it that we ignore real evidence as well.
  - Nasrudith 6y ago
    They already had that to get people to ignore video evidence - just scream "It is out of context!" while committing acts that could never be excused under any circumstances.
tjr225 6y ago
This doesn't look very convincing. Also, I didn't know you could run python in google drive.
- soared 6y ago
  >There’s a telltale wonkiness to the faces in the videos made with this algorithm, which makes its handiwork easy to recognize; that is part of the deepfakes’ humor. These imperfections—and the surrealist quality Part of most meme's style is purposeful bad photo editing, so the poor deepfake quality is part of the format.
- gwern 6y ago
  Not Drive, Colab. Colab is a VM service. (Although you can mount your Google Drive, and many Colab users will.)
echelon 6y ago
Deep Fakes are merely the next iteration of Photoshop. Before the 90's, faking photographs required extremely labor intensive processes performed by experts. Multiple exposure tricks, splicing, painting. One could have expected that the development of Photoshop would have been hailed as the end of trustworthy information, but instead we got memes and photo manipulations such as https://www.snopes.com/fact-check/shark-attacks-navy-diver/ https://www.snopes.com/fact-check/shark-attacks-navy-diver/ I think Deep Fakes are good. They're going to breathe new life into media and make it significantly easier to make films, music, and all kinds of art. The public will come to understand AI-generated media as a part of daily life. People are not that stupid. The media (the only ones decrying this) need to stop with the Luddite hysteria. I've actually been quite involved with deep fakes and wrote a few pretty big sites to help democratize the tech (the average person can't run Google colabs and Python) : https://vo.codes https://vo.codes (over 40 celebrity and cartoon voices) https://trumped.com https://trumped.com (just Donald Trump) I've already got a real time voice conversion software nearly finished. I can do Discord/Team Speak and talk as Trump and Gilbert Gottfried. I'll probably do the same for video soon.
- anthony_romeo 6y ago
  While what you say is reasonable given a sufficiently long span of time, I think it's also reasonable to expect that there will be very significant growing pains between now and when the public ultimately comes to understand AI-generated media as a part of daily life.
  - echelon 6y ago
    I think we're maybe a year away from that. Most of these results are being produced from open source models that are difficult to set up and run. I joined a few Discord servers that were started this year for deep fake video and voice, and they're now huge. We're also just now seeing subreddits explode with content [1]. My website, vo.codes, is starting to take off on Twitter. My phone is blowing up with retweets, mentions, and followers. Youtube is overflowing with the stuff [2] Give it a year and these results will be everywhere. The kids love it. Memes are going to normalize it. [1] https://www.reddit.com/r/mediasynthesis https://www.reddit.com/r/mediasynthesis [2] https://www.youtube.com/watch?v=hRlo8_13Zyw https://www.youtube.com/watch?v=hRlo8_13Zyw
throwaway743 6y ago
Relevant shameless art project plug. Nlp+voice clone model of Joe Biden. Streaming on hold until interactive chat is tested and complete. Disclaimer: I'm not a fan of either of the two parties. Trump version to come. https://twitch.tv/biden_unleashed https://twitch.tv/biden_unleashed

68 points by jpindar 6y ago

I think this is actually a good thing. The more the average person is familiar with deepfake tech, the less likely they are to get misled.

SQueeeeeL 6y ago
I like the philosophy, but tricking people isn't very hard. Apparently, some shaking camera footage of a computer monitor was all it took to discredit a political dissident https://www.nytimes.com/2020/08/23/world/asia/cambodia-facebook-disinformation.html https://www.nytimes.com/2020/08/23/world/asia/cambodia-faceb...
- mythobit 6y ago
  It seems like as soon as it confirms their bias they are willing to ignore if something is real or not.
  - Spivak 6y ago
    You say "they" like this isn't something that every single person in the world does constantly -- nobody is immune to it. Confirmation bias is just a derogatory term for people having priors.
    - makomk 6y ago
      Yeah, and it really is everyone. For example, remember when the White House Press Secretary tweeted a video that had apparently been doctored to make it look like a journalist had attacked one of their staffers? There was a really clear, convincing comparison that overlaid a semi-transparent version of that video on the original, showing that it leapt ahead in the part where his hand went near her because it had been sped up to make the interaction look more violent. Many people spread it, including YouTube debunker Captain Disillusion with this message: "Getting lots of requests on the topic, but there's nothing for me to examine. This person has examined it, very simply and clearly. And I concur." https://twitter.com/cdisillusion/status/1060564297103917056 https://twitter.com/cdisillusion/status/1060564297103917056 Now, having compared the two videos myself and not found anything like this, obviously I had a look at it - and worked out the trick almost immediately. The comparison had the overlay ahead of the original by the same number of frames the whole way through from frame 1, but it was so faint that during normal viewing the difference was only visible in fast-moving parts, creating the illusion it leapt ahead in those parts. Easiest trick imaginable, could probably have been done trivially with film a century ago, and fooled someone who'd built a career and reputation around debunking faked videos along with many others. (As for the White House video, that's a long story but the short version is that it seemed to be pretty much exactly what it purported to be, and the only actual "doctoring" was probably the obvious, intentional stuff like repeating parts of the video zoomed in.)
Spivak 6y ago
This assumes you even need to go that far to successfully mislead people. The most authoritative sources of information in the world right now today are the written and spoken word. I mean people wouldn't just lie right? For the world you're describing to make sense people would actually have to demand video evidence to believe something.
tdeck 6y ago
That still leaves the other problem - people will eventually stop believing real video evidence. Cops will claim videos of their abuse are deepfakes, and people will believe them. Politicians will claim that recordings of them soliciting bribes are deepfakes, and they'll be off the hook. Concrete evidence of misdeeds is one of our best ways to hold powerful people accountable, but with deepfakes they can simply flood the web with fake incriminating evidence and we'll become so accustomed to ignoring it that we ignore real evidence as well.
- Nasrudith 6y ago
  They already had that to get people to ignore video evidence - just scream "It is out of context!" while committing acts that could never be excused under any circumstances.

tjr225 6y ago

This doesn't look very convincing. Also, I didn't know you could run python in google drive.

soared 6y ago
>There’s a telltale wonkiness to the faces in the videos made with this algorithm, which makes its handiwork easy to recognize; that is part of the deepfakes’ humor. These imperfections—and the surrealist quality Part of most meme's style is purposeful bad photo editing, so the poor deepfake quality is part of the format.
gwern 6y ago
Not Drive, Colab. Colab is a VM service. (Although you can mount your Google Drive, and many Colab users will.)

echelon 6y ago

Deep Fakes are merely the next iteration of Photoshop. Before the 90's, faking photographs required extremely labor intensive processes performed by experts. Multiple exposure tricks, splicing, painting. One could have expected that the development of Photoshop would have been hailed as the end of trustworthy information, but instead we got memes and photo manipulations such as https://www.snopes.com/fact-check/shark-attacks-navy-diver/ https://www.snopes.com/fact-check/shark-attacks-navy-diver/ I think Deep Fakes are good. They're going to breathe new life into media and make it significantly easier to make films, music, and all kinds of art. The public will come to understand AI-generated media as a part of daily life. People are not that stupid. The media (the only ones decrying this) need to stop with the Luddite hysteria. I've actually been quite involved with deep fakes and wrote a few pretty big sites to help democratize the tech (the average person can't run Google colabs and Python) : https://vo.codes https://vo.codes (over 40 celebrity and cartoon voices) https://trumped.com https://trumped.com (just Donald Trump) I've already got a real time voice conversion software nearly finished. I can do Discord/Team Speak and talk as Trump and Gilbert Gottfried. I'll probably do the same for video soon.

anthony_romeo 6y ago
While what you say is reasonable given a sufficiently long span of time, I think it's also reasonable to expect that there will be very significant growing pains between now and when the public ultimately comes to understand AI-generated media as a part of daily life.
- echelon 6y ago
  I think we're maybe a year away from that. Most of these results are being produced from open source models that are difficult to set up and run. I joined a few Discord servers that were started this year for deep fake video and voice, and they're now huge. We're also just now seeing subreddits explode with content [1]. My website, vo.codes, is starting to take off on Twitter. My phone is blowing up with retweets, mentions, and followers. Youtube is overflowing with the stuff [2] Give it a year and these results will be everywhere. The kids love it. Memes are going to normalize it. [1] https://www.reddit.com/r/mediasynthesis https://www.reddit.com/r/mediasynthesis [2] https://www.youtube.com/watch?v=hRlo8_13Zyw https://www.youtube.com/watch?v=hRlo8_13Zyw

throwaway743 6y ago

Relevant shameless art project plug. Nlp+voice clone model of Joe Biden. Streaming on hold until interactive chat is tested and complete. Disclaimer: I'm not a fan of either of the two parties. Trump version to come. https://twitch.tv/biden_unleashed https://twitch.tv/biden_unleashed