6 ms·

Just yesterday it occurred to me that Deep Fakes could be used to create some of the most potent disinformation campaigns ever. Already now we have had memes wi

Just yesterday it occurred to me that Deep Fakes could be used to create some of the most potent disinformation campaigns ever. Already now we have had memes with misattributed quotes and photos edited to associate Clinton with the devil, satanic rituals and so on. Think of how much more impactful slightly off quotes and speeches would be.

samcday 8y ago
Yes, it definitely has the potential to become a major issue. So much so that the DoD is already working on tools to detect deepfakes [1]. Of course this will very likely devolve into an arms race akin to antivirus vendors vs malware developers. The fundamental problem is going to be that many people seem to actually like being in their comfortable echo chamber. So, if they're presented with a video that reaffirms their hunch that the Clintons run a child trafficking ring from under a pizza store, those people seem to be unlikely to venture out and do the necessary due diligence to verify the information they're being given :( [1]: https://medium.com/mit-technology-review/the-defense-department-has-produced-the-first-tools-for-catching-deepfakes-db03998f8b65 https://medium.com/mit-technology-review/the-defense-departm...
- carry_bit 8y ago
  > those people seem to be unlikely to venture out and do the necessary due diligence to verify the information they're being given Especially since people in those circles have for the last year been pushing the idea that the attention on deep fakes is preemptive cover for such a video rumored to be out there.
  - mmjaa 8y ago
    And .. what if it is preemptive cover? What do we do about that circumstance? Like, seriously, this is shaping up to be a cyberwar nobody wants to admit. Either we can trust video sources from now on, or we can't - how is it going to be possible for TPTB to continue to divert public attention with video, if we can no longer trust our own eyes?
    - vokep 8y ago
      Hardware-based signing-while-recording, and only consider previously recorded self-verified unsigned recordings legitimate, otherwise unsigned=assumefake.
      - Bartweiss 8y ago
        > only consider previously recorded self-verified unsigned recordings legitimate, otherwise unsigned=assumefake Can you clarify this part, especially "self-verified unsigned"? I don't think I understand it - presumably you could generate camera-and-video-specific signatures which an edit couldn't reproduce, but it's not obvious to me how you would verify them without access to the original camera. As far as sign-while-recording, I agree. We'll quickly end up in a world where video of ongoing events is signed as it's taken. The simplest tactic I can think of is submitting a hash of the video file to some public store, which at least authenticates when it was shot. For an event with a well-known time (a speech, public protest, etc), that should suffice to prove the footage isn't faked, and corroboration of multiple videos should prove it's not an outright fabrication (e.g. pre-rendering a whole different speech). For time-nonspecific events like a video of a suspicious meeting, this wouldn't be enough. A hardware signature might prove that a video was real, but we'd still face a significant change in that unsourced videos floating around the web couldn't be treated as convincing. We're already getting there with photos, of course, but at the moment a clear, high-res video is pretty trustworthy without a source.
rookonaut 8y ago
You can watch this video of Obama calling Trump names already: https://www.youtube.com/watch?v=cQ54GDm1eL0 https://www.youtube.com/watch?v=cQ54GDm1eL0 Those times have already begun.
skummetmaelk 8y ago
Whether it is done or not doesn't even matter. It also does not matter whether deep faking videos is made illegal. All that matters is that it is possible to do so, and supporters of political candidate A will claim with possible deniability that political candidate A never said "X". While also claiming that political candidate B definitely said "Y", even though this may be false. Soon there will be no way to prove whether or not the videos are actually real and people will believe whatever fits their preconceived notions.
Zecar 8y ago
> Already now we have had memes with misattributed quotes and photos edited to associate Clinton with the devil, satanic rituals and so on Right, like my favorite one is the idea that she took $150 million from Russia in exchange for a shipment of our uranium. Those funds went to the Clinton Foundation a _charity_. Sheesh.
- arcticfox 8y ago
  > Those funds went to the Clinton Foundation a _charity_. Sheesh While independent investigations have concluded there was no wrongdoing, the reason above is really no defense. The Trump Foundation, for example, has shown how nonprofits can be manipulated to the benefit of the founders.
  - travisoneill1 8y ago
    No _proof_ of wrongdoing. They also failed to come up with any plausible reason for the payment other than wrongdoing.
- Cthulhu_ 8y ago
  Don't all rich people stash their billions in charities to avoid taxes anyway?
- Chris_Chambers 8y ago
  Wow, you typed "sheesh" with such conviction that you finally persuaded me to believe that the "charity" is not essentially her family's personal slush fund. Pack it up my fellow alt-right neo-Nazis, looks like we got dunked on by an exasperated use of an exclamation yet again.
- bacondude3 8y ago
  I don't know the details of this Clinton-uranium-$$ story, but I can think of a few questions to ask... 1. If you were going to give somebody an illegal bribe, would you write "bribe" in the For: line on the check? (no, you would hide or disguise your bribe) 2. What plausible, legitimate reasons are there for Russia to donate to the Clinton Foundation? 3. Is charitable giving to anybody, especially in quantities exceeding $25-50 million, normal for Russia? Cthulhu_ and arcticfox are definitely both right that charities are used by many rich people to avoid taxes and just because it's a charity doesn't mean the owner isn't benefiting from it. I'd appreciate hearing your response. Not saying it happened one way or the other, just making some observations based on the tiny bit I've heard about this. Edit: Regarding #2, a quick search turned up [1]. I don't know anything about the Borgen Project, but a quick glance at the page doesn't look too suspicious. So, assuming that page is correct and that Russia donated similar quantities in 2013 as they did the same year the uranium incident happened, they donated over 50% to the Clinton Foundation. If those assumptions are fair and that actually is the case, it sure sounds suspicious to me. [1]: https://borgenproject.org/russias-charitable-giving/ https://borgenproject.org/russias-charitable-giving/
taneq 8y ago
Just yesterday? The very first examples we saw of this kind of tech was heads of state being puppeted to say things. That and porn are about all its been used for.
mc32 8y ago
Deep fakes lack context. There are no witnesses, etc. So they’ll be most “effective” in trying to establish something in the past. It’ll be harder to produce the context where it’s believable in the near past (present minus up to a day a month a year?). So they may be useful in sowing distrust in someone by establishing they did something untoward in the past, but less likely to incite immediate reaction about something happening now, or recent past, once people become used to the possibility.
- Casseres 8y ago
  People will "remember" things that didn't happen.
  - pbhjpbhj 8y ago
    People already remember things that didn't happen; and public figures already get maligned by news outlets lying about what they did and where they went. Even when the situations are revealed to be lies.
- michaelt 8y ago
  Presumably you'd fake things it would be plausible the politician was trying to keep secret. So you'd expect there to be no witnesses. For example you'd fake hidden camera footage of politicians beating prostitutes and accepting brown envelopes from russian agents.
netsharc 8y ago
There was a "middle finger scandal" in 2015 where German journalists found a clip of the man who was to be the Greek finance minister giving a speech and saying "[Greece] should stick the finger to Germany" and doing the gesture as well, and of course stoking the typical "outrage culture". Strangely a German TV satire team have claimed they doctored the video to make it look like he's giving the finger: https://www.youtube.com/watch?v=Vx-1LQu6mAE https://www.youtube.com/watch?v=Vx-1LQu6mAE (although I stil wonder if they did the opposite: doctor the gesture out of the video, IMO the gesture itself was par for the course for him and perfectly acceptable).
sigi45 8y ago
In theory you would need to visit all events in person again as you will no longer be able to trust anything around you. We would need to invent something like public/private key auth which signs every videostream and which allows you to trust specific sources like reporters from NYT for example.
twic 8y ago
We had a crude version of this a few months ago: https://www.marketwatch.com/story/video-of-acosta-incident-posted-by-white-house-press-secretary-contains-extra-frames-2018-11-08 https://www.marketwatch.com/story/video-of-acosta-incident-p...
cwkoss 8y ago
IIRC, the majority Pizzagate was centered around Clinton's campaign chair John Podesta, not the candidate herself. John and Tony Podesta collect some very creepy art - google it. Certainly not evidence of a crime, but fairly-public figures closely connected to a presidential campaign collecting art of children bound and in morbid situations raises some eyebrows for sure.
scotty79 8y ago
Ok. One generation might be a bit confused thinking that what they see on video is what actually happened. But further generations as the technology becomes common will just intuitively understand that video by itself proves nothing and it was most likely made for the lulz than recorded as it happened. Video of Trump saying something already proves nothing especially not that he ever said it because now he says he never did. And all that even without deep fake technology. People care less and less about history of reality. They become aware that it's to large extent unknowable drowned in narratives.

samcday 8y ago

Yes, it definitely has the potential to become a major issue. So much so that the DoD is already working on tools to detect deepfakes [1]. Of course this will very likely devolve into an arms race akin to antivirus vendors vs malware developers. The fundamental problem is going to be that many people seem to actually like being in their comfortable echo chamber. So, if they're presented with a video that reaffirms their hunch that the Clintons run a child trafficking ring from under a pizza store, those people seem to be unlikely to venture out and do the necessary due diligence to verify the information they're being given :( [1]: https://medium.com/mit-technology-review/the-defense-department-has-produced-the-first-tools-for-catching-deepfakes-db03998f8b65 https://medium.com/mit-technology-review/the-defense-departm...

carry_bit 8y ago
> those people seem to be unlikely to venture out and do the necessary due diligence to verify the information they're being given Especially since people in those circles have for the last year been pushing the idea that the attention on deep fakes is preemptive cover for such a video rumored to be out there.
- mmjaa 8y ago
  And .. what if it is preemptive cover? What do we do about that circumstance? Like, seriously, this is shaping up to be a cyberwar nobody wants to admit. Either we can trust video sources from now on, or we can't - how is it going to be possible for TPTB to continue to divert public attention with video, if we can no longer trust our own eyes?
  - vokep 8y ago
    Hardware-based signing-while-recording, and only consider previously recorded self-verified unsigned recordings legitimate, otherwise unsigned=assumefake.
    - Bartweiss 8y ago
      > only consider previously recorded self-verified unsigned recordings legitimate, otherwise unsigned=assumefake Can you clarify this part, especially "self-verified unsigned"? I don't think I understand it - presumably you could generate camera-and-video-specific signatures which an edit couldn't reproduce, but it's not obvious to me how you would verify them without access to the original camera. As far as sign-while-recording, I agree. We'll quickly end up in a world where video of ongoing events is signed as it's taken. The simplest tactic I can think of is submitting a hash of the video file to some public store, which at least authenticates when it was shot. For an event with a well-known time (a speech, public protest, etc), that should suffice to prove the footage isn't faked, and corroboration of multiple videos should prove it's not an outright fabrication (e.g. pre-rendering a whole different speech). For time-nonspecific events like a video of a suspicious meeting, this wouldn't be enough. A hardware signature might prove that a video was real, but we'd still face a significant change in that unsourced videos floating around the web couldn't be treated as convincing. We're already getting there with photos, of course, but at the moment a clear, high-res video is pretty trustworthy without a source.

rookonaut 8y ago

You can watch this video of Obama calling Trump names already: https://www.youtube.com/watch?v=cQ54GDm1eL0 https://www.youtube.com/watch?v=cQ54GDm1eL0 Those times have already begun.

skummetmaelk 8y ago

Whether it is done or not doesn't even matter. It also does not matter whether deep faking videos is made illegal. All that matters is that it is possible to do so, and supporters of political candidate A will claim with possible deniability that political candidate A never said "X". While also claiming that political candidate B definitely said "Y", even though this may be false. Soon there will be no way to prove whether or not the videos are actually real and people will believe whatever fits their preconceived notions.

Zecar 8y ago

> Already now we have had memes with misattributed quotes and photos edited to associate Clinton with the devil, satanic rituals and so on Right, like my favorite one is the idea that she took $150 million from Russia in exchange for a shipment of our uranium. Those funds went to the Clinton Foundation a _charity_. Sheesh.

arcticfox 8y ago
> Those funds went to the Clinton Foundation a _charity_. Sheesh While independent investigations have concluded there was no wrongdoing, the reason above is really no defense. The Trump Foundation, for example, has shown how nonprofits can be manipulated to the benefit of the founders.
- travisoneill1 8y ago
  No _proof_ of wrongdoing. They also failed to come up with any plausible reason for the payment other than wrongdoing.
Cthulhu_ 8y ago
Don't all rich people stash their billions in charities to avoid taxes anyway?
Chris_Chambers 8y ago
Wow, you typed "sheesh" with such conviction that you finally persuaded me to believe that the "charity" is not essentially her family's personal slush fund. Pack it up my fellow alt-right neo-Nazis, looks like we got dunked on by an exasperated use of an exclamation yet again.
bacondude3 8y ago
I don't know the details of this Clinton-uranium-$$ story, but I can think of a few questions to ask... 1. If you were going to give somebody an illegal bribe, would you write "bribe" in the For: line on the check? (no, you would hide or disguise your bribe) 2. What plausible, legitimate reasons are there for Russia to donate to the Clinton Foundation? 3. Is charitable giving to anybody, especially in quantities exceeding $25-50 million, normal for Russia? Cthulhu_ and arcticfox are definitely both right that charities are used by many rich people to avoid taxes and just because it's a charity doesn't mean the owner isn't benefiting from it. I'd appreciate hearing your response. Not saying it happened one way or the other, just making some observations based on the tiny bit I've heard about this. Edit: Regarding #2, a quick search turned up [1]. I don't know anything about the Borgen Project, but a quick glance at the page doesn't look too suspicious. So, assuming that page is correct and that Russia donated similar quantities in 2013 as they did the same year the uranium incident happened, they donated over 50% to the Clinton Foundation. If those assumptions are fair and that actually is the case, it sure sounds suspicious to me. [1]: https://borgenproject.org/russias-charitable-giving/ https://borgenproject.org/russias-charitable-giving/

taneq 8y ago

Just yesterday? The very first examples we saw of this kind of tech was heads of state being puppeted to say things. That and porn are about all its been used for.

mc32 8y ago

Deep fakes lack context. There are no witnesses, etc. So they’ll be most “effective” in trying to establish something in the past. It’ll be harder to produce the context where it’s believable in the near past (present minus up to a day a month a year?). So they may be useful in sowing distrust in someone by establishing they did something untoward in the past, but less likely to incite immediate reaction about something happening now, or recent past, once people become used to the possibility.

Casseres 8y ago
People will "remember" things that didn't happen.
- pbhjpbhj 8y ago
  People already remember things that didn't happen; and public figures already get maligned by news outlets lying about what they did and where they went. Even when the situations are revealed to be lies.
michaelt 8y ago
Presumably you'd fake things it would be plausible the politician was trying to keep secret. So you'd expect there to be no witnesses. For example you'd fake hidden camera footage of politicians beating prostitutes and accepting brown envelopes from russian agents.

netsharc 8y ago

There was a "middle finger scandal" in 2015 where German journalists found a clip of the man who was to be the Greek finance minister giving a speech and saying "[Greece] should stick the finger to Germany" and doing the gesture as well, and of course stoking the typical "outrage culture". Strangely a German TV satire team have claimed they doctored the video to make it look like he's giving the finger: https://www.youtube.com/watch?v=Vx-1LQu6mAE https://www.youtube.com/watch?v=Vx-1LQu6mAE (although I stil wonder if they did the opposite: doctor the gesture out of the video, IMO the gesture itself was par for the course for him and perfectly acceptable).

sigi45 8y ago

In theory you would need to visit all events in person again as you will no longer be able to trust anything around you. We would need to invent something like public/private key auth which signs every videostream and which allows you to trust specific sources like reporters from NYT for example.

twic 8y ago

We had a crude version of this a few months ago: https://www.marketwatch.com/story/video-of-acosta-incident-posted-by-white-house-press-secretary-contains-extra-frames-2018-11-08 https://www.marketwatch.com/story/video-of-acosta-incident-p...

cwkoss 8y ago

IIRC, the majority Pizzagate was centered around Clinton's campaign chair John Podesta, not the candidate herself. John and Tony Podesta collect some very creepy art - google it. Certainly not evidence of a crime, but fairly-public figures closely connected to a presidential campaign collecting art of children bound and in morbid situations raises some eyebrows for sure.

scotty79 8y ago

Ok. One generation might be a bit confused thinking that what they see on video is what actually happened. But further generations as the technology becomes common will just intuitively understand that video by itself proves nothing and it was most likely made for the lulz than recorded as it happened. Video of Trump saying something already proves nothing especially not that he ever said it because now he says he never did. And all that even without deep fake technology. People care less and less about history of reality. They become aware that it's to large extent unknowable drowned in narratives.