8 ms·

High-tech deception of ‘deepfake’ videos

71 points by creaghpatr 8y ago

SmooL 8y ago
I have to agree that this will be inevitable. Fake videos _will_ be created to spread misinformation. I bring you no solutions. Our best programmatic tool for these kinds of video analysis is currently the exact same technology that enables it in the first place. Furthermore, any neural network that can detect fake videos will only be used adversarially to train a better deep fake creator! The only solution I see is the camera somehow cryptographically signing the video, and enabling users to easily check whether the signature is valid. However I don't think this is feasible, due to the amount of cooperation we'd need between camera makers and video distribution platforms, especially since it's not really in the businesses direct line of best interest help. The world's going to start getting real surreal.
- bytematic 8y ago
  Can we develop something to put in videos that are real? Maybe something built into the software or maybe even some icon you have to hold up behind someone like the president to at least make deepfakes difficult/impossible? Just spit-balling as I also see no point in video analysis
  - zzzcpan 8y ago
    We already have developed something. Only thing missing now is cultural shift in the media to apply a little bit of effort into signing videos.
    - sp332 8y ago
      That might be solving the wrong problem. You could believe that a video is from a particular journalist, but you might not trust that the journalist didn't tamper with the video before publishing it.
  - prepend 8y ago
    You need to develop a unique, unpredictable set of expressions that deepfake will be unable to fake. Or perhaps active tattoos that display private key signed messages that describe what they’re saying and doing.
- deleted 8y ago
  [deleted]
- creaghpatr 8y ago
  The only thing that can be done is use contextual evidence to disprove the possibility of a video being real but that puts a disproportionate on the victim of a fake video. Will the government be in the business of commenting on the veracity of whatever video(s) they anticipate coming? And if they say it's fake (or real), will the public believe them?
  - sp332 8y ago
    It will go back to being like a newspaper, where you can fake-quote anybody saying anything. Only established sources with some reputation on the line will be trustable. A random citizen won't be able to whip out a cell phone and get evidence that will be taken seriously. Credibility will be more centralized again.
- zzzcpan 8y ago
  To make it feasible you don't need much, just journalists signing videos and uploading them somewhere besides publishing anywhere unsigned. This will make every unsigned video easily verifiable, just not with crypto.
  - binarymax 8y ago
    That's fine for journalists, but does nothing to prevent fakes captured by the populace. A good deal of our journalism now is by on scene witnesses. What can we do about fakes posted on social media purported to be the real capture of some scandal or event?
    - fooey 8y ago
      I think what we'll see is something less like the Jordan Peele version of Obama based on a public clip, and something more like a faked version of the Romney 47% video Start with something naturally unfocused and exists only as a one off original, and it would be extremely hard to disprove without comparing it to other video of the same event.
    - JetSpiegel 8y ago
      We are already vulnerable to this attack, they are just called "viral marketing campaigns".
  - 21 8y ago
    In the last years there have been quite a few cases of journalists resigning from very prestigious publications (NYT, ...) because they were caught fabricating news.
    - zzzcpan 8y ago
      Yeah, well, it doesn't matter much whether they use false information or not, as their purpose is to paint a picture they want people to see either way.
  - msla 8y ago
    > To make it feasible you don't need much, just journalists signing videos and uploading them somewhere besides publishing anywhere unsigned. And in order to implement that, you'll be talking to people who'd sign a video by getting a pen, some paper, and a flatbed scanner. I'm not entirely joking. My point is that this proves that there comes a point in every field where technical illiteracy goes from being annoying for you (you can't use the best tools anymore, and your workflow is now sub-par) to annoying for others (you can't collaborate effectively) to completely unworkable, as when your work product is now worthless because you can't demonstrate it isn't a cheap forgery. And some people are just not technical, through stubbornness or otherwise. This transition will hurt. Maybe tools will follow the demand. I'm not entirely sanguine.
- isostatic 8y ago
  The last 20 years certain elements have been pushing distrust and hatred for both politicians and journalists. However I'd hope that we move to some agreed standard of signing of videos, so their source can be confirmed by the player. It doesn't matter if that source is the bbc, washington post, alex jones, national enquirer, the onion, or "Bob Joe Billy-Bob the Third", it means you know where the video came from, and can trust (or not) it based on that information. A video with a bbc banner showing the president saying something could easily be faked. One that is signed by the BBC will only be fake if the BBC itself is publishing it as a fake, and that comes with a reputational impact. Sites that reversion that video (recompress it etc) can simply sign it by themselves. They could claim that they took an original BBC item to do it, and then it's how much you trust that site. People see someone, be it Donald Trump, or John Simpson, or Walter Mondale, and instinctively know that person said what they can see. That's changing, and it's going to be a long time before our brains change. Even now, you read some rubbish that you consciously know is rubbish, but part of it lodges in your subconcious and affects your thinking down the line. At least if you know who created the video, it gives you a fighting chance. With this system anyone can still publish, even anonymously, but reputations are hard to gain and easy to lose. I don't like the daily mail, but I know if they publish a video of Obama saying "Killmonger was right, and President Trump is a total and complete dipshit", I'll probably believe he did. If he didn't, that will come out, and the daily mail is off the list of sources I trust to not make crap up (rather than just bend things), and it makes it far easier for reasonable people to dismiss it. This moves the onus on news sources to be more discerning in their facts, but if they get it wrong then they lose their readers. By signing the news, they are backing it with their reputation. If Alex Jones has signed it, then he's putting his reputation on the line (if he has any reputation for saying something).
  - vkou 8y ago
    > A video with a bbc banner showing the president saying something could easily be faked. One that is signed by the BBC will only be fake if the BBC itself is publishing it as a fake, and that comes with a reputational impact. Nobody cares much about the reputational impact. Fringe tabloid fake-news, and mainstream 'fake'-news gets plenty of viewers. BBC itself has manufactured video 'evidence' during the Libyan war. [1] Yet, most people think it's an accurate source of news. [1] Their 'live broadcasts' showed video of people in India waving the Indian tri-color, which the BBC has claimed was rebels in Libya's Green Square waving the old Libyan tri-color. But, eh, who cares about honesty - when you've got a politically sensitive story to tell.
- JumpCrisscross 8y ago
  > the amount of cooperation we'd need between camera makers and video distribution platforms Apple could single-handedly launch a "verified video" system.
  - ttul 8y ago
    Dear Tim, Please build this. — everyone
- avivo 8y ago
  Unfortunately, even if we can pull it off "cryptographically signing a video" doesn't "solve" the problem. How would that verification be presented to a user? How/why would it influence their beliefs/actions? (As mentioned elsewhere in this thread, I go into a bit more detail here: https://www.washingtonpost.com/news/theworldpost/wp/2018/02/22/digital-reality/ https://www.washingtonpost.com/news/theworldpost/wp/2018/02/... ) Will try to write a HN friendly tldr/update soon.
- oneshot908 8y ago
  The best solution (IMO of course) is to stop unquestionably believing anything you see, read, or hear. Why most people choose otherwise has saddened me since childhood. The worst solution is to panic and attempt to ban technology like this insuring that only the bad actors will use it. I want everyone down to kids to be able to make fake videos to their little heart's content so we can all internalize this inevitable Black Mirrorization of reality. And your proposed solution is fantastic BTW. Unfortunately, there are more than enough bad actors among us to social engineer those signatures out of otherwise trustworthy agencies and a general lack of intellectual curiosity on the part of most people to even bother to check for such things. Imagine the day we have a video where HRC confesses to the pedophile colony on Mars and coming up next, Trump tells everyone where he buried the bodies, and if people look, there will be bodies there (never mind that they're stolen cadavers). How will people cope with a world like this? It is almost like we need pop art and pop culture to drive this point home before political mass media does it in the worst possible way.
  - SmooL 8y ago
    | The best solution (IMO of course) is to stop unquestionably believing anything you see, read, or hear. Why most people choose otherwise has saddened me since childhood. Agreed, but I think it's been long proven most people do. People hear what they want to hear. Maybe that will change, but I'd say it seems unlikely at this point. | he worst solution is to panic and attempt to ban technology like this insuring that only the bad actors will use it. Very true. | I want everyone down to kids to be able to make fake videos to their little heart's content so we can all internalize this inevitable Black Mirrorization of reality. Increasingly getting there. It only takes one person to make a nice GUI, post it to github. | How will people cope with a world like this? Probably poorly if history has taught us anything. | It is almost like we need pop art and pop culture to drive this point home before political mass media does it in the worst possible way. Very intriguing. Traditionally, art and culture change as a reaction to current affairs, as opposed to a pre-emption. I'm not sure how to pragmatically flip that paradigm.
- emodendroket 8y ago
  Doctored audio and video isn't really a new problem though. More prosaically, you could just redact a recording to give a misleading impression and not rely on any fancy technology.
craftyguy 8y ago
I'd be more concerned with US officials using this technology to indict innocent people who disagree with them. Being able to conjure up a confession for anything by anyone is extremely powerful, given that the alternatives to accomplish the same thing today are basically apprehension and torture/threats.
- frgtpsswrdlame 8y ago
  That's an interesting application. With skepticism spreading about many forensic tools like fingerprints, burn analysis, bite analysis, etc. the confession has stayed the ultimate indicator of guilt to most of the public. I wonder if we may be entering another era of 'reasonable doubt'.
- creaghpatr 8y ago
  >Being able to conjure up a confession for anything by anyone is extremely powerful Being able to dismiss credible video evidence as 'deepfake' is equally powerful. In any case, it would be extremely risky to hang a case on a single confession video that the accused claims is fake, given the current pattern of prosecution the DOJ would never bring that case; it would be embarrassing in the public eye and probably overturned on appeal, if it even made it to trial.
  - germinalphrase 8y ago
    Garry Kasparov has often accused V. Putin of maintaining his public influence by successfully attacking the idea that a verifiable truth can be established about his/oligarchical misdeeds. Surely, there are techniques to make educated assessments about the probability that [this] happened rather than [that] - but the public at large is neither literate in statistics nor receptive to uncertainty.
    - flyingcircus3 8y ago
      That's my view as well. In a time of information overload especially, people seem to be far more concerned in simply ending uncertainty, than validating the answer that precipitates that certainty. I believe this is referred to as the firehose model of propoganda. It effectively normalizes tu quoque fallacies, as the people whole have taken on a new set of facts are completely unprepared to debate the merits of those facts, and instead fallback to defending their newfound certainty.
mgraybosch 8y ago
You should never trust digital media. You should be extremely selective about what analog media you trust. If you weren't there to see it happen yourself, all you've got to go on is somebody else's perception. Be careful who you trust.
- zkascak 8y ago
  As a filmmaker I feel that completely distrusting a medium just because it is digital might be a little harsh. I think that we may want to consider using cryptographic signature methods as a way to mark materials as coming from the source they claim to be from. Though this is not a perfect solution. Though I still work with analog film, though it is becoming more and more cost prohibitive everyday. Analog can just as easily be altered. If you looked at a silver gelatin print, if the printmaker was talented you probably would never be able to detect the changes to the image that were made during the printing process. Is the resulting image a fake, or is it just an artists vision?
zkascak 8y ago
As a filmmaker I am both in awe and horrified with this. Some of these techniques can be used to make fixes in motion picture where it would be far too expensive to reshoot. But there is the opposite where this technology can be used to falsify a video that can potentially be used to make false confessions or change what a person said for nefarious purposes. Welcome to the world of what was once thought to be science fiction.
drivingmenuts 8y ago
When this happens (not if), I expect it will play right into the current administration's habit of dismissing anything that disagrees as fake news. For the administration that follows? Who knows? I only have evidence for right now. I do think most of it will come from people lower down the political food chain (the meme creators, etc) who seek to convince others of their POV out of desperation and/or maliciousness.
segmondy 8y ago
Anyone know where deepfakes moved to? It was great seeing what was going on in /r/deepfakes but due to the "offensive nature" of some of the videos that subreddit was killed. Anyone know the best place to watch and observe the latest happenings in the scene?
- k__ 8y ago
  Good question, I was just waiting for the all Nicolas Cage version of LotR
  - prepend 8y ago
    It’s a bit of a pain to follow progress without the subreddit, but progress is still being made- https://youtu.be/QhxTTshL3b0 https://youtu.be/QhxTTshL3b0
avivo 8y ago
It's not a totally lost cause — I wrote a piece on what we can do about this: https://www.washingtonpost.com/news/theworldpost/wp/2018/02/22/digital-reality/ https://www.washingtonpost.com/news/theworldpost/wp/2018/02/... It's a first step, and my thinking has also evolved since—expect more updates in this space soon.
- majos 8y ago
  From this article: > AI researchers and platform technologists need similar in-house or third-party societal impact review boards to help them evaluate the potential unintended consequences of their work. As a machine learning researcher myself (albeit one who is probably unlikely to run afoul of one of these boards), this suggestion for research gives me pause. In the context of deepfakes, are you suggesting that it would have been more prudent to stop the research that made it possible before it had a chance to do so? This seems like a good way to ensure that less scrupulous parties harness destructive technologies before more scrupulous parties (the kind who'd adopt review boards). I imagine this debate to some extent has already played out with nuclear weapons and bioweapons. I'm much less familiar with these two spheres, but as I recall research continued in both.
  - avivo 8y ago
    Lots of details to be worked out, but yes there are models in other fields. You bring up a real concern and there are ways to address it. I'm not suggesting stopping research on this. But the way in which that research is communicated, what the focus is, and the way which particular sorts of replication is enabled or discouraged has an impact. An example that the HN crowd likely knows better is vulnerability/exploit disclosure. There are good ways to do it and harmful ways to do it. You can perhaps think about deepfakes as an exploit for the human sensors/cognition that handle authentication.
gnicholas 8y ago
Will FB, Twitter, YouTube, and other social networks create tools to flag deepfakes? It wouldn't solve the problem, but at least it could blunt the spread. It could also raise awareness of the issue, so that if people see a video that seems "too good to be true" on a different platform, they would know that it might not be real.
- mizzack 8y ago
  Inevitably you'll hear cries of censorship with varying degrees of veracity.
  - gnicholas 8y ago
    I would think that there'd be enough deepfakes on both sides that everyone would recognize that iff there are tools that accurately detect deepfakes, it would be useful to have them enabled on social networks. They don't need to censor the videos — just flag them as having deepfake characteristics.
    - mizzack 8y ago
      Like when Facebook (briefly) tagged articles from certain sources as being disreputable and it led to an _increase_ in clicks to those articles? People will believe what they want to believe. We're also seeing that they don't like mega corps acting as arbiters of truth. So, I'm not sure what, if any, effect this moderation/curation effort would have.
      - gnicholas 8y ago
        I think what we really need is an embarrassing term for "someone who was fooled by a deepfake" (like "catfish"), so that there will be a social stigma that incentivizes people to not be tricked by them. I don't currently have any better ideas that "getting deepfaked". Suggestions?
noja 8y ago
The only solution to this is signed live lifecasting to a blockchain for every person on earth. I'm only half kidding.
- rohit2412 8y ago
  How about a sort of steganographic signing with assemytric cryptography.
mjw_byrne 8y ago
Deepfakes are surely no more a "problem" than photoshopped still images? Everyone knows that a still image can be altered to (convincingly) depict something that never happened. So, a still image is not considered good evidence of anything unless it has a verifiable provenance, chain of custody etc. Presumably that wasn't always the case, but now that photo editing tech is universally accessible, we all know we can't trust a still image unless its authenticity can be corroborated somehow. The deepfake tech merely puts video into the same category of (un)trustworthiness. The potential for abuse depends on people's trust that video is always authentic. So the "solution" is simply to make it as widely-known as possible that video is now easy to fake.
- roywiggins 8y ago
  The volume alone might be a problem. It takes some Photoshop skills to pull off an acceptable fake. If there's a turnkey website that can spit them out en masse, that's something different than having to actually hire a human to do it by hand. Previously it took at least one CGI artist to convincingly fake video, probably several. This inherently limits how much of a problem it can be, because it doesn't scale. AI scales. I mean- either deep learning AI automation is going to be important and useful, or it isn't. If replacing all drivers with robots is important, then automated turnkey fakery has the potential for disrupting something, isn't it?
  - mjw_byrne 8y ago
    I agree, but I think my original point covers this - if video proves to be very easy to fake at scale (e.g. using your example of a turnkey website) then that should result in equally large-scale scepticism about video as evidence. The internet may drown in annoying deepfake-based memes but as long as everyone knows its all fake, there seems little potential for the kind of serious harm people are worried about (reputational damage, election meddling etc.)
    - save_ferris 8y ago
      > ...as long as everyone knows its all fake, There's a shockingly large number of people who aren't even convinced that election meddling took place in 2016, or that propaganda was disseminated to many people on social media, despite unanimity from US intelligence. If we as a society can't all agree that these things even happened, how can we hope to properly educate people on the existence/prevalence of fake video? It seems like common sense to many people, but we can't assume that everyone will accept these truths based on the political rhetoric we've seen in the last few years.
baxtr 8y ago
A crucial thing will be to detect these videos as they appear just like it is possible to detect Photoshoped material
evan_ 8y ago
Any "solution" to this that relies on crypto/signing will fail. Propagandists will just make their fake videos and then record them off of a computer monitor and claim that they're a whistleblower "leaking" the video from an internal system they can't otherwise access. I don't really think reputation has a better chance of solving this either, people are all-too-happy to believe anything that confirms their personal beliefs and ignore whatever contradicts them regardless of source. It's interesting that a lot of the comments here look at how to prove a certain video is real (and assume that people will accept that any video without a certain trait is fake) rather than proving directly that a certain video is fake.

71 points by creaghpatr 8y ago

I have to agree that this will be inevitable. Fake videos _will_ be created to spread misinformation. I bring you no solutions. Our best programmatic tool for these kinds of video analysis is currently the exact same technology that enables it in the first place. Furthermore, any neural network that can detect fake videos will only be used adversarially to train a better deep fake creator! The only solution I see is the camera somehow cryptographically signing the video, and enabling users to easily check whether the signature is valid. However I don't think this is feasible, due to the amount of cooperation we'd need between camera makers and video distribution platforms, especially since it's not really in the businesses direct line of best interest help. The world's going to start getting real surreal.

bytematic 8y ago
Can we develop something to put in videos that are real? Maybe something built into the software or maybe even some icon you have to hold up behind someone like the president to at least make deepfakes difficult/impossible? Just spit-balling as I also see no point in video analysis
- zzzcpan 8y ago
  We already have developed something. Only thing missing now is cultural shift in the media to apply a little bit of effort into signing videos.
  - sp332 8y ago
    That might be solving the wrong problem. You could believe that a video is from a particular journalist, but you might not trust that the journalist didn't tamper with the video before publishing it.
- prepend 8y ago
  You need to develop a unique, unpredictable set of expressions that deepfake will be unable to fake. Or perhaps active tattoos that display private key signed messages that describe what they’re saying and doing.
deleted 8y ago
[deleted]
creaghpatr 8y ago
The only thing that can be done is use contextual evidence to disprove the possibility of a video being real but that puts a disproportionate on the victim of a fake video. Will the government be in the business of commenting on the veracity of whatever video(s) they anticipate coming? And if they say it's fake (or real), will the public believe them?
- sp332 8y ago
  It will go back to being like a newspaper, where you can fake-quote anybody saying anything. Only established sources with some reputation on the line will be trustable. A random citizen won't be able to whip out a cell phone and get evidence that will be taken seriously. Credibility will be more centralized again.
zzzcpan 8y ago
To make it feasible you don't need much, just journalists signing videos and uploading them somewhere besides publishing anywhere unsigned. This will make every unsigned video easily verifiable, just not with crypto.
- binarymax 8y ago
  That's fine for journalists, but does nothing to prevent fakes captured by the populace. A good deal of our journalism now is by on scene witnesses. What can we do about fakes posted on social media purported to be the real capture of some scandal or event?
  - fooey 8y ago
    I think what we'll see is something less like the Jordan Peele version of Obama based on a public clip, and something more like a faked version of the Romney 47% video Start with something naturally unfocused and exists only as a one off original, and it would be extremely hard to disprove without comparing it to other video of the same event.
  - JetSpiegel 8y ago
    We are already vulnerable to this attack, they are just called "viral marketing campaigns".
- 21 8y ago
  In the last years there have been quite a few cases of journalists resigning from very prestigious publications (NYT, ...) because they were caught fabricating news.
  - zzzcpan 8y ago
    Yeah, well, it doesn't matter much whether they use false information or not, as their purpose is to paint a picture they want people to see either way.
- msla 8y ago
  > To make it feasible you don't need much, just journalists signing videos and uploading them somewhere besides publishing anywhere unsigned. And in order to implement that, you'll be talking to people who'd sign a video by getting a pen, some paper, and a flatbed scanner. I'm not entirely joking. My point is that this proves that there comes a point in every field where technical illiteracy goes from being annoying for you (you can't use the best tools anymore, and your workflow is now sub-par) to annoying for others (you can't collaborate effectively) to completely unworkable, as when your work product is now worthless because you can't demonstrate it isn't a cheap forgery. And some people are just not technical, through stubbornness or otherwise. This transition will hurt. Maybe tools will follow the demand. I'm not entirely sanguine.
isostatic 8y ago
The last 20 years certain elements have been pushing distrust and hatred for both politicians and journalists. However I'd hope that we move to some agreed standard of signing of videos, so their source can be confirmed by the player. It doesn't matter if that source is the bbc, washington post, alex jones, national enquirer, the onion, or "Bob Joe Billy-Bob the Third", it means you know where the video came from, and can trust (or not) it based on that information. A video with a bbc banner showing the president saying something could easily be faked. One that is signed by the BBC will only be fake if the BBC itself is publishing it as a fake, and that comes with a reputational impact. Sites that reversion that video (recompress it etc) can simply sign it by themselves. They could claim that they took an original BBC item to do it, and then it's how much you trust that site. People see someone, be it Donald Trump, or John Simpson, or Walter Mondale, and instinctively know that person said what they can see. That's changing, and it's going to be a long time before our brains change. Even now, you read some rubbish that you consciously know is rubbish, but part of it lodges in your subconcious and affects your thinking down the line. At least if you know who created the video, it gives you a fighting chance. With this system anyone can still publish, even anonymously, but reputations are hard to gain and easy to lose. I don't like the daily mail, but I know if they publish a video of Obama saying "Killmonger was right, and President Trump is a total and complete dipshit", I'll probably believe he did. If he didn't, that will come out, and the daily mail is off the list of sources I trust to not make crap up (rather than just bend things), and it makes it far easier for reasonable people to dismiss it. This moves the onus on news sources to be more discerning in their facts, but if they get it wrong then they lose their readers. By signing the news, they are backing it with their reputation. If Alex Jones has signed it, then he's putting his reputation on the line (if he has any reputation for saying something).
- vkou 8y ago
  > A video with a bbc banner showing the president saying something could easily be faked. One that is signed by the BBC will only be fake if the BBC itself is publishing it as a fake, and that comes with a reputational impact. Nobody cares much about the reputational impact. Fringe tabloid fake-news, and mainstream 'fake'-news gets plenty of viewers. BBC itself has manufactured video 'evidence' during the Libyan war. [1] Yet, most people think it's an accurate source of news. [1] Their 'live broadcasts' showed video of people in India waving the Indian tri-color, which the BBC has claimed was rebels in Libya's Green Square waving the old Libyan tri-color. But, eh, who cares about honesty - when you've got a politically sensitive story to tell.
JumpCrisscross 8y ago
> the amount of cooperation we'd need between camera makers and video distribution platforms Apple could single-handedly launch a "verified video" system.
- ttul 8y ago
  Dear Tim, Please build this. — everyone
avivo 8y ago
Unfortunately, even if we can pull it off "cryptographically signing a video" doesn't "solve" the problem. How would that verification be presented to a user? How/why would it influence their beliefs/actions? (As mentioned elsewhere in this thread, I go into a bit more detail here: https://www.washingtonpost.com/news/theworldpost/wp/2018/02/22/digital-reality/ https://www.washingtonpost.com/news/theworldpost/wp/2018/02/... ) Will try to write a HN friendly tldr/update soon.
oneshot908 8y ago
The best solution (IMO of course) is to stop unquestionably believing anything you see, read, or hear. Why most people choose otherwise has saddened me since childhood. The worst solution is to panic and attempt to ban technology like this insuring that only the bad actors will use it. I want everyone down to kids to be able to make fake videos to their little heart's content so we can all internalize this inevitable Black Mirrorization of reality. And your proposed solution is fantastic BTW. Unfortunately, there are more than enough bad actors among us to social engineer those signatures out of otherwise trustworthy agencies and a general lack of intellectual curiosity on the part of most people to even bother to check for such things. Imagine the day we have a video where HRC confesses to the pedophile colony on Mars and coming up next, Trump tells everyone where he buried the bodies, and if people look, there will be bodies there (never mind that they're stolen cadavers). How will people cope with a world like this? It is almost like we need pop art and pop culture to drive this point home before political mass media does it in the worst possible way.
- SmooL 8y ago
  | The best solution (IMO of course) is to stop unquestionably believing anything you see, read, or hear. Why most people choose otherwise has saddened me since childhood. Agreed, but I think it's been long proven most people do. People hear what they want to hear. Maybe that will change, but I'd say it seems unlikely at this point. | he worst solution is to panic and attempt to ban technology like this insuring that only the bad actors will use it. Very true. | I want everyone down to kids to be able to make fake videos to their little heart's content so we can all internalize this inevitable Black Mirrorization of reality. Increasingly getting there. It only takes one person to make a nice GUI, post it to github. | How will people cope with a world like this? Probably poorly if history has taught us anything. | It is almost like we need pop art and pop culture to drive this point home before political mass media does it in the worst possible way. Very intriguing. Traditionally, art and culture change as a reaction to current affairs, as opposed to a pre-emption. I'm not sure how to pragmatically flip that paradigm.
emodendroket 8y ago
Doctored audio and video isn't really a new problem though. More prosaically, you could just redact a recording to give a misleading impression and not rely on any fancy technology.

craftyguy 8y ago

I'd be more concerned with US officials using this technology to indict innocent people who disagree with them. Being able to conjure up a confession for anything by anyone is extremely powerful, given that the alternatives to accomplish the same thing today are basically apprehension and torture/threats.

frgtpsswrdlame 8y ago
That's an interesting application. With skepticism spreading about many forensic tools like fingerprints, burn analysis, bite analysis, etc. the confession has stayed the ultimate indicator of guilt to most of the public. I wonder if we may be entering another era of 'reasonable doubt'.
creaghpatr 8y ago
>Being able to conjure up a confession for anything by anyone is extremely powerful Being able to dismiss credible video evidence as 'deepfake' is equally powerful. In any case, it would be extremely risky to hang a case on a single confession video that the accused claims is fake, given the current pattern of prosecution the DOJ would never bring that case; it would be embarrassing in the public eye and probably overturned on appeal, if it even made it to trial.
- germinalphrase 8y ago
  Garry Kasparov has often accused V. Putin of maintaining his public influence by successfully attacking the idea that a verifiable truth can be established about his/oligarchical misdeeds. Surely, there are techniques to make educated assessments about the probability that [this] happened rather than [that] - but the public at large is neither literate in statistics nor receptive to uncertainty.
  - flyingcircus3 8y ago
    That's my view as well. In a time of information overload especially, people seem to be far more concerned in simply ending uncertainty, than validating the answer that precipitates that certainty. I believe this is referred to as the firehose model of propoganda. It effectively normalizes tu quoque fallacies, as the people whole have taken on a new set of facts are completely unprepared to debate the merits of those facts, and instead fallback to defending their newfound certainty.

mgraybosch 8y ago

You should never trust digital media. You should be extremely selective about what analog media you trust. If you weren't there to see it happen yourself, all you've got to go on is somebody else's perception. Be careful who you trust.

zkascak 8y ago
As a filmmaker I feel that completely distrusting a medium just because it is digital might be a little harsh. I think that we may want to consider using cryptographic signature methods as a way to mark materials as coming from the source they claim to be from. Though this is not a perfect solution. Though I still work with analog film, though it is becoming more and more cost prohibitive everyday. Analog can just as easily be altered. If you looked at a silver gelatin print, if the printmaker was talented you probably would never be able to detect the changes to the image that were made during the printing process. Is the resulting image a fake, or is it just an artists vision?

zkascak 8y ago

As a filmmaker I am both in awe and horrified with this. Some of these techniques can be used to make fixes in motion picture where it would be far too expensive to reshoot. But there is the opposite where this technology can be used to falsify a video that can potentially be used to make false confessions or change what a person said for nefarious purposes. Welcome to the world of what was once thought to be science fiction.

drivingmenuts 8y ago

When this happens (not if), I expect it will play right into the current administration's habit of dismissing anything that disagrees as fake news. For the administration that follows? Who knows? I only have evidence for right now. I do think most of it will come from people lower down the political food chain (the meme creators, etc) who seek to convince others of their POV out of desperation and/or maliciousness.

segmondy 8y ago

Anyone know where deepfakes moved to? It was great seeing what was going on in /r/deepfakes but due to the "offensive nature" of some of the videos that subreddit was killed. Anyone know the best place to watch and observe the latest happenings in the scene?

k__ 8y ago
Good question, I was just waiting for the all Nicolas Cage version of LotR
- prepend 8y ago
  It’s a bit of a pain to follow progress without the subreddit, but progress is still being made- https://youtu.be/QhxTTshL3b0 https://youtu.be/QhxTTshL3b0

avivo 8y ago

It's not a totally lost cause — I wrote a piece on what we can do about this: https://www.washingtonpost.com/news/theworldpost/wp/2018/02/22/digital-reality/ https://www.washingtonpost.com/news/theworldpost/wp/2018/02/... It's a first step, and my thinking has also evolved since—expect more updates in this space soon.

majos 8y ago
From this article: > AI researchers and platform technologists need similar in-house or third-party societal impact review boards to help them evaluate the potential unintended consequences of their work. As a machine learning researcher myself (albeit one who is probably unlikely to run afoul of one of these boards), this suggestion for research gives me pause. In the context of deepfakes, are you suggesting that it would have been more prudent to stop the research that made it possible before it had a chance to do so? This seems like a good way to ensure that less scrupulous parties harness destructive technologies before more scrupulous parties (the kind who'd adopt review boards). I imagine this debate to some extent has already played out with nuclear weapons and bioweapons. I'm much less familiar with these two spheres, but as I recall research continued in both.
- avivo 8y ago
  Lots of details to be worked out, but yes there are models in other fields. You bring up a real concern and there are ways to address it. I'm not suggesting stopping research on this. But the way in which that research is communicated, what the focus is, and the way which particular sorts of replication is enabled or discouraged has an impact. An example that the HN crowd likely knows better is vulnerability/exploit disclosure. There are good ways to do it and harmful ways to do it. You can perhaps think about deepfakes as an exploit for the human sensors/cognition that handle authentication.

gnicholas 8y ago

Will FB, Twitter, YouTube, and other social networks create tools to flag deepfakes? It wouldn't solve the problem, but at least it could blunt the spread. It could also raise awareness of the issue, so that if people see a video that seems "too good to be true" on a different platform, they would know that it might not be real.

mizzack 8y ago
Inevitably you'll hear cries of censorship with varying degrees of veracity.
- gnicholas 8y ago
  I would think that there'd be enough deepfakes on both sides that everyone would recognize that iff there are tools that accurately detect deepfakes, it would be useful to have them enabled on social networks. They don't need to censor the videos — just flag them as having deepfake characteristics.
  - mizzack 8y ago
    Like when Facebook (briefly) tagged articles from certain sources as being disreputable and it led to an _increase_ in clicks to those articles? People will believe what they want to believe. We're also seeing that they don't like mega corps acting as arbiters of truth. So, I'm not sure what, if any, effect this moderation/curation effort would have.
    - gnicholas 8y ago
      I think what we really need is an embarrassing term for "someone who was fooled by a deepfake" (like "catfish"), so that there will be a social stigma that incentivizes people to not be tricked by them. I don't currently have any better ideas that "getting deepfaked". Suggestions?

noja 8y ago

The only solution to this is signed live lifecasting to a blockchain for every person on earth. I'm only half kidding.

rohit2412 8y ago
How about a sort of steganographic signing with assemytric cryptography.

mjw_byrne 8y ago

Deepfakes are surely no more a "problem" than photoshopped still images? Everyone knows that a still image can be altered to (convincingly) depict something that never happened. So, a still image is not considered good evidence of anything unless it has a verifiable provenance, chain of custody etc. Presumably that wasn't always the case, but now that photo editing tech is universally accessible, we all know we can't trust a still image unless its authenticity can be corroborated somehow. The deepfake tech merely puts video into the same category of (un)trustworthiness. The potential for abuse depends on people's trust that video is always authentic. So the "solution" is simply to make it as widely-known as possible that video is now easy to fake.

roywiggins 8y ago
The volume alone might be a problem. It takes some Photoshop skills to pull off an acceptable fake. If there's a turnkey website that can spit them out en masse, that's something different than having to actually hire a human to do it by hand. Previously it took at least one CGI artist to convincingly fake video, probably several. This inherently limits how much of a problem it can be, because it doesn't scale. AI scales. I mean- either deep learning AI automation is going to be important and useful, or it isn't. If replacing all drivers with robots is important, then automated turnkey fakery has the potential for disrupting something, isn't it?
- mjw_byrne 8y ago
  I agree, but I think my original point covers this - if video proves to be very easy to fake at scale (e.g. using your example of a turnkey website) then that should result in equally large-scale scepticism about video as evidence. The internet may drown in annoying deepfake-based memes but as long as everyone knows its all fake, there seems little potential for the kind of serious harm people are worried about (reputational damage, election meddling etc.)
  - save_ferris 8y ago
    > ...as long as everyone knows its all fake, There's a shockingly large number of people who aren't even convinced that election meddling took place in 2016, or that propaganda was disseminated to many people on social media, despite unanimity from US intelligence. If we as a society can't all agree that these things even happened, how can we hope to properly educate people on the existence/prevalence of fake video? It seems like common sense to many people, but we can't assume that everyone will accept these truths based on the political rhetoric we've seen in the last few years.

baxtr 8y ago

A crucial thing will be to detect these videos as they appear just like it is possible to detect Photoshoped material

evan_ 8y ago

Any "solution" to this that relies on crypto/signing will fail. Propagandists will just make their fake videos and then record them off of a computer monitor and claim that they're a whistleblower "leaking" the video from an internal system they can't otherwise access. I don't really think reputation has a better chance of solving this either, people are all-too-happy to believe anything that confirms their personal beliefs and ignore whatever contradicts them regardless of source. It's interesting that a lot of the comments here look at how to prove a certain video is real (and assume that people will accept that any video without a certain trait is fake) rather than proving directly that a certain video is fake.