9 ms·

Deep fakes are yet another prime example of how the writing is on the wall (i.e. it's clear that this technology can/will be used for nefarious purposes - esp.

• javanix 7y ago
  What regulations / standards would actually stop these things from getting out though?
  • eyegor 7y ago
    The only answer here is drm esque file signing. Any attempts at detecting fakes will only work while the tech is young. Ofc this requires a web of trust and verification techniques to be applied, since videos are often re encoded.
    • make3 7y ago
      you mean for literally every piece of video taken everywhere..
    • saalweachter 7y ago
      Ok, I've signed my cellphone footage of a crime being committed. Now what? How does that prove my footage was taken where I said, when I said?
      • eyegor 7y ago
        I was thinking more along the lines of hardware manufacturers implementing a secure enclave style of signing files as they're recorded. The sig could be tagged with location/date if available, but at least the video would be verifiable. The web of trust would have to exist for reprocessors and media outlets who like to edit footage.
        • jdietrich 7y ago
          >at least the video would be verifiable Nope, it'll just give well-executed fakes a veneer of legitimacy. A state-level actor will have easy access to the signing keys. Non-state-level actors could potentially extract them from a device or bribe an engineer at a third-tier Chinese OEM. A reasonably competent hardware hacker could desolder the CCD from a signed device and feed whatever video signal they like into it.
• lowken10 7y ago
  We’ve had the ability to doctor photos for 20+ years now and it has not been an issue.
• umvi 7y ago
  So you think it would be better if politicians have a knee jerk emotional reaction like with 3d printed guns and start shutting down research and making it a felony to publish deep fake videos?
  • canada_dry 7y ago
    Another recent example where regulations were (relatively) quick to address some new tech/gadget: drones. It was recognized by law makers that drones could be used for nefarious purposes and needed some form of regulation. I'm not saying that drone registration thwarts illegal use, but at least something was done by regulators.
    • icebraining 7y ago
      > at least something was done That's an example of the politician's fallacy. Doing something can be much worse than doing nothing. Did drone registration actually improve anything?
    • alpaca128 7y ago
      That drones could be used for nefarious purposes was kind of obvious after years of deadly US drone strikes. That didn't seem to tip off regulators, however. Not until anyone could buy drones capable of carrying heavy cameras or other equipment anywhere. Either way these regulations don't help much when people just don't care, as can be seen with all the drones flying too close to airports.
      • zentiggr 7y ago
        Pretty sure nobody was going to buying surplus/refurb Predators and flying them around the neighborhood. When the recreational/commercial models got range and mass enough to get in the way of aircraft, that's when the first round of "whoa!" kicked in. Still going to take more work to figure out reasonable controls.
    • jdietrich 7y ago
      >I'm not saying that drone registration thwarts illegal use, but at least something was done by regulators. Drone regulation has done precisely nothing to thwart illegal use, while imposing substantial cost and inconvenience on legitimate actors. The laws don't stop me from buying a heavy-lift drone from China, nor do they stop me from doing something nefarious with it. At best, the regulations have slightly reduced the risk of inadvertent airprox incidents, but they have been utterly useless in addressing the sorts of risks they were supposed to combat.
• gojomo 7y ago
  The sorts of bad actors or 'enemies' who'd deploy deep fakes for advantage aren't typically discouraged by 'regulations'. On the other hand, there is a reasonable amount of active research on both detecting current faking-techniques, and methods of adding cryptographic attestation from point-of-recording. I don't believe "detection" can win in the end, as widespread detection-technology can generally be used to tune better fabrications. So ultimately we'll have to rely on: "do we trust the specific chain-of-people-and-sensors-and-relays that brought this evidence to our purview?" And various kinds of constantly-applied cryptographic signing & timestamping can help with that, though interpreting the challenging cases will require a lot of abstract expertise. (So again, for most people, it may reduce to: "who do you choose to trust?")
  • extrapickles 7y ago
    I wonder how long it will be before someone stuffs a crypto module into the same package as a PDM microphone to provide a digitally signed audio stream.
  • throwaway5752 7y ago
    Yes, we should not underestimate plain old social solutions. I remember when you used to answer the phone every time because caller id didn't exist. Who answers their phone now? This just mainstreams exceptional validation workflows that exist in finance for establishing identity and preventing phishing and other forms of identity attack. We just won't be able to trust anyone is actually saying anything except after confirmation via non-repudiable channels.
    • armagon 7y ago
      I think this is a regional thing. I'm from a rural community in Canada, and of course we answer the phone. Just floors me to visit my mother-in-law in Texas; when we visited eight years ago, she'd answer the phone; now she doesn't answer her cell or her home phone unless it makes a distinctive ring. I'd hate to need to get ahold of her using someone else's phone.
      • 7y ago
• deleted 7y ago
  [deleted]
• make3 7y ago
  that's an issue with electing presidents and powerful people that are over 70 years old lol
• philwelch 7y ago
  We’ve had photographic deep fakes for decades now. How does this fundamentally change anything more than Photoshop already did?
  • CamperBob2 7y ago
    "We've had computers for decades now. How does this fundamentally change anything more than the IBM 360 did?" -- investor being asked for funding in 1978 by a couple of hippies in Cupertino
    • philwelch 7y ago
      Do you actually have a response or are you just being contemptuous and dismissive to amuse yourself at my expense?
      • CamperBob2 7y ago
        Well, if you put it that way... :-P It's well understood that quantity -- and availability to the masses -- has a quality all its own. I do believe we're facing an arms race, in which purveyors of synthesized bullshit will likely fight the defenders of truth to a draw at best, and more likely win. No disrespect intended but I don't agree that your Photoshop analogy is valid.
  • alpaca128 7y ago
    Photoshop doesn't have a "put this person's face on this other person's shoulders"-button that anyone can use. Creating convincing deepfake videos is still a lot of work, but a couple years ago altering one's face in live streams took also much more effort than pressing a button in a free mobile app.
  • Bartweiss 7y ago
    We've also had video since before convincing photo edits were available. So people learned that we shouldn't trust a photo of Martians, or a sound clip of the President confessing to murder, but should hold out for video. When footage of Rob Ford doing crack cocaine shows up, we believe it's real. And when an investigative reporter wants a verifiable record of something, they resort to a high-quality video. Even if video deepfakes aren't any more realistic than image or audio fakes have been, they're important because there's no fallback. With perfect-accuracy photo and audio fakes, we'd have to be more skeptical. But with perfect-accuracy photo, audio, and video fakes, we'd suddenly be back to the pre-photography era when there was essentially no way for an untrusted source to convince you that something had happened. Deepfakes aren't flawless, we're not at that point, but the specifics of video are less important than the impact of having some medium which can't be convincingly faked.
• 2804t3qwp 7y ago
  I'm curious to know why you feel like regulation and standards are the solution here. It seems fairly clear that the propaganda metagame heavily favors disinformation and is only getting better at creating disinformation products. If we accept that premise, then it seems like the real problem is that we're still trying to preserve the idea that the internet is a tool for learning. Certainly parts of the internet are, but if the bloggers vs. professional press are anything to go by then it seems like the most practical solution is to establish lists of trusted entities and the contexts in which they're trusted. The managing of lists is regulatory in a sense but the trust side of the equation seems more social than bureaucratic, people need to be much less trustful of the internet.
• jrockway 7y ago
  People have been faking videos for as long as video has existed. Deepfakes is just a scary new word for what people would call "CGI" or "VFX".
  • jdietrich 7y ago
    > Deepfakes is just a scary new word for what people would call "CGI" or "VFX". "Ryzen 3950x" is just a scary new word for "z80" or "6502". Deepfakes don't provide any fundamentally new capability, but they do reduce the cost and time required to create a convincing fake video by multiple orders of magnitude. That completely changes the threat model, from "our propaganda rival occasionally releases a fake video that we have to debunk" to "our propaganda rival is producing thousands of fake videos every day and we can't even keep track of them".
    • Bartweiss 7y ago
      > Deepfakes don't provide any fundamentally new capability I'm not sure this is entirely true, although your larger point is quite good. People have been faking video since the dawn of video, yes. But for several decades, a clear, high-res video of a specific person has been pretty much inviolate. Faking a steady closeup of a president or famous actress would have been unthinkable - no VFX hoax ever convincingly impersonated Nixon. As a result, bad actors had to resort to deceptive editing, low-quality "covert" footage, or very rarely an exceptional look-alike. Even Hollywood had to rely on old footage and rewrites when an actor died. Today, we're right on the edge of changing that. MIT can revive the President, but not perfectly, and only to the standards of a 70s news video. Lucasfilms can revive Carrie Fisher, but not well enough to fool an alert viewer, and not in a natural/unedited setting. Within a few years, we might hit a level that tricks even the most acute viewer, and is only caught by forensic analysis. We've been there with photographs for years, but it's new ground for video.
      • pc86 7y ago
        The point is that nobody talks about making altered photographs illegal, or having federal regulations around altering photographs, or anything remotely as ridiculous. But because it's video everybody is losing their minds.
• csommers 7y ago
  "can/will" - deception of this magnitude has already been seen: Adnan Hajj.
• krick 7y ago
  This comment kinda scares me, because it is exactly that panicky reaction that makes all these stupid regulations come to life, and you are reminding me that it's not really "just stupid politicians", but in fact scared, easy to manipulate crowds behind them.
  • undersuit 7y ago
    What if we put the regulations and enhanced standards on to the people who we are most worried will abuse our trust? We'll never be able to stop a malicious actor from Deepfaking our leaders and spreading disinformation, but at the very least we should have something in place to make sure our leaders aren't the malicious actors using Deepfakes and react accordingly with punishments when they are.
• Bartweiss 7y ago
  As far as technical responses, there are at least four different goals to pursue regarding fakes: 1. Origin tracing. This is the microdot example: it's not meant to catch deepfakes, but link nefarious instances to their source. But the existing technical options here seem to be a mix of privacy-eroding (printers are dumb compared to phones/computers, and you'd have to prevent or ban sharing non-marked media) and ineffective (bulk color copying requires physical access to a device that's hard to make or modify; deepfakes can be constructed on a server in another country, or have their identifiers scrubbed after the fact). 2. Reactive, general verification. This is just an arms race between fakers and observers, like catching art forgers. Existing fakes have tells like a modified 'halo' around faces. Right now we only see manual checks, but major content hosts like Youtube could flag "suspected deepfake" like they do "music copyright strike". (Or hopefully better than that...) But it depends on staying ahead of fakers, and once the tells are too subtle to simply watch for it will only work when hosts or viewers choose to validate content. 3. Proactive, general verification. This corresponds to hard-to-implement, easy-to-verify security features like UV watermarks on money or prescriptions. But those rely on controlled supply, and decades of DRM failures tell us that digital fakes are much easier to make and safer to pass than physical ones. I don't expect this to expand beyond closed groups like news orgs giving out auto-watermarking cameras. 4. Specific authentication: not eradicating fakes but proving certain videos are legitimate. This is the most plausible, interesting category. We can't even prevent manipulative photo edits today, so we're unlikely to prevent manipulative deepfakes, but today we can prove specific aspects of specific images are legitimate. People will still believe fakes and lack proof of some real events, but this prevents a more fundamental transition to a "post-truth" era; we'll still have known-good records of key events. We've had low-tech authentication since the dawn of photography - think of hostage photos taken with a daily newspaper to prove "this image is newer than this day". As photo editing emerged, steganography developed to catch out altered elements. Cryptographic signing took that further, allowing us to prove that an image is unaltered from a specific keyholder. We even have the reverse of the old newspaper photo; publishing a hash or encrypted file lets us date an image back to a specific time without having to actually release it. Proving that a file is authentic to the world, not just an owner, is trickier. But we already have some steps: deepfakes take time, so any livestreamed video is not being edited that way after the fact. Authenticating something time-specific like a Presidential speech would only require combining that rapid turnaround with proof that the video wasn't prepared in advance; until on-the-wire editing becomes convincing a newspaper in the background would suffice. Quite likely we'll see more complex arrangements eventually, like trusted hosts that issue random values and demand their use in rapid responses. None of this is going to stop people from believing fakes, but nothing ever has. What's more significant is whether we maintain the ability to create records which can be verified and trusted.
• SimbaOnSteroids 7y ago
  naive question, won't any deep faked video leave some pretty readily detectable artifacts in the underlying video file?
  • godelski 7y ago
    If they are visible to the model, then these artifacts could also be generated.
    • SimbaOnSteroids 7y ago
      I mean the model producing artifacts that wouldn't be found in the genuine article.
• pc86 7y ago
  This comment is so egregiously out of touch I hesitate to even respond at all, let alone to do so civilly, however in the interest of discussion I'm going to try to. Which "enemy" do you envision would use deep fakes for a nefarious purpose, yet will stop short once they realize there are - gasp! - regulations around them? And as far as I'm aware, at least in the United States, there are no regulations requiring the use of microdots in printers or copiers, they're done of the manufacturers' own accord to aid law enforcement. The idea that we should have ill-informed, knee-jerk regulations in reaction to every mildly upsetting technological fad is antithetical to everything the vast majority of technologists believe in and stand for.
  • fossuser 7y ago
    You said you'd try to respond civilly, but your comment is openly hostile and this takes away from your valid core point.
  • gambiting 7y ago
    It just reminds me of my 85-year old grandfather who still in 2019 maintains that photocopiers should be banned for private ownership because people can make illegal copies of documents on them.
  • briffle 7y ago
    I get your point your making, but I think your reacting a bit harshly. We don't need one regulation that solves all our problems. We can have 'defense in depth'. Current laws that put microdot detection in color scanners will not deter huge resources to counterfeiting like state backed operations. But they do prevent the local meth junkie down the road from making a bunch of fake 20's to get his next fix. At the sametime, the paper that we use is also highly regulated. Its not impossible to get something that is similar, but its not easy. (and not cheap). The points of all these layers, is to prevent the casual and common crimes. By doing that, you can spend your resources on the larger operations.
• syshum 7y ago
  Sadly they do not really need deep fake technology to spread fake news