15 ms·
The US military is funding an effort to catch deepfakes and other AI trickery
- crb002 8y agoLol. More like DARPA wants to use deepfakes in psyops without getting caught.
- alexeldeib 8y agoCarries as much risk of being used against, no? If not more for getting fooled by deep fakes into doing something dumb. They want to weaponize it, sure, but protect themselves.
- some_random 8y agoWell yeah, though its worth noting that the potential damage is far, far greater than the potential utility. The Western Intelligence Community has incredible power and very little oversight. There's a long way to fall if public opinion suddenly swings against them because someone publicized a deepfake of CIA, DGSE, MI6, and AIVD agents ritually sacrificing a child.
- AnimalMuppet 8y agoI think the military wants to not have deepfakes done to them. Here's a video of your officer giving orders. Should you obey them? (Note that "no" as an automatic answer is just as bad as "yes". The only correct answer is "yes if authentic, no if not". But how do you know if it's authentic?)
- mathinpens 8y agoAnyone know the name of the grant/program/challenge ? I didn't see it in the article..
- joeyo 8y agoMediFor - https://www.darpa.mil/program/media-forensics https://www.darpa.mil/program/media-forensics Looks like the BAA came out in 2015: https://www.fbo.gov/spg/ODA/DARPA/CMO/DARPA-BAA-15-58/listing.html https://www.fbo.gov/spg/ODA/DARPA/CMO/DARPA-BAA-15-58/listin...
- sandworm101 8y agoWhy bother creating these fakes? Saying something loud and often enough is all you need. Throwing money at social media is far more effective than creating deepfakes. Also, if the US military did spot a deep fake, do they have the trust for people to believe them? They are picking an uphill fight.
- josefresco 8y agoThese fakes are just another tool like social media. Social media alone might be effective, but add in these "deep fakes" and it becomes even more.
- russdpale 8y agoFor a one time 'hit' on a target, knowing fake news spreada faster than true news. By the time its shown to be fake, the damage has been done to the target.
- unit91 8y agoThere's also a lot of potential criminal use for them, like proof-of-life for a person who's actually dead.
- some_random 8y ago>Saying something loud and often enough is all you need. Depressingly true, but keep in mind that words < pictures < video when it comes to provoking an emotional reaction. >Also, if the US military did spot a deep fake, do they have the trust for people to believe them? Depends on how their deepfake detection method works, if it's opensource, or at least publicly available and verifiably functional, then who wouldn't believe them? No one thinks their GPS is lying to them.
- dzink 8y agoAny Deepfake detection tool would also be a GAN sparring partner for the Deepfake makers.
- EGreg 8y agoIf you consider this a two sided game, then you can have the fakes quickly become so good that hardly anyone can tell. And I'm not just talking about image fakes. You can have MCTS find the best arguments for ludicrous statements, and other paths to justify fake things that look just like real arguments.
- ben_w 8y agoThat is the point of a GAN. We’re missing something about the problem space and/or human cognition or their output would already be indistinguishable from reality to all humans.
- varelse 8y agoYep, they're a 3D reconstruction of points from a 4D space a (massively simplifying the fact that the 2D video frames themselves are high-dimensional data) and that's just the video. Bring a 3D camera into the game and the underlying distribution and see if hilarity ensues IMO.
- amelius 8y agoHence they probably shouldn't make it public for it to be effective.
- musage 8y agoEffective how? The article quote someone talking about "saving democracy", but the use of the military saying "we think X is fake but we can't tell how we came to that conclusion" seems very limited, and depending on the content and target audience of the fake would just get a shrug and "of course they'd say that". Are there even past examples of any military, US or otherwise, injecting itself into public discourse over something to clear things up, instead of doing the usual opposite?
- 2bitencryption 8y agoSlight tangent, but let me just say that "deepfakes" is the perfect sci-fi terminology for the technology. I could imagine an entire series (or part of a series) about "deepfakes". I hope the name sticks, as the technology inevitably becomes more common.
- romaniv 8y agoRight now the visuals for these fakes seem to fall apart at higher resolutions or when there is any significant head rotation. This is actually better than what we have in the realm of images, where almost anyone can create a "realistic" Photoshop of the wast majority of scenes.
- bhhaskin 8y agoIt's only a matter of time though.
- Florin_Andrei 8y agoLikely you're right. OTOH, when time is a factor (as in movies - temporal succession of images), perhaps simple curve fitting may never be quite perfect? Perhaps you need anticipation, and counterfactual thinking, cause and effect, and all that? Also, like in video games, maybe even some understanding of real world physics may be required for a perfect fake.
- yorwba 8y agoMost of early physics was done by curve fitting, and on very small datasets. Simple Newtonian dynamics shouldn't be much of a hurdle for a neural network. That's about as much anticipation of cause and effect as you'd need for faking most kinds of videos.
- John_KZ 8y agoIt's already possible, but nobody has written a public-domain implementation of it. Probably because it can sell for a lot of money. I bet someone has this software right now.
- calebh 8y agoIs there a way to cryptographically sign a video to make sure that it's tamper free? I'm thinking about some sort of encryption that happens directly on the camera while a video is being recorded.
- londons_explore 8y agoThe camera could sign it while recording... But whats to stop someone reverse engineering the camera and using the signing key to sign something else?
- some_random 8y agoWouldn't that also allow one to determine the camera with which the video was recorded? Not exactly something a whistle-blower would want.
- danenania 8y agoYou can use a signature to verify that a video hasn’t been modified, but that doesn’t really help in itself with fakes since fakes can be signed too. And if the cameras are doing the signing, you’d just need to extract the private key from the camera and use it to sign fakes. It would require something like the secure enclave to protect the private key and, of course, we’d then be trusting the camera manufacturers.
- JoshTriplett 8y agoNot to mention creating a world in which people can't build their own cameras, or change the software on those cameras...
- ViViDboarder 8y agoSure you can. Just wouldn’t be usable as factual proof of an event.
- smallnamespace 8y agoThe ability to fake something is equivalent to the ability to rewrite the past. As our ability to fake things increases, more and more of our knowledge then comes into doubt. I think we'll end up in a world where the only way to prove what the past really is if we hash and stuck on a blockchain somewhere. One can then see PoW as a constant tax being paid to maintain a literal link to a past when deepfakery did not exist.
- londons_explore 8y agoFancy making a bot which crawls the web, makes a massive merkle tree, and then puts the head of the tree into a bitcoin transaction? You could then monetise it by charging a nominal fee for the path from any given bit of data to a timestamped bitcoin transaction. The disadvantage is the cost to do this involves "get access to all data on the web", but perhaps you could partner with the web archive or something to do it?
- Xeoncross 8y agoFrom what I see of history, as humans we already do this by killing of all "invalid" copies of history (books, art, humans, etc...) and replacing them with the new "fixed" versions. Now that making copies is as easy as owning a camera or USB drive it is dis-information that is the new way to change history.
- spookthesunset 8y ago> I think we'll end up in a world where the only way to prove what the past really is if we hash and stuck on a blockchain somewhere LOL. Blockchain to the rescue! Except. Which fork is the right one? The one with the most hash power? How can you be sure that one is right? Which ethereum fork is the "correct" fork? Which bitcoin fork is the "correct" fork? Blockchains don't help for shit with proving something existed. The are fully mutable data structures--it just requires more effort than a simple UPDATE statement.
- toomuchgov 8y agoOr, you know, printed books from the past. Real physical, historical evidence.
- majos 8y agoSide note: I wish the "killer app" for deepfakes hadn't been porn. The technology was impressive, but having such a scuzzy popular usecase made sharing it with others kind of fraught.
- bhhaskin 8y agowhat would the other use be? Manipulation of new and media?
- Maybestring 8y agoTV/Cartoon mashups, AR 'stickers', plenty of silly entertainment possibities.
- dfxm12 8y agoPorn was the killer app for online video streaming and secure online payments too, though. The Internet owes a lot to porn.
- crumbshot 8y agoSexual gratification is one of our most primal urges. Makes sense that it would be a driving factor in this, despite the strong taboo in some cultures.
- satokema 8y agoHalf the galaxy's engineers are busy trying to jam the AIs while the other half are busy trying to jam the AI jammers.
- Florin_Andrei 8y agoStarts to resemble biologic ecosystems.
- TeMPOraL 8y agoAnd here we thought we're better than nature. No, we've just replaced competition of designs with competition of goals.
- mjh2539 8y agoWhy isn't this a solution: Everyone has some gpg-like setup. If we see a statement or video or audio by a person accompanied by a public (graphic or audible) key that checks out...ok, done, verified. If we don't, then we can justifiably disregard the statement or video or audio.
- falcon620 8y agoI really like this idea. Every video frame from a political/public personality is cryptographically signed with their private key and edge viewing devices are able to tell if it's properly signed or not. It's easy to imagine this idea becoming a part of video (be it VOD or broadcast) encoding standards in the future, once this becomes a critical thing. In terms of UX: one could imagine a simple indication on TVs: an RGB led that shines in different colors depending on verified authenticity levels.
- cortesoft 8y agoMost of the shocking, status-quo challenging videos these days are cell-phone videos taken by bystanders. Those videos are how we find out about police brutality and other abuses by authorities. I don't see how this video signing system will help verify a cell phone video taken by a random bystander.
- falcon620 8y agoWell, no, it won't. It will however make sure that you're watching actual Obama 2, and not some computer simulation of him doing some disgusting thing.
- jcims 8y agoSensor supplier could embed the crypto in the hardware as close to the imager as possible and issue signing keys for each device. OEM would write firmware to extract that from the camera bitstream and dump it into the metadata of the image (something along those lines anyway) The image and signature could be added as authentication layers and then a third layer added to allow post-production (crop/levels/etc). Right click in your browser to 'see original image' with a little padlock in the corner. Ideally this would include GPS info to reduce likelihood that someone would just project a desired image on a screen and photograph that.
- michaelbuckbee 8y agoHumans have such issues with confirmation bias [1] that I worry it's not going to matter. The last election already had algorithmically generated artificial news designed to sway people (note: I'm talking FB articles with headlines like "$POLITICIAN just insulted $NICHE_AUDIENCE. STOP THEM." that linked to articles that were scraped/spun from other sources). They relied upon people not actually reading and just hitting like/forward/heart whatever and spreading the top level message. We are not going to be able to convince ourselves, never mind other people, that video (which has been the gold standard for "truth" for nearly a century) isn't actually real. Case in point: https://www.youtube.com/watch?v=cQ54GDm1eL0 https://www.youtube.com/watch?v=cQ54GDm1eL0 (which was included in the article) has comments from people deeply confused by it on YT. 1 - https://www.newyorker.com/magazine/2017/02/27/why-facts-dont-change-our-minds https://www.newyorker.com/magazine/2017/02/27/why-facts-dont...
- 0xfeba 8y agoWell hopefully there will be such an influx of fakes everywhere that people will at least begin to accept that anything can be faked, and hopefully take the next step of thinking a little more critically about what they hear next.
- craftyguy 8y agoI see it going to the other extreme, where people believe anything can be faked, and politicians, criminal, etc start to dismiss evidence of their wrongdoings as fake, and people believe them.
- laythea 8y agoIf such a system to automatically detect this was created, then I would suppose it would become the story instead. Whatever method used would have to come up with a reference photo (or "model"), ie. what the photo "should" be. And this tech can then be used for the purpose of more convincing deep fakes and other AI trickery. Hurray! Progress! :)
- AlexandrB 8y agoWe're rapidly heading into a world where every form of non-fictional media has to be mediated by a trusted third party because we have no way to distinguish what's true from what's fabricated with our own senses. This is a complete reversal of the verifying power that images, audio, and video once provided to everybody regardless of income or education. Think about some of the potent imagery that came out of wars like WWII and Vietnam, for example. We're not too far away from the manufacture of literal "fake news" out of whole cloth. I'm not sure this bodes well for the idea of an informed electorate.
- deleted 8y ago[deleted]
- ballenf 8y agoWe may later view the short period in which A/V recordings as authoritative over first-hand accounts was a brief "glitch" in the history of the world. The ability to fake such things means that we have to return to trusting our fellow human and making wise choices about who can be trusted. Fingerprints, DNA samples will remain of varying degrees more difficult to fake, but one must assume that those too will fall as paragons of unassailable guilt (or innocence, but usually guilt).
- sambull 8y agoThose should already have fallen. Gross human incompantance/perverse incentives to use science as a blackbox for prosecution
- tonto 8y agoYou really just don't like forensic science or what
- on_and_off 8y agohttps://www.wired.com/story/dna-transfer-framed-murder/ https://www.wired.com/story/dna-transfer-framed-murder/ DNA is not the end all of an investigation
- freeflight 8y agoBlaming fakes for all kinds of things is just the extension of the "war on privacy and anonymity". Why are fakes supposedly such a big problem? Their identity might be fake, but that doesn't make their ideas any less attackable and that's what it should be about: Ideas, not identities. If an idea is good I couldn't care less who had it, I only care about the idea itself. Yet all the public discourse focuses on "fakes spreading wrong ideas", even the recent Facebook EU hearing had that as a major topic, with Zuckerberg constantly reiterating that catching fake profiles, who could influence elections, is one of Facebook's top priorities (which I don't doubt). But barely anybody seems to make the effort to think this trough consequently, once you do that, you realize that we are heading in exactly the same direction China is heading: No anonymity, Social media becoming the de-facto replacement for government institutions. Is that really the world we want to live in?
- jimrandomh 8y agoThey are specifically worried about attacks on politicians in the form of fake videos that look like those politicians saying incriminating things. In that case, it matters a great deal whether the person said those things or not.
- s2g 8y agoStating lies as fact isn't spreading ideas.
- cortesoft 8y agoThis isn't about ideas, it is about faking facts. You are telling me you don't care if someone created a video of you viciously murdering someone, and then that video was used to convict you? Sure, ideas can stand on their own, but we have to have facts that we can know so that we can use those facts to judge the ideas. If we can fake images and videos, we can create fake facts, and that is certainly dangerous.
- mehrdadn 8y agoAre there high-resolution deepfakes out there yet, or are they within sight? One s that you couldn't catch by your eye if you just zoomed in a bit?
- ryanwaggoner 8y agoI don’t understand the hand-wringing about this. We have had this problem for decades with all forms of media other than video. Text, photos, and audio are all easily faked and can be done so to a degree indistinguishable by 99.99% of people. Why weren’t we all terrified that bad actors would create fake images or audio files that the stupid electorate would find so persuasive that it would overthrow governments and swing elections? What makes video different? And in particular, why won’t video quickly join the list of things that people don’t trust without a credible source vouching for it, like we do everything else?
- artemisyna 8y agoAlas - the 200 years or so by which humans had visual 'proof' that something did or did not happen is rapidly about to end. :)
- L_226 8y agoA potential stopgap: increase the necessary computation power required for fakes by only releasing 'official' media with multiple viewing angles and device signatures. e.g. press conference containing multiple mobile phone camera videos of the speaker from different audience perspectives. Probably the models will eventually catch up and allow this too, to be faked.
- 23443463455 8y agoQuite interesting is the statement about protecting democracy. Democracy has been so perverted and set up for this very moment of vulnerability over the many mutations and perversions that have been introduced in ill guided and mindless support of ostensible “improvements”. To anyone who can evaluate things objectively and isn’t swayed by propaganda, it is quite clear that the world is heading towards authoritarianism as democracy is being dismantled and perverted and poisoned right before our very eyes to the cheers of the ignorant masses and “experts” and the “educated” who always find themselves at the heart of all human calamities. So it’s what we are doing, authoritarianism our of utter ignorance for how to preserve what others handed created and handed us. At least it lasted for going on 250 years, let’s try more authoritarian socialism. That will surely work … until the elite reveal that they were just using it for a bait and switch. Oops. Can we have our democracy back, we will say; and the inevitable response will be, nah, we kind of like that you gave us control of your lives, we choose to keep it that way, sucker.