10 ms·
Stopping deepfake news with an AI algorithm that can tell when a face doesnt fit
- turblety 6y agoNow we can use this new AI to train another AI that can defeat it. And so continues the great cat vs mouse chase. I don't think this is a problem that is ever going to be solved. Deep fakes will become more and more popular, and harder to detect.
- IgorPartola 6y agoI feel like it’s time to bring in Heinlein’s Fair Witness concept: https://www.urbandictionary.com/define.php?term=Fair%20Witness https://www.urbandictionary.com/define.php?term=Fair%20Witne... If you think about it, when we had relatively few journalists and their reputation was on the line with every story they wrote they in a way were fair witnesses. Since we have moved to a hybrid model of professional journalists and a huge caste of YouTubers, podcasters, bloggers, and twitters, it has become impossible to hold everyone to a high journalistic standard. At the same time Fox News has brought on an era of politicized news so the bias is now inherent. Maybe if we had a few hundred fair witnesses that offered a professional service we could have a way to verify facts. Then again, it’s one thing to have a fair witness present facts, it’s another to have that witness’s likeness be deep faked and broadcast all over the web saying something they didn’t actually say. Another solution may be regulatory: anyone broadcasting a deep fake with a huge disclaimer that it is a fake gets a 20% of their net worth fine. Draconian but I can see it working.
- ben_w 6y agoWe need something, but Fair Witnesses seem both exploitable and unachievable given the way human memories work.
- dvtrn 6y ago“Expert witnesses” then?
- ben_w 6y ago“Expert” is a misused term, much like “exponential”. I see the value of the press as being as much of an intelligence agency as the combined CIA, NSA, and FBI, but serving the electorate and investigating all those with power — politicians, police, military, religious, business, and so on — so that the electorate can make informed decisions. I don’t know how to get there from here. Heck, I don’t even know if we’ve ever really been there, or if the press has always been comfortably manipulated by those with power. But yeah, I assume genuine experts are part of the solution.
- contravariant 6y agoOff topic, but 'exponential decrease' is my favourite misuse, simply because exponential decrease is slower than a linear (steady) decrease.
- contravariant 6y agoEven if achievable it seem inhumane. And besides it seems far more achievable to just have trustworthy journalists, even though that's already difficult in itself.
- amelius 6y agoAntibiotics resistance is also a cat and mouse game. It's still useful to keep playing the game though.
- sildur 6y agoThe difference is that all the bacteria can become resistant to the antibiotic soon after it is created.
- dijksterhuis 6y agoAs can deepfakes. All it takes is an additional step to optimise wrt the model that tries to catch it.
- sildur 6y agoYeah, I was dismissing the analogy with antibiotics because usually it takes quite some time between creating the antibiotic and germs being resistant. But with deepfakes the arms race is almost instantaneous. The moment something appears to tell apart deepfakes, the moment people can train deepfakes against that something.
- amelius 6y agoPerhaps the deepfake checking should be a third-party service? So you can only check so many deepfakes per day, limiting these attacks (i.e., you can't realistically put the checking inside a training loop). Just an idea ...
- deleted 6y ago[deleted]
- nottorp 6y agoRef: http://www.gutenberg.org/ebooks/29579 http://www.gutenberg.org/ebooks/29579 Robert Sheckley's Watchbird
- formerly_proven 6y agoIsn't this the idea of getting better results using the adversarial network approach? So this would be inherently ill-suited to stopping deepfake news, yeah?
- DarthGhandi 6y agoit's great at making them better, yes
- giancarlostoro 6y agoIf the people trying to stop deepfakes can do it so can the people trying to produce them anyway. The best we can do is some way of digitally signing videos and software displays it as authentic or not but then they just make fake youtube style sites that say its authentic. The problem will go on and on.... Fake news will always spread for one reason or the next.
- the8472 6y ago> but then they just make fake youtube style sites that say its authentic. Even simpler, someone will build an optical setup that sends deepfake pixels onto a signing image sensor.
- harryf 6y agoSeems like an AI arms race just begun
- jcims 6y agoIt's just a slobbering infant trying to roll off the bed at this point. The battleground of what is real and fake will very quickly move into 'superhuman' realms of sensitivity, leaving us meaty minions as spectators trying to figure out which AI to trust.
- tersers 6y agoI used the AI to destroy the AI
- nolaspring 6y agoSomeone just found the other half of their deep fake GAN
- JimiofEden 6y agoThis seems like test driven development, but applied on a much larger scale.
- alkonaut 6y agoThis seems like it’s “adversarial” to deep fakes. I wonder what that can be used for.
- arielbaz 6y agosee also "Training a deep learning model for deepfake detection" ( https://news.ycombinator.com/item?id=22433711 https://news.ycombinator.com/item?id=22433711 )
- guscost 6y agoCool tech, but most people will have tuned out what we know as "the news" (or become jaded to its purpose beyond entertainment) before a tool like this is necessary.
- chiefalchemist 6y agoAI novice here. It would seem to me that the detection algorithm can be repurposed into making the original less detectable. A recursion that never ends. An advantage quickly becomes disadvantage. Worse. All information - even facts and truth - has been subverted. What happens when there is no trust? Is this not a road to the New Darker Ages?
- 1f60c 6y agoThat's what I'm thinking. I don't want to diminish the value of this research, but this cat-and-mouse game is like a GAN[0] with extra steps. [0]: https://en.wikipedia.org/wiki/Generative_adversarial_network https://en.wikipedia.org/wiki/Generative_adversarial_network
- yes_man 6y agoThe problem will increasingly be "whose algorithm do we believe". Internet has revealed that people believe mostly what they want to. We have seen that a large subset of people are believing Bill Gates is behind the pandemic. Why would masses somehow be more rational in picking the most rigorous and objective neural network to recognize deepfakes, than they are in making sense of the world in general? In the end we will have multiple competing entities claiming to have the best deepfake recognition, all with their own agenda
- ericlewis 6y agoI think religion might have proved that people will believe whatever they want to first. The book Sapiens really opened my eyes to people.
- paulryanrogers 6y agoWell it took me 35 years but I escaped religion despite being indoctrinated from a young age. And leaving was a difficult choice but education and evidence were the keys that freed me. Piercing cognitive biases is hard, maybe harder if those most in need feel forced.
- TheSpiceIsLife 6y agoBe kind to yourself and know that a fair swathe of the early years of life are necessarily of low agency.
- ericlewis 6y agoI could have clarified more- I’m saying that even with these sort of deep fakes that breaking people out might be just as hard. Especially if objective proof becomes “malleable” similar to how religion can lack objective proof, manipulating people in a new way means we need methods.
- paulryanrogers 6y agoYour point seemed clear enough to me. Perhaps my response drifted too far. I see education as a common solution to the "it's all relative" and "no one can know for sure" shortcuts some people take to hold on to what's more comfortable to believe. And just as religion may often teach people to turn off their critical thinking skills they can be taught to use them again. And human brains evolved to recognize unhealthy or unnatural facial imagery for millennium. So unmasking the more successful techniques may be enough to break us out of the "anything can be fake" malaise.
- hairofadog 6y agoCan god create a fake so deep that they themselves can’t detect it?
- neutrallinked 6y agoDNA will set them apart
- neutrallinked 6y agoWe should also look towards simpler alternatives. ( For example some days back I also read about a profile made on a social media platform with a "generated" face ) 1) For image platforms to prevent "generated" profile with fake faces, may be request 2 profile pictures and not one. A task that is difficult to achieve at high resolution. 2) For deep fake videos .. May be the idea be to fight the "video" part and not the "deep fake" part. By that I mean "signed" content. ABC news should "sign" generated videos and so should other publishing houses. So that any other source claiming forged voice or certain kind of "faked" content are not able to do so.
- dependenttypes 6y agoThere is a simpler alternative imo: consider any news that do not list all of their sources as fake news. Treat them as if they were scientific papers.
- varbhat 6y agoWhat if it gives false positive results? What if it tags non-deepfake news as deepfake-news ?
- bsaul 6y agoMaybe advanced DRMs will be a way forward ? Have camera fingerprint / sign the video, and then every editing videos fingerprint & sign the changes performed, and send everything to a ledger ? Only way to make sure a video is actually coming from the real world...
- ed25519FUUU 6y agoWe don’t need this for photographs. Why do we need it for video?
- 29athrowaway 6y agoDeepfakes are generated by a generative adversarial network. There are two networks: a generator, and a discriminator. The generator generates a result, the discriminator evaluates that result. This AI that detect fakes faces could be used to train the discriminator so that the GAN generates even better results.
- adfhnionio 6y agoI find all this concern a little bit overblown. Yes, it is a problem that we can make extremely convincing fakes, but we've had fakes that can fool non-experts for a very long time. The Soviet Union doctored a great many photographs in a way invisible to me (examples: https://en.wikipedia.org/wiki/Censorship_of_images_in_the_Soviet_Union https://en.wikipedia.org/wiki/Censorship_of_images_in_the_So...). Why are we more concerned about this fakery than about airbrushing? The solution is the same as it's always been: stick to trustworthy sources and insist that all evidence is traced and corroborated. It remains easy to learn the truth as long as you make a good faith effort to do so. In the worst case we can just stop trusting photographs altogether. We got by just fine before the camera was invented; we can do fine after it becomes obsolete.
- mooneater 6y agoBecause good fake video is now possible for the first time, good video was nearly impossible to fake. Because people believe certain leaders who are willing to use false evidence. Because the populace is not prepared to be that discerning.
- Erlich_Bachman 6y agoHey, we all know that this will not be the end of deep fakes. It will just another channel of information to think about. Now we have to care about whether this algorithm is correct, or maybe it is also manipulated by the other political party to claim that the others' picture is fake... But it had to be created at some point, it had to exist. This is just an inevitable next step of the progress.
- deleted 6y ago[deleted]
- trott 6y agoLooking at the diagram, this appears to be just an LSTM slapped on top of a CNN. If so, I'm failing to see any novelty in this approach. RNNs on top of CNNs have been used before, including for deepfake detection. See for example: https://arxiv.org/abs/1905.00582 https://arxiv.org/abs/1905.00582 The recent DeepFake Detection Challenge threw a lot of manpower at the problem earlier this year, BTW.
- codecamper 6y agoThis is getting stupid. People.. aging, cancer, Alzheimer's still exist. Maybe time to reprioritize & organize efforts?
- Nasrudith 6y agoThose are completely different deep specalties and best funded in parallel anyway given diminishing returns. Complaining about something that irrelevant to capabilities is like saying brain surgeons suck at inventing green energy.
- ThomPete 6y agothis is the virus killer war all over again
- Ijumfs 6y agoThe deepfakes which support the officially blessed narrative will not receive scrutiny, while authentic videos will be "proved" to be deep fakes by some black box machination. But we've known not to trust anything we didn't personally see ourselves for many decades.
- olivermarks 6y agoThe challenge with algos like this is that they could be used to claim events that actually did happen didn't. So as an example credible/convincing footage of Jeffrey Epstein by a pool in Paraguay last week could be 'identified as a deep fake' and discredited, despite other supporting facts and information that lent credence to veracity.
- wu-tsy 6y agoCan be used as an algorithm for a new discriminator in the GAN.
- mbrumlow 6y agoThis will just be like anti virus. Peole will start to train their algos to fool the current algo that is is being used. We will be a never ending fight.
- 3wolf 6y agoThat's GAN training in a nutshell.
- peterthehacker 6y agoAre deepfake face swaps a real problem yet? I can’t recall any major controversies in recent history that were caused by a deepfake face swap. > This technique can be used to create compromising videos of virtually anyone, including celebrities, politicians, and corporate public figures. I’ve read a lot of concerned comments like this but I haven’t read about any real world examples of a controversy caused by a deepfake.
- nine_k 6y agoNow we need to stop believing our eyes twice: first when we see a natural-looking picture but know it is a deepfake, and second when we see a natural-looking picture we believe is not a deepfake, but the computer tells us it is.
- m0zg 6y agoIf this is differentiable (which it seems like it should be), it could be used adversarially to create better deepfakes, BTW.
- danieldrehmer 6y agoas Zizek would say, "And then you use this as the basis for a GAN that optimises for faces that do fit, making better deepfakes and so on and so on"
- vffhfhf 6y agoPeople believe what they want to believe. Just a basic search and 5 minutes in wikipedia about a topic will make you better than most people who sprout nonsence. But you got to have a right mindset. Human brain hates change. You have to force the bitch to take in new information and process it and prepare it for further info on the topic.