10 ms·
A new way to detect ‘deepfake’ picture editing
- quotemstr 5y agoIsn't literally every deepfake detection technique just an oracle that you can use to train a better deepfake creator?
- drclau 5y agoAt some point in the future, AI will make better humans than naturally occurring humans.
- XorNot 5y agoThe built in "touch up my appearance" feature in my phone, various video chat software etc. means that future is already here [1]. [1] https://mothership.sg/2021/03/japanese-biker-actually-man/ https://mothership.sg/2021/03/japanese-biker-actually-man/
- Cthulhu_ 5y agoTangentially reminds me of the film Surrogates, where people only went out in their surrogate, a remote controlled android - in some cases, a younger, better looking version of people. In other cases something completely different of course.
- deepsun 5y agoFriedrich Nietzsche called that "Superhuman", or "Übermensch".
- jimmySixDOF 5y agoI know. It's like we are going to need reverse Zero Days after a while. As soon as they publish it becomes the old new way to detect Deepfakes.
- water8 5y ago-1 days
- jtsiskin 5y agoEspecially if it’s differentiable
- mattchamb 5y agoI would interested in a way to automatically edit real photos of people in such a way that it looks the same but now tests positive for being a deepfake.
- dmurray 5y agoYes, but if the "fake detection" technique is computationally much faster than the generation technique, having access to a better oracle is not so helpful to the deepfake creators, who are bottlenecked by the other phase.
- lmilcin 5y ago> An image owner can modify their image in subtle ways which are not themselves very visible, but will sabotage any attempt to inpaint it by adding visible information determined in advance by the markpainter. Hmm... so if I ask it to add an apple to the scene it just looks for an apple and says "yep, found an apple". That is not an algorithm to detect fakes, it is an algorithm to detect predetermined edits. Steganography/watermarking has been not only known but used for practical purposes for a long time.
- axiosgunnar 5y agono i think that sentence means you can still use photoshop manually to laboriously edit stuff… but the super convenient AI editing will fail, meaning 99,9% of the population won‘t be able to create deepfakes
- lmilcin 5y agoThat's exactly what I meant. Watermarking can be used to find portions of the photo that have been tampered with. It is actually pretty simple. And using AI for this is pretty stupid in my opinion. You just disperse a little bit of additional signal in the photo, maybe divide it into lots of blocks (they don't need to be rectangles and they can overlap). Think of it as every small piece of the photo having its own signature embedded. If you don't know how the signature was embedded or what key was used for it, you are going to disturb the signature and your editing attempt will be foiled. Not only that, but you will be able to tell which parts of the picture were touched and how so you can tell whether the picture was just cropped, brightness changed, or something else happened to it. As you see, no need for AI...
- _Algernon_ 5y agoThe remaining 0.1 % happen to be the ones that matter. It helps no-one to eliminate all but state actors from creating deep fakes.
- lmilcin 5y agoYep, that's pretty sad conclusion to this. Actually, it is probably even worse than having deepfakes proliferated. In this case people would expect the photos could have been manipulated. On the other hand if only handful of players can do this it can be used with much more impact.
- mattowen_uk 5y ago> ...a photo agency that makes stock photos available on its website with copyright watermarks can markpaint them in such a way that anyone using common editing software to remove a watermark will fail; the copyright mark will be markpainted right back. So watermarks can be made a lot more robust. I fail to see how this can actually combat against someone with the patience to manually paste out copyright marks using something like a clone brush, or even a normal brush ? If someone wants to steal your work, they will. There's no sure-fire way to stop them.
- VSerge 5y agoIt won't defeat manual edit, nor do the authors claim it. They do however make a solid point in saying it will defeat cheap automated manipulation / deepfakes at scale that would otherwise be easy with inpainting features. Anything that can help us differentiate authentic images from manipulated ones, especially when it comes to news, is imo a welcome addition in the arsenal of societies attached to the concept of verifiable facts.
- terryf 5y agoAuthentic images do not exist. They always include a viewpoint. Example here: https://2.bp.blogspot.com/-rHeyzklapsE/W5-yUBVnbxI/AAAAAAAAAV8/aSkhUIpu07oSWd-zK2KNpWlit_tBD34zACLcBGAs/s1600/17684960-37071110-37-0-1512736896-1512736898-650-1-1512736898-650-67540fb40a-1512806202.jpg https://2.bp.blogspot.com/-rHeyzklapsE/W5-yUBVnbxI/AAAAAAAAA...
- rocqua 5y agoI'd say cropping an image leaves the image 'authentic'. Even if it changes the meaning. It's when you start painting out objects that you are making a 'non-authentic' picture.
- d--b 5y agoAdversial techniques are specific to the model used, no? You'd need to know all the models that one might use to tamper with a picture and modify the image in such a way that it screws them all. And the first thing that will happen is that people are going to train their models with these new 'tamper-protected' images...
- thih9 5y agoI'm having trouble following. Can someone explain this in easier terms? What scenario does this address and how does this work exactly?
- rplnt 5y agoThe OP's title is incorrect. This doesn't detect deepfakes, it serves for people to watermark their images in a way that are hard to remove by conventional ML approaches.
- wokwokwok 5y agoscenario: you want to resell 50000 istock phots, but they are watermarked, making the free high resolution versions valueless. solution: you use a denoising filter to reconstruct the watermark pixels to plausible original values. Profit! this: instead of just simple obvious watermarks, you can instead encode visually indistinct fake-noise that deliberately confuses denoising neural networks. They claim “We find that we can target multiple different models simultaneously with our technique.”, ie. it is reasonably generic. how? Eh, that’s complicated, look up “adversarial neural networks”, there’s a fairly high level overview here: https://towardsdatascience.com/how-to-systematically-fool-an-image-recognition-neural-network-7b2ac157375d https://towardsdatascience.com/how-to-systematically-fool-an...
- deleted 5y ago[deleted]
- dmos62 5y agoThe post is pretty hard to understand. Here's the paper and an excerpt from the abstract: https://arxiv.org/abs/2106.00660 https://arxiv.org/abs/2106.00660 > First, we show how an image owner with access to an inpainting model can augment their image in such a way that any attempt to edit it using that model will add arbitrary visible information. We find that we can target multiple different models simultaneously with our technique. This can be designed to reconstitute a watermark if the editor had been trying to remove it. Second, we show that our markpainting technique is transferable to models that have different architectures or were trained on different datasets, so watermarks created using it are difficult for adversaries to remove. Markpainting is novel and can be used as a manipulation alarm that becomes visible in the event of inpainting.
- mzs 5y agoThanks, this is the code with examples in the readme: https://github.com/iliaishacked/markpainting https://github.com/iliaishacked/markpainting
- Etheryte 5y agoIt's hard to see this as much anything but snake oil. While the technique allegedly allows you to target multiple models, it still requires you to know what adversarial model you're targeting before the fact. If the adversary sees there's visible artifacts they'll just change a few parameters or use a different model and be good to go none the less. In short, while this might be academically interesting, I can't help but feel that this is a futile field to work in. A good way to see why is to look at it this way: a skilled human can manually manipulate an image so that it isn't possible to tell that the image has been manipulated. It is then a matter of teaching the model to do the same, which might take time or be computationally prohibitive for now, but eventually it will be done.
- danuker 5y ago> I can't help but feel that this is a futile field to work in. Considering that training a model costs thousands of dollars, and that people reuse models instead of training from scratch, I somewhat disagree with this in general, for now. But this specific technique implies the attacker sees the result, which means they can try different models, until one does not produce artifacts.
- hwers 5y agoModels come in a broad spectrum and while it's true that gpt-3 costs a ton, many do not. Any mobile ready snapchat esque real time deep fake filter will typically be super fast and cheap to train (since the model has to be lightweight to run on a phone). There's also transfer learning from an expensive model which could easily be modified to remove whatever 'architecture flagging' is built into the structure.
- sigmoid10 5y agoA few weeks ago, there was a post here about that guy who deepfaked female profile pictures. He showed that the visible artifacts go away by merely scaling up the number of parameters in the model. Give it a few years and these fake images will be impossible to detect unless we agree on some way of cryptographically signing images from real cameras.
- mdre 5y agoOK cool, but the commonly used Photoshop's content aware fill, which they supposedly target, isn't even using any ML. They don't know what they're doing or what? https://en.wikipedia.org/wiki/PatchMatch https://en.wikipedia.org/wiki/PatchMatch
- shtps 5y agoWhile all this stuff around deep fakes and its detection is interesting I'm still left wondering why we don't use cryptography to prove if an image is authentic or not.
- vletal 5y agoBecause most people can not even use cryptography to sign more important stuff such as email.
- throw2500 5y agoBecause the world is impure and the link between the real world and the mathematical domain will never be ironclad. For instance, say that Joe makes a deepfake and then signs it with his key. Sure, it's beyond doubt (assuming keys weren't leaked, etc) that Joe either took or created the picture, but that doesn't in itself tell you whether Joe made a deepfake or not. It's the same as in supposed blockchain logistics operations. If Fred the farmer says he harvested x bags worth of grain but some were stolen before he could ship them, there's no way to mathematically verify whether the theft actually took place or the harvest just came up short. In both cases you're going to need some kind of monitoring, and that's the purpose deepfake detection algorithms would serve.
- rocqua 5y agoWill signing come from the camera? At that point you run into DRM. Besides, what will the camera sign? The produced JPG? The raw file? What about rescaling? Do we want ZKPs that a JPG was achieved by nothing more than re-scaling and tone-mapping another JPG or RAW file? Those ZKPs are going to be massively big and slow to verify.
- shtps 5y agoMy main concern is the faking of official statements by public figures where having those signed addresses the issue of deep fakes. Of course there has to be infrastructure in place for this and people must know that you can verify them. If we get to a point in society where unsigned images, or images signed from a questionable source are looked at skeptically, then that alone is a step in the right direction. I don't know the final solution and I don't claim to know it, but cryptography seems like a step in the right direction.
- Rucadi 5y agoThe better you can detect deepfakes, the better deepfakes will be using that method as discriminator.
- hwers 5y agoThis is all interesting but a huge part of me suspects that this is an arms war that will be impossible to win in the long run.
- literallyaduck 5y agoWouldn't you just build a training set where ML made good edits vs visible ones and train it to avoid the bad version?
- mojomark 5y agoCouldn't reputable photo/video authors just publish content as an NFT? If the NFT is not traced back to a reputable source, then the content is deemed unreliable.
- Extigy 5y agoWhy not just sign the content with a digital signature?
- barbazoo 5y agoYou mean devices could sign the images they create and then wherever someone publishes the picture you'd be able to validate the signature using that device's (published somewhere) public key? That would be cool indeed. Except it wouldn't change anything. Facebook doesn't care about fakes and people wouldn't care whether the picture someone shared hasn't been tampered. Also whoever manipulated the picture could just sign it again probably.
- mojomark 5y agoSome people do care about authenticity though, hence the genesis of the OP's work to try and authenticate and it's why people still read news papers vice trusting their clickbait Google "news" feed. I'm just saying that I'd trust some encryption cert (whatever it is) traced back to a known more reputable source (say Financial Times and the like) than an algorithm that is just guessing that a photo/video is an un-altered genuine original vice a fake. The challenge is establishing an author's credibility in the first place. If I trace a cert back to joe schmo off the street or some empty front business, then I'd probably not trust the content.
- this_was_posted 5y agoBecause then you would lose in the buzzword bingo game.
- macksd 5y agoThe whole point of GANs is to train a model until it can defeat detection by another model. Virtually any technique for detection that can be reproduced could just be incorporated into the training loop and rendered almost useless.
- ford_o 5y agoHowever, the detection process could be so slow, that it wouldn't be practical.
- d0100 5y agoHear me out: blockchain cameras? \s But maybe we'll just have to limit cameras outputting NFT videos, or signed frames as the only source of trusted media
- kkielhofner 5y agoWorking on it! Main comment here: https://news.ycombinator.com/item?id=28401256#28407184 https://news.ycombinator.com/item?id=28401256#28407184
- GenerocUsername 5y agoThe github is better than the blog: https://github.com/iliaishacked/markpainting https://github.com/iliaishacked/markpainting
- refactor_master 5y agoDoesn’t take many rounds to jpeg the living pixel out of anything, and still spread it as fake news, with a baity text.
- enduku 5y agoSeems like compression is not such a bad deterrent for adversarial deepfakes.