10 ms·
Want to spot a deepfake? Look for the stars in their eyes
- raisedbyninjas 2y agoThe sample images don't show a large difference between the real and generated photo. The light sources in the real photo must have been pretty close to the subject.
- batch12 2y agoHow does the deep fake have the same eye shape, same cropping and same skin blemishes as the real image? Did they inpaint eyes and call it deepfake for training?
- brabel 2y agoWell, nice find, but now all the fakes have to do is add a new layer of AI that knows how to fix the eyes.
- chbint 2y agoIndeed, but useful nonetheless. Solving it may be a challenge for a while, and deep fakes generated before a solution becomes broadly available will remain detectable with this technique.
- spywaregorilla 2y agoThe commercial use cases for generative art is not to make images that experts cannot discern as fake. It would be very expensive to annotate training images to have physically correct reflection images and the value would be negligible. Realistically, if you want to produce something that is impossible to prove fake, you would have a vastly easier time doing such edits manually. We are very, very, very far from being able to push button churn out undiscernable fakes. Even making generically good outputs for art is a careful process with lots of iteration.
- kvemkon 2y agoOr just use conventional algorithm, since the fix is about formal physics. Although it will not be a true 100% fix, it could be good enough to make this test rather useless, because even now: > "There are false positives and false negatives..."
- deleted 2y ago[deleted]
- bqmjjx0kac 2y agoI wouldn't be shocked if phone cameras accidentally produced weird effects like this. Case in point: https://www.theverge.com/2023/12/2/23985299/iphone-bridal-photo-three-poses-explanation-panorama-photoshop-generative-ai https://www.theverge.com/2023/12/2/23985299/iphone-bridal-ph...
- Mashimo 2y agoOr https://www.theverge.com/2023/3/13/23637401/samsung-fake-moon-photos-ai-galaxy-s21-s23-ultra https://www.theverge.com/2023/3/13/23637401/samsung-fake-moo... And there is also software that fixes you eyes for selfies and video calls.
- leidenfrost 2y agoAFAIK deepfakes can't mimic strong gesticulations very well, nor mimic correctly a head facing sideways. Or was that corrected?
- BugsJustFindMe 2y ago> deepfakes can't There is a big difference between can't and don't. Every next generation will do more than what the previous generation did.
- SXX 2y agoWhile I'm not expert on state of the art you should keep in mind there is huge difference with deepfakes created 100% from scratch and those created via face swap and style transfer. Basically it's easier to create believable gesticulations when there is footage of actual person as raw material.
- spywaregorilla 2y agoYou think we can't generate a picture of a head facing sideways? Obviously incorrect.
- fourthark 2y agoThe argument being that there is not very much video footage of people turning their heads, therefore not enough data to train deep fake videos / filters.
- spywaregorilla 2y agoVideos are still very much in the baby phase. There are way, way easier tells when a video has been faked. We're talking about images. Turned head images are very much in scope.
- nottorp 2y agoI can't read TFA because it's probably HNed. However an artist friend of mine said generated images are easy to spot because every pixel is "perfect". Not only eyes. Explained pretty well why I thought even non realistic ones felt ... uncanny.
- KineticLensman 2y ago> However an artist friend of mine said generated images are easy to spot because every pixel is "perfect". It depends on the art. There was a discussion here a while ago that mentioned the use of Gen AI to create images of knitted / crocheted dolls. The images looked okay at a quick glance but were clearly fake because some of the colour changes weren't aligned with the position of the stitches. E.g. strands of supposed hair overlaid on the underlying stitched texture. I'm sure there are lots of similar examples in other forms of art where the appearance is physically impossible.
- krisoft 2y ago> However an artist friend of mine said generated images are easy to spot because every pixel is "perfect". What does perfect mean? Does the pixels drawing 15 fingers count as "perfect"? I think this heuristic is liable to fail in both directions. You will find images made by humans where "every pixel is perfect" (whatever that means) and you will also find AI which does mimic whatever imperfection you are looking for.
- nottorp 2y ago> What does perfect mean? Nothing out of focus or less detailed than other parts of the image...
- cthalupa 2y agoEven the most cursory browsing of civitai or genai image generation subreddits shows this to not be true. Focus, bokeh, etc. are all things that can be generated by these models.
- Suppafly 2y ago
- jmmcd 2y agoThey love saying things like "generative AI doesn't know physics". But the constraint that both eyes should have consistent reflection patterns is just another statistical regularity that appears in real photographs. Better training, larger models, and larger datasets, will lead to models that capture this statistical regularity. So this "one weird trick" will disappear without any special measures.
- actionfromafar 2y agoWouldn't also the adverserial model training have to the take "physics correctness" into account? As long as the image detects as "<insert celebrity> in blue dress", why would it care about correct details in eyes if nothing in the "checker" cares about that?
- Filligree 2y agoCurrent image generators don’t use an adversarial model. Though the ones that do would have eventually encoded that as well; the details to look for aren’t hard-coded.
- actionfromafar 2y agoInteresting. Apparently, I have much to learn.
- themoonisachees 2y agoGP told you how they don't work, but not how they do: Current image generators work by training models to remove artificial noise added to the training set. Take an image, add some amount of noise, and feed it with it's description as inputs to your model. The closest the output is to the original image, the highest the reward function. Using some tricks (a big one is training simultaneously on large and small amounts of noise), you ultimately get a model that can remove 99% noise based only on the description you feed it, and that means you can just swap out the description for what you want the model to generate and feed it pure noise, and it'll do a good job.
- gyosko 2y agoIt seems that even discussion about AI is getting really polarized like everything else these days. Comments are always one of these two types: 1 -> AI is awesome and perfect, if it isn't, another AI will make it perfect 2 -> AI is just garbage and will always be garbage
- surfingdino 2y agoNobody has given me a good reason to use it or proof that what it does is more than recombining what it hoovers up, so... I'm in the second camp.
- wvenable 2y agoYou could just... try it. It's very impressive what it can do. It's not some catch-all solution to everything but it saves me hours of time every week. Some of the things it can do are really quite amazing; my real-life example: I took a picture of my son's grade 9 math homework worksheet and asked ChatGPT to tell me which questions he got wrong. It did that perfectly. But I use for the more mundane stuff like "From this long class definition, can you create a list of assignments for each property that look this: object1.propertyName = object2.propertyName" and poof.
- EGreg 2y ago1 also says “anything bad that AI does was already bad before AI and you just didnt care, scale is irrelevant”.
- ben_w 2y agoI have seen those comments; but I do wonder, to what extent that is because the comments' authors intended such positions vs. the subtlety and nuance are hard to write and easy to overlook when reading? (Ironically, humans are more boolean than LLMs, the word "nuance" itself seems a bit like ChatGPT's voice). I'm sure people place me closer to #1 than I actually feel, simply because I'm more often responding to people who seem to be too far in the #2 direction than vice versa.
- 2y ago
- Y_Y 2y agoIf you can see the difference than so can the computer. If the computer can see it we have a discriminator than we can use in a GAN-like fashion to train the network not to make that mistake again.
- HumblyTossed 2y agoIsn't it easier to simply look for all the 6 fingered hands?
- olivierduval 2y agoWarning: photoshopped portraits (and most of pro portraits ARE photoshopped, even slightly) may add "catch lights" in the eyes, to make the portrait more "alive" So that kind of "clues" only shows that the picture has been processed, not that the people on the picture doesn't exists or is a deepfake
- acomjean 2y agoWhen I shot events years ago, I always used a flash for fill, even outdoors. People like the glint in the eyes that it added. Before the photoshop times you could sus out lighting setups based on the reflections.
- radicality 2y agoAnd the non-professional pictures, like your everyday smartphone pictures everyone takes, pass through so many layers of computational photography that sometimes make it pretty far off from reality.
- SXX 2y agoI wonder how true this is for face swap. Since actual scammers likely wouldn't generate deepfakes completely from scratch or static image.
- symisc_devel 2y agoWell, they are relatively easy to spot with the current AI software used to generate them especially if you are dealing on a daily basis with presentation attacks aka deepfakes for facial recognition. FACEIO has already deployed a very powerful model to deter such attacks for the purpose of facial authentication: https://faceio.net/security-best-practice#faceSpoof https://faceio.net/security-best-practice#faceSpoof
- neom 2y agoRandom thought: GCHQ and IDF specifically seek out dyslexic employees to put on spotting "things out of place" be it a issue in a large amount of data, or something that seems wrong on a map, to a picture that contains something impossible in physics. Something about dyslexic processing provides an advantage here (not sure if I'd take this or reading at 1 word per hour), given GPTs are just NNs, I wonder if there is any "dyslexic specific" neurology you could build a NN around and apply it to problems neurodivergent minds are good at? Not sure what I'm really saying here as I only have armchair knowledge.
- singingwolfboy 2y agohttps://archive.ph/pDf1x https://archive.ph/pDf1x
- keybored 2y ago> In an era when the creation of artificial intelligence (AI) images is at the fingertips of the masses, the ability to detect fake pictures – particularly deepfakes of people – is becoming increasingly important. The masses having access to things wasn’t a cutoff point for me.
- Nullinker 2y agoI would actually argue that once the masses are aware about certain technology existing and being in widespread use it becomes much easier to convince someone that a particular instance that the data is not trustworthy, so the ability to detect it through technological means becomes less important. In the stage before widespread use people are much more easily tricked because they are unaware that others have certain capabilities which they never experienced first hand.
- nick238 2y agoYou're missing the flip side: falsely believing something is forged. Now that the technology is so accessible and widespread, someone could deny truth by saying whatever audio/visual evidence was deepfaked, and people will believe it.
- throw4847285 2y agoAlso be on the look out for high flyin' clouds and people dancin' on a string.
- RobotToaster 2y agoInteresting, some portrait photographers use cross polarised light to eliminate reflection from glasses, but it has the side effect of eliminating reflection from eyes.
- adwi 2y ago> The Gini coefficient is normally used to measure how the light in an image of a galaxy is distributed among its pixels. This measurement is made by ordering the pixels that make up the image of a galaxy in ascending order by flux and then comparing the result to what would be expected from a perfectly even flux distribution. Interesting, I’d only heard of the Gini coefficient as an econometric measure of income inequality. https://en.m.wikipedia.org/wiki/Gini_coefficient https://en.m.wikipedia.org/wiki/Gini_coefficient
- ploika 2y agoSome decision tree algorithms use it to decide what variable to split on when creating new branches.
- buildsjets 2y agoAlso found it interesting, but for it's technical merits, as I recently had to glue some code together to analyze/compare droplet size from still frames of a high speed video of a pressurized nozzle spraying a flammable fluid. (into a fire! neat! fire! FIRE!) This approach might have been useful to try. I ended up finding a way to use ImageJ, an open source tool published by the NIH that biologists use to automatically count bacterial colony-forming units growing on petri dishes, but it was very slow and hacky. It was not perfect, but it gave an objective way to quantify information from a large body of existing test data with zero budget. https://en.wikipedia.org/wiki/ImageJ https://en.wikipedia.org/wiki/ImageJ
- deleted 2y ago[deleted]
- GaggiX 2y agoDid they try using this method on something that is not StyleGAN?
- butlike 2y agoI don't understand the "galaxy" terminology in the sentence: "To measure the shapes of galaxies, we analyse whether they're centrally compact, whether they're symmetric, and how smooth they are" Can someone explain?
- meatmanek 2y agoGiven that this is from the Royal Astronomical Society, I think they're literally talking about galaxies. They're then using these same scoring functions to characterize the reflections on the subjects' eyes, and comparing the result for the two eyes -- real photos should have similar values, generated images have more variation between the two eyes.
- AlbertCory 2y agoI took a film lighting class a long, long time ago at a community college. Even then, you could look at a closeup and tell where the lights were by the reflections in the eyes.
- crazygringo 2y agoI don't know, the example photos of deepfakes here seem... pretty good. If that's the worst they could find, then this doesn't seem useful at all. Even in the real photos, you can see that the reflections are different in both position and shape, because the two eyeballs aren't perfectly aligned and reflections are going to be genuinely different. And then when you look at the actual "reflections" their software is supposedly detecting (highlighted in green and blue) and you compare with the actual photo, their software is doing a terrible job detecting reflections in the first place -- missing some, and spuriously adding others that don't exist. Maybe this is a valuable tool for spotting deepfakes, but this webpage is doing a terrible job at convincing me of that. (Not to mention that reflections like these are often added in Photoshop for professional photography, which might have similar subtle positioning errors, and training on those photos reproduces them. So then this wouldn't tell you at all that it's an AI photo -- it might just be a real photo that someone photoshopped reflections into.)
- sour-taste 2y agoI suspect that detecting ai generated content will becomes an arms race just like spam filtering and seo. Business will be built on using secret ml models detecting smaller and smaller irregularities in images and text. It'll be interesting to see who wins
- notorandit 2y agoOnce you highlight any inconsistency in AI-generated content, IMHO, it will take a nothingth of a second to "fix" that.
- grvbck 2y agoAm I missing something here, or are the authors incorrectly using the term "deepfake" where "AI-generated" would have been more appropriate? There's a lot of comments here discussing how generative AI will deal with this, which is really interesting. But if somebody's actual goal was to pass off a doctored/AI-generated image as authentic, it would be very easy to just correct the eye reflection (and other flaws) manually, no?
- threatripper 2y agoI really wonder where the limit is for AI. Reality has an incredible amount of detail that you can't just simulate or emulate entirely. However, our perception is limited, and we can't process all those details. AI only has to be good enough to fool our perception, and I'm confident that every human-understandable method for identifying fakes can be fooled by generative AI. It will probably be up to AI to identify AI-generated content. Even then, noise and limited resolution will mask the flaws. For many forms of content, there will simply be no way to determine what's real.
- Rury 2y agoThe necklace in the right photo is a more obvious giveaway, whereas you have to look closely at eyes to see if they match.
- ziofill 2y agoOk. But it does feel like we’re scraping the bottom of the barrel.
- borgchick 2y agospy vs spy, round n
- chefandy 2y agoAh! This is a great technique! Surely now that it's published it would be easily remediable in a compositing program like Nuke, but for more casual efforts, it's a solid test.
- ggm 2y agoOut of interest, how many CAPTCHA are or were part of training? Is there any factual basis to the belief that's what it descended to?
- plasticeagle 2y agoAny algorithm that claims the ability to detect AI automatically must always be possible to circumvent. All one has to do is incorporate the algorithm in your image generation process, and perturb or otherwise modify your output until the image passes the test.
- zeristor 2y agoEnhance. The film Blade Runner for a large but was about determining hunting down androids that were so close to being human. Not part of the test, but a nifty party of the film, was about using a photograph to see what clues were in a picture by looking deeply into reflections. As has been said, this omission can be added as a test in generating the AI images in time, but I just loved how this inadvertently reminded me of Blade Runner.
- constantcrying 2y agoThese articles are incredibly unhelpful. I have no doubt that in a short amount of time AI models will also have learned the statistical dependence between reflections in eyes. It is inevitable that this "trick" will become obsolete. This just gives a the false impression, both about AI and how to spot AI generation.
- deleted 2y ago[deleted]