8 ms·
Many AI researchers think fakes will become undetectable
- neonate 3y agohttp://web.archive.org/web/20240120190542/https://www.economist.com/science-and-technology/2024/01/17/many-ai-researchers-think-fakes-will-become-undetectable http://web.archive.org/web/20240120190542/https://www.econom...
- ang_cire 3y agoI would imagine that models could be trained on specific image generators to detect them. This is honestly the obvious solution; AI is actually good at finding very subtle patterns.
- hiddencost 3y agoAdversarial setting. People will train models on adversarial detectors to hide from them. The comments in this article have deeply studied the method you're suggesting.
- ang_cire 3y agoWhich is pointless in targeted models like I'm talking about. If I train a model that is *only* trying to detect e.g. stable-diffusion-2-1, training an adversarial model is pointless, because then you're just using a different model anyways. You would be spending a lot of training time when you could just use another already-extant model. State actors, for example, don't need to use public models for their generators anyways, so it's not like non-state actors would have true-positive images to train models on to counter said actors' models.
- agumonkey 3y agowouldn't this feed a race to generators being even closer to indistinguishable ?
- ang_cire 3y agoPotentially, but I don't think in practice, because most image generation is not trying to be photorealistic. Most models are trying to stylize their outputs, either by default (e.g. a Miyazaki-anime-like model), or in response to the prompt. Using AI image generation for misinformation is by far the vast minority of its use. And if the output is stylized, it's going to be much more obvious if it has come from an AI model, because a human is going to have a much harder time reproducing a specific ML model's style on-demand (e.g. if their art teacher asks them to sketch a face in the same style, to prove their homework wasn't faked).
- jvanderbot 3y agoThe problem is this isn't like cryptography where you can prove something. You can train a detector on a generator, or at least what you think the generator is, but you can never know for sure. And any "likely fake" assessment can just be refuted by a sock puppet that vouches for the content.
- willsmith72 3y agoThis is the main discussion point of the article > Just as machines can be trained to reliably identify cats, or cancerous tumours on medical scans, they can also be trained to differentiate between real images and ai-generated ones. But the conclusion the researchers made was detection at some point wont be reliable and fakes will win
- ang_cire 3y agoThe tools the article discusses are *general* AI-detection, meant to be fed any image and determine whether it was AI-generated. I'm talking about a detection model that only detects images produced by one generator/source model, so rather than a one-to-many process of having one tool check an image for many potential source generators' artifacts, you'd have many individually-trained detection models, each being run against a single image/text block.
- agumonkey 3y agoSome people raise question on what it implies in society at large. Anything digital is now untrustworthy ?
- figassis 3y agoI think it’s always been, we just knew that the effort needed to fake many things requires some degree of skill available to a few and was often expensive. But perfect fakes have always existed. This is why artists usually sign their work? What we will need today is to start signing things digitally in the same way, which I think will require very good, decentralized pki, unlike what we have with email. Essentially, different pki should work like dns, and find the pki for someone, and cache it., so that your signature works in china and the moon. Updating it should work the same way, the old one should always work and new ones should have the old pub keys appended to them so any public key can be used to verify all past signatures.
- deleted 3y ago[deleted]
- hammyhavoc 3y agoAnything is inherently untrustworthy, hence "zero trust" as a concept. You can't even trust physical things. Counterfeiting has been a thing since at least 400 BC with fake Greek coins.
- js8 3y agoWe already have undetectable fakes. Anybody can buy a blue check mark on Twitter and pretend to be somebody else. Anybody can also domain squat. The way we detect them is if this becomes a problem, the authentic entity will speak against it and the society will rectify. Just as these fakes are a non-problem in practice, AI fakes will be a non-problem too.
- gumballindie 3y ago> the authentic entity will speak against it and the society will rectify Society is already speaking against stolen content and fakes produced using it. How are we rectifying this?
- big_whack 3y agoMuch cruder fakes than you are describing are a huge problem in practice, costing billions per year in fraud to Americans alone. Just because you can detect them, doesn't mean your grandma can. The societal implications of undetectable fakes are off the charts.
- thimkerbell 3y agoWhat does it mean that next to nobody is saying this? Are the ones who know, voting with their feet? And where is that?
- hammyhavoc 3y agoPeople are saying this. You're on HN though, the demographics here can tend to skew money-grubbing techbros with who is the loudest and most defensive of anything they've got money riding on. Remember the crypto and NFT hype on HN? The bottom-line is only what matters to a good chunk of people here.
- wmeredith 3y agoCalling the current fakes a "non-problem" is one of the hottest takes I've heard in some time.
- 3y ago
- CuriouslyC 3y agoI wouldn't be surprised if the real way we reached AGI was that the cat and mouse game between people trying to detect AI and people trying to sneak AI output past detection. That is essentially a hyper-analytic version of the Turing test - if humanity as a whole cannot discern human output from AI output given all the tools of science at its disposal, then we have failed to prove that it isn't intelligent, which is really more of a scientific question formulation anyhow. If you think about it, it's sort of like a real life GAN algorithm.
- t_mann 3y agoAssuming it'll be true, I'm not sure it has to be just a bad thing. We got used to treating videos/photos as the kind of objective evidence that they never were. It's not hard for an image to be both factually correct/unaltered, and yet giving a distorted view of reality. Cropping out parts, leaving out some bigger, non-visible contexts, such distortions can be really easy to miss, too. How faithful an image is depends a great deal on how it was put together and presented. So what we should really be asking ourselves when we see images is how much we trust the sources. A proliferation of AI fakes could make the need for such reflection more obvious.
- treprinum 3y agoWW3 or some other high-intensity conflict could be potentially launched by highly coordinated deepfakes. Imagine creating some videos offending an aggressive religion with many combatants or generating event videos that didn't happen and managing narrative as the uproar progresses etc.
- jjj123 3y agoNone of that strikes me as meaningfully different than regular old propaganda.
- pixl97 3y agoQuantity and speed are qualities. Getting hit with a snowball could hurt you but is most apt to annoy you. Getting hit with an avalanche is apt to kill you. But hey, they are both snow, right?
- redcobra762 3y agoHowever, it's unclear what plays the role of mass in this analogy.
- Loughla 3y agoWhen it's one video of an angry mob killing bystanders, it's easy to dismiss. When it's hundreds, from different viewpoints, it's much harder.
- seydor 3y agoAs long as the detectors are other AIs, that's pretty self-evident
- scrps 3y agoInsane idea: generate a ton of fakes of yourself, some subtle, some obvious... Which mirror in the forest or mirrors is actually you?
- hammyhavoc 3y agoThe one caught on CCTV hundreds of times per day for years on end with your gait analyzed.
- scrps 3y agoThis has what to do with flooding so many fakes of a person that all photos are assumed fake by the public?
- hammyhavoc 3y agoHaving plenty of existing training data to figure out which the real you is?
- scrps 3y agoI think our wires are getting crossed. Who is the threat in your scenario?
- hammyhavoc 3y agoAnyone and anybody. If the data exists, the data can be (ab)used. Microsoft getting hacked by hackers to find out how much they knew about the hackers is the case in point. This is why "surveillance by design" and backdooring things is a terrible idea, because we need less data collection, and we need more security, not less.
- scrps 3y agoI agree with the bulk of what you are saying but at the same time there are different levels of threats, gait analysis and access to massive troves of data is state-level or central corps. Where most ordinary people run into trouble is decidedly very well down that particular foodchain.
- spydum 3y agoWon't part of the answer be digital signatures matched to the media, signed by the users real identity? I have wondered if this would happen to CCTV / Ring / security cams for a while, but yet to have seen it.
- hammyhavoc 3y agoSigning something isn't proof that it's legitimate. Signing something is just proof that you control the means to be able to sign something cryptographically, hence the problem of compromised keys. DMA attacks anyone?
- solardev 3y agoIsn't this the whole point? When did the Turing test become a negative..?
- persnickety 3y agoEver since humans started seeing competition to their abilities.
- password54321 3y agoTuring test was a thought experiment. I don't know why people keep holding it up as some sort of holy grail for testing intelligence.
- solardev 3y agoIt's not some holy grail, it's just one way to think about AI. Regardless of the test used, it's just sad to me that popular views of AI now tend to be more Skynet and HAL than Transcendence or Her. Guess a few decades of big tech enshittification will do that to the psyche, lol.
- ndjshe3838 3y agoWould it be possible for cameras to have a hardware private key? And then sign the images in hardware so that you can prove an image came from a specific camera and hasn’t been altered?
- pixl97 3y agoSounds risky if you're taking pictures of stuff the government doesn't want you to.
- joshuanapoli 3y agoA photo signed as an authentic iPhone image wouldn’t completely pinpoint the photographer, but would be good evidence that the image is authentic.
- pixl97 3y agoOver any period of time, no probably not. If the algorithm is simple as "This was signed by an iphone" then you have a fleet of a billion+ iphones that all must remain secure and unbroken for the entire time this fleet of phones exist trends to unlikely. Given time and motivation all systems are broken. A much harder to break system is one uses a secret only exists on each individual phone + the generic algorithm that iphones use, but this comes back to individuals being traced.
- JanisErdmanis 3y ago> uses a secret only exists on each individual phone There is actually a way to prevent individuals from being traced while having a strong membership proof. This can be achieved by issuing signatures on a relative generator created in an exponentiation mix together with a set of output pseudonyms. One can picture it as strands braided together where it is hard to trace the link between the input and output strands, while it is very easy to tell that such a link exists. More on that can be found on: https://github.com/PeaceFounder/ShuffleProofs.jl https://github.com/PeaceFounder/ShuffleProofs.jl
- joshuanapoli 3y agoDigital signatures are one solution: traceability back to an authentic source would be good evidence that the item isn’t fake. This might be provided so some degree by the hardware (the sensor in a camera, for example) or the authoring software.
- 05 3y agoYeah I’m sure state actors that are capable of creating exploits using undocumented registers on Apple silicon [1] won’t be able to break camera sensor DRM.. [1] https://securelist.com/operation-triangulation-the-last-hardware-mystery/111669/ https://securelist.com/operation-triangulation-the-last-hard...
- joshuanapoli 3y agoIt’s a level of difficulty that will rule out most producers of fake content. The point is that tracking the provenance of digital items is certainly going to be more important in the future. A signature by Apple silicon is one piece of evidence. It could also be signed by the photographer’s identity. The account who uploads it gives another piece of evidence. Unfortunately, we should probably not trust anonymous content in the future.
- ThomPete 3y agonothing different than virus / anti-virus. It will be fine
- hammyhavoc 3y agohttps://www.independent.co.uk/news/deepfake-nude-westfield-high-school-nj-b2440793.html https://www.independent.co.uk/news/deepfake-nude-westfield-h... Is anti-virus really going to fix this?
- ThomPete 3y agoYes.
- robviren 3y agoI like to ponder if it is possible to create something that can actually be trusted. I think the complexity and obscurity of a medium would actually suit it to be a trustable recording device. For instance, an audio device recording and storing something outside of human hearing range. No model would be trained to generate those frequencies in that format so it would be useful as a check for if it is real. Something could be said about insanely high resolution imagery. You just need something outside the normal models. Obvious potential for people to adapt, but a short term option maybe.
- herbst 3y agoI didn't know there was any doubt about that
- wolverine876 3y agoFundamental law regarding AI should make it illegal, IMHO, to impersonate a human being. Every AI output should clearly state it's from a computer (no using the anthropomorphic word 'intelligence'): 'This text is generated by a computer.' Give a valid reason to do otherwise. I can't think of one, unless you want to mislead people. Corporations can still have their chatbot customer service, and add that tagline. The only reason not to is so their customers think they are talking to a human.
- Calavar 3y agoThe question is how you would enforce a regulation like that when any guy with a few grand to burn can set up their own rig and start churning out images/video/text using their own finetune of an open model.
- wolverine876 3y agoThe point would be to enforce it on corporations, political parties, etc. - to prevent the powerful from using it destructively. Also, it's not clear that an individual can do it nearly as well as a well-resourced company with scientists and lots of processing power.
- squigz 3y agoAh, yes... corporations and political parties: 2 entities that famously never attempt to circumvent laws whenever possible.
- wolverine876 3y agoWhat do you conclude from that? No laws? They would be (they are) completely unrestrained.
- RandomLensman 3y agoDo you research who wrote a politician's speech, for example? What is the outcome you want to achieve with such regulation?
- thom 3y agoOne phenomenon I’ve seen several times during recent conflicts is people swearing blind that an image must be fake, and even insisting some vague shadow is a sixth finger proving their point, even when the photo later proves to have pretty reliable provenance. It’s a reasonable immune response and maybe we’ll all end up there but it seems just as worrying and tragic as fake images themselves.
- 15457345234 3y agoIt's not worrying and tragic, it's entirely logical. The news media have utterly debased themselves recently by publishing ridiculous levels of gaslighting and fake news. When you see _anything_ on the internet these days your first instinct should be to ask 'did any of this actually happen?'
- rany_ 3y agoBefore AI fakes, people used to claim that images and videos are photoshopped and CGI. Nothing has changed, only the bar for entry is lower.
- odyssey7 3y agoThey’re already undetectable for most —- media consumers are currently making their own judgement calls to distinguish deepfakes, based on their intuition and what strangers and friends on the internet tell them. They may be detectable in theory, but every security issue is a zero-day for as long as mitigations are lacking due to accessibility or other reasons.
- cudgy 3y agoPersonally I prefer the idea that all people be convinced that video can be fake without detection, instead of only a few bad actors knowing this.
- lionkor 3y agoUmbrella salesman predicts it will rain? This seems like a copyright problem to me, a legal one, not a technological one
- amai 3y agoIf the fines for fakes would skyrocket and include 100% of a company’s revenue and many years in jail we probably would see fewer of them. However advertising industry would basically cease to exist.