9 ms·
Researchers tested AI watermarks and broke all of them
- natch 3y agoWho was it, Eric Schmidt, who said we need to get over it, there is no privacy? I feel like we have the same energy here for authenticating human origin of content.
- TestingTest5 3y agoWas only a matter of time anyways...
- deleted 3y ago[deleted]
- skilled 3y agohttps://archive.ph/1F0Ng https://archive.ph/1F0Ng
- brap 3y agoPeople have been trying to watermark digital media for decades, when there was (still is) a very strong financial incentive to get it working. It never worked. I don’t think it ever will work.
- ipnon 3y ago“Information wants to be free.”
- wly_cdgr 3y agoMore like, "people want to steal information"
- treyd 3y agoCopying information isn't the same as stealing. To steal means to take away.
- artninja1988 3y agoControl+C, Control+Vs in your path
- wyldfire 3y agoYou can still declare success if you lower the bar to "we can catch leaks/pirates and in particular we can know which internal folks should no longer be trusted. ... as long as they don't attempt to circumvent the fingerprint"
- ActorNightly 3y agoYou are confusing access restrictions with signing. You can easily sign digital media to show that it was made by you.
- ygjb 3y agoYou are confusing a digital signature for evidence of anything other than an attestation. If you create a digital record, then sign it, then that signature is only an attestation of may claim you make, not evidence of that claim. That is the problem with relying on technology to establish trust - the moment you attach an economic benefit to a technology you incentivize people to circumvent it, or to leverage it to commit fraud.
- rakkhi 3y agoIt’s like captcha, highly annoying to users and authors, but if you don’t want to pay it works against low spend bots
- ShamelessC 3y agoI'm pretty certain the article is saying it is _not_ like captcha in that it is so trivial to circumvent that it's completely useless, rather than just useless sometimes.
- obblekk 3y agoFor written text, the problem may be even harder. Identifying the human author of text is a field called "stylometry" but this result shows that some simple transformations reduce the success to random chance [1]. Similarly, I suspect watermarking LLM output is probably unworkable. The output of a smart model could be de-watermarked by fine tuning a dumb open source model on the initial output, and then regenerating the original output token by token, selecting alternate words whenever multiple completions have close probabilities and semantically equivalent. It would be a bit tedious to perfectly dial in, but I suspect it could be done. And then ultimately, short text selections can have a lot of meaning with very little entropy to uniquely tag (e.g., covfefe). [1] https://dl.acm.org/doi/abs/10.1145/2382448.2382450 https://dl.acm.org/doi/abs/10.1145/2382448.2382450 Curious if Scott Aaronson solved this challenge...
- kromem 3y agoAlso, most stylometry work isn't well fitted to active attempts to forge another author, and is more about distinguishing authorship in works with uncertain attribution.
- COAGULOPATH 3y agoThe idea of telling a human generated "the quick brown fox..." from a machine-generated one was always a fantasy. Text has no birthmark. Current LLMs have stylistic quirks imprinted on them by RLHF (ChatGPT's endless "it should be noted" and "it is important to remember that" verbiage is a good example), but they learned those from human writing.
- great_psy 3y agoIt seems it would be much easier to watermark non-ai images instead. Aka crypto signature. That will be much harder to evade, but also pretty hard to implement. I guess we will end up in the middle ground, where any non-signed image could be ai generate, but for most day to day use it’s ok. If you want something to be deemed legit (gov press release, newspaper photo, etc) then just sign it. Very similar to what we do for web traffic (https)
- jacobr1 3y agoWe need to focus on the other direction. How can we have chains of trust for content creation, such as for real video. Content can be faked, but not necessarily easily faked from the same sources that make use of cryptographic signing. The attacks can sign the own work, so you'd need ways to distinguish those cases, but device level keys, organizational keys, distribution keys all can provide provenance chains that can be used by downstream systems to _better_ detect fraud, though not eliminate it.
- sacrosancty 3y ago[dead]
- tudorw 3y ago"Magnetic anomalies are generally a small fraction of the magnetic field. The total field ranges from 25,000 to 65,000 nanoteslas (nT). To measure anomalies, magnetometers need a sensitivity of 10 nT or less." Would signing content with a cryptographically consistent encoding of this field be workable?
- floren 3y agoI was thinking the other day about embedding keys in cameras, etc. but came up with the problem that you could just wire up a computer that BEHAVES like a CCD sensor and send whatever the hell you feel like in to the signing hardware, so you feed in your fake image and it gets signed by the camera as though it were real. I assume smarter people than me have put much more time into the problem, so I'd be interested to see any good resources on the subject.
- jacobr1 3y agoI think you'd need device levels keys. You couldn't trust any particular image ... but you could perhaps know where it came from, which you gives you a better substrate upon which to infer trust.
- tshaddox 3y agoI think you're essentially describing the hardware DRM supply chain. For example, HDCP is a DRM scheme where Intel convinces (or legally requires) every manufacturer of HDMI output devices (e.g. set-top boxes, Blu-ray players) in the world to encrypt certain video streams. Then, Intel requires manufacturers of HDMI input devices (e.g. TVs) to purchase a license key that can decrypt those video streams. This license agreement also requires the manufacturer to design their device such that the device key cannot be easily discovered and the video content cannot be easily copied. Then, Intel gets media companies to include some extra metadata in video media like Blu-ray discs. This metadata can contain revoked device keys, so that if a TV manufacturer violates the terms of the license agreement (e.g. leaks their key, or sells a device that makes copies of video content), that manufacturer's TVs won't be able to play new content that starts including their key in the revocation list. Of course, Intel's HDCP master key was either leaked or reverse-engineered, so anyone can generate their own valid device keys. Intel will probably sue you if you do this, I guess.
- KaiserPro 3y agoWe already have well established systems to prove the provenance of images and other sources. At the moment the internet is a wash with bullshit images. Its imperative that news outlets are at a high enough standard to actually prove the provenance of them. You don't trust some bloke off facebook asserting that something is true, its the same for images.
- epivosism 3y agoWasn't this obvious from the get go that this can't work? If AI will eventually generate say 10k by 10k images, I can resize to 2.001k by 1.999k or similar, and I just don't get how any subtle signal in the pixels can persist through that. Maybe you could do something at the compositional level, but that seems restrictive to the output. Maybe something about like larger regions average color balance or something? But you wouldn't be able to fit many bits in there, especially when you need to avoid triggering accidentally. Also: here are some play money markets for whether this will work: https://manifold.markets/Ernie/midjourney-images-can-be-effectivel https://manifold.markets/Ernie/midjourney-images-can-be-effe... https://manifold.markets/Ernie/openai-images-have-a-useful-and-har https://manifold.markets/Ernie/openai-images-have-a-useful-a...
- charcircuit 3y agoNormal watermarking solutions can survive resizes.
- Lammy 3y ago> Wasn't this obvious from the get go that this can't work? It needs to publicly fail first to manufacture consent for full surveillance of every human interaction with any computer. Nobody would ever want that otherwise.
- MelatoninGreat 3y ago[flagged]
- 998244353 3y agoThe actual paper seems to be https://arxiv.org/abs/2310.00076 https://arxiv.org/abs/2310.00076.
- bulla 3y agoWhat happened to C2PA?
- whywhywhywhy 3y agoI’ll never get over the “invisible_watermark” Python package being entirely visible to the naked eye, obviously degrades the image in an way that’s unacceptable and even easily spottable on any image once you know what it looks like.