7 ms·

Text AI watermarks will always be trivial to remove

• ch_sm 2mo ago
  here‘s what i don‘t get about this whole discussion. AI companies already store all prompts and responses for future training. just make an API that returns the string distance between a previously generated paragraph and the query? that would sidestep this whole problem class. regulators could even specify how that has to work. what am i missing?
  • dpoloncsak 2mo ago
    Local models?
    • dTal 2mo ago
      Local models enable: - watermark-free generation - the stripping of watermarking from the output of SAAS models Any discussion of watermarking is dead in the water in a world where we are permitted to have these things. I fear for the future.
      • thisoneworks 2mo ago
        Chill my dude. This is just a sane default which will catch normies copy pasting stuff from claude and chatgpt. It's good enough.
        • dTal 2mo ago
          Do you not want "normies" to have local AI? Do you believe that your ability to use esoteric software makes you special, makes you immune to law? The days where you can find refuge in running "weird hacker crap" instead of 100% goobermint approved software are numbered because news flash: AI means there are no normies anymore. Anyone can do anything. No, I will not chill. The war on general purpose computing is gonna get real hot real soon.
        • bethekidyouwant 2mo ago
          Pasting it where?
          • queenkjuul 2mo ago
            Homework assignments, spam emails, LinkedIn
            • deleted 2mo ago
              [deleted]
            • bethekidyouwant 2mo ago
              the api to detect if text is AI generated will be for large bespoke companies, governments etc. not teachers or the general public,
              • queenkjuul 2mo ago
                Universities are functionally large companies
                • bethekidyouwant 2mo ago
                  Oh so you already knew that? If they give it to every teacher it’s the same as giving it to everyone so they’re not going to do that.
                  • queenkjuul 2mo ago
                    And giving it to employees of a large company is materially different in what way?
      • dpoloncsak 2mo ago
        Local models also enable: - not being locked into a provider - not being forced to have your prompts saved by a possible competitor - an alternative to the duolopy we quickly see forming - offline access I fear for the future without local models, much more than the future with them, and would rather everyone had access to a local model than be certain we catch everyone copy+pasting LLM responses. Watermarking would be cool, but it's not worth losing local for.
        • dTal 2mo ago
          To be clear, we agree. The problem is that unless local AIs become "normie friendly" real damn quick, we're gonna lose em, because they're damned inconvenient to power. That is what I fear.
          • dpoloncsak 2mo ago
            My apologies, I misunderstood your original comment to mean "We shouldn't have local models if it breaks watermarking" but yes, I think we're on the same page now
          • bethekidyouwant 2mo ago
            I don’t get how you’re gonna lose them how is a guy from Brussels gonna come to your house and turn off your computer?
            • dTal 2mo ago
              In the short term, governments can suppress free distribution of open weight LLMs. For instance, they could "ask" Hugginface to require an account with a university affiliated email address to download anything. It won't stop you outright, but it will force it underground, slowing everything to a crawl. In the long run... well, general purpose computing is under threat anyway. Platforms with locked bootloaders and mandatory digital signatures outnumber those that don't. Even on "PCs", the openness is merely a cultural norm observed for a particular market segment[0] by Apple and Microsoft - for they are the only ones with true root keys. Even a brand new motherboard comes with Microsoft signing keys pre-flashed, and you are permitted to enroll your own only by grace. The technical infrastructure to flip a switch and lock it all down is now in place, ready to go at the stroke of a pen. Will you be able to get Llama.cpp from "the app store"? I hazard not. If you think this sounds hyperbolic, look at what is happening to Android. [0] They don't even segment it the same! Apple segments by "does it have a keyboard", and Microsoft segments it by "does it have a native x86 processor". Apple runs different software on the same basic hardware, Microsoft runs the same basic software on different hardware, but both have created artificially restricted second class computing categories.
      • nradov 2mo ago
        No need to fear, the future looks bright!
      • graemep 2mo ago
        Very few people will use local models. In practical terms most people use a few big providers.
        • dTal 2mo ago
          Is that a desirable state of affairs?
  • mirashii 2mo ago
    > AI companies already store all prompts and responses for future training. They store some prompts and responses, not all, that's what you're missing.
  • bonoboTP 2mo ago
    > AI companies already store all prompts and responses for future training. They claim to only do this when you agree to this in your personal settings. Though Google does say they will train on it, unless you disable history and only use ephemeral chats. Anthropic has a setting for it and claims not to train by default. Also it would be vulnerable to attacks and privacy problems. You could search for substrings about some suspected information, like "John Smith's medical records show advanced cancer" etc. Of course you'd have to guess the phrasing but still.
    • ch_sm 2mo ago
      Right, my bad; I know they claim to not store _all_ prompts and responses, that was a bit cynical/hyperbolical of me. However, regulation could force them to do it — with all the downsides that come with that. Your privacy argument, on the other hand, makes total sense. Not even security measures like homomorphic encryption or locality-sensitive hashing could fix that basic issue.
  • johnnyo 2mo ago
    1. AI company buys and trains on an author’s book when it gets published, it’s now part of the training data. 2. Attacker asks the LLM for the opening sentences of the book, it goes into the generated responses database. 3. Later, a malicious user shows that the first few sentences of the authors book are identical to a previously generated response.
    • dpoloncsak 2mo ago
      Ignoring the other technical hurdles of the idea...this problem you outlaid is solved with a timestamp in the database, right? You can easily prove if the prompt was before/after publishing date?
      • johnnyo 2mo ago
        If it’s before publication date, does that prove the author used AI inappropriately in their writing? Not really, there are any number of reasons why they might do that. In my own personal writing, sometimes I run it through LLMs to give grammar or writing suggestions.
  • echoangle 2mo ago
    Wouldn't this become pretty difficult to do at scale over time? Is there a way to compare the similarity in a database of responses without doing a search over every entry and comparing them? Because that would probably become pretty slow if literally every LLM output is saved and has to be scanned.
  • rsynnott 2mo ago
    Well, for a start, they’re probably on dodgy ground there with other European regulations. You’re not really supposed to hold onto hold onto potentially very sensitive data that you have no legitimate interest (a term of art; it doesn’t just mean “I want to keep this”) in indefinitely. As far as I know, most of these require user consent for retention?
• ramesh31 2mo ago
  Yeah but it's like saying "Masterlocks will always be easy to pop off with a hammer". Of course, but by doing so you are actively engaging in fraud, which then puts the onus on you and whoever you are attempting to deceive.
  • buf 2mo ago
    Except this isn't like saying that at all. This isn't fraud, because it's legal in almost every circumstance.
    • ramesh31 2mo ago
      >almost every circumstance Key phrase. And I'm not saying fraud in the legal liability sense. If you're not trying to hide the fact that something was LLM generated, then you have no reason to remove it. If you are trying to hide it, then there's probably a reason, i.e. you would face consequences for doing so, therefore it is fraud.
      • burnte 2mo ago
        Or, you simply edit the text the LLM generated ruining the hidden message. That's not even remotely fraud. There are legitimate reasons to edit text. There are fewer legit reasons to bash off someone else's lock with a hammer.
• happytoexplain 2mo ago
  Yeah, but it's better than nothing. People underestimate the value of rules that only take malice and a little knowledge to break. And they tend to exaggerate that underestimation if they... don't like the rule.
  • JoshTriplett 2mo ago
    Exactly. If this works on pull requests, for instance, it'd be really useful for projects trying to do a first-pass filter to close slop spam.
    • hennell 2mo ago
      Can it work on code itself? Obviously if you get AI to write the PR description that could be watermarked, but code isn't going to like random unicode, and the sythID approach feels like it would fall apart in the quite strict syntax of most code.
      • pavon 2mo ago
        If a language supports unicode it will have a fairly permissive definition of whitespace, and it will be easy to generate permutations of the whitespace that meet the syntax requirements. Of course I would want my code formatting tool to normalize that all to plain 0x20 spaces. But it would still be a helpful "brown M&M" test of did you even read CONTRIBUTING and run the code formatter before submitting this PR?
        • JoshTriplett 2mo ago
          Code submitted to a project should follow the formatting standards of that project and its programming language, which doesn't leave much room for flexibility in whitespace. However, there are many ways to write comments, and many ways to write PR descriptions and commit messages, and often many choices of words to name identifiers, and other potential sources of bits of entropy with no functional impact. It doesn't take that many bits to encode a robust "AI was here" indicator.
• firefoxd 2mo ago
  I feel like this is going to end up being like cookie laws. It sounds good, I don't know how any one benefits from it. Currently I can recognize AI text because I read thousands of ai generated text. I know that 110% of yahoo finance news is generated. I don't want to read an AI generated personal blog, but if I do what's the problem really? Other than the companies distinguishing AI text for getting better training data, how do people benefit from watermarked text exactly?
  • tcdent 2mo ago
    This feature is born out of the same government that created the cookie laws, so it makes sense to me that it would follow similar politically-valuable but practically-questionable set of beliefs. Really, Anthropic (and the other labs that follow) are just trying to satisfy the requirements of the law so they can continue to serve the EU. Wether it's actually effective is something else entirely.
    • Lorkki 2mo ago
      The cookie laws and GDPR are hampered by malicious compliance, sure, but also outright intentional non-compliance. What's needed is more efficient oversight and enforcement.
      • theoreticalmal 2mo ago
        How would more efficient compliance help the constant annoying pop ups?
        • setopt 2mo ago
          According to the rules, accepting or refusing cookies should be equally difficult. Making them both difficult makes people not use your site, so the incentive should be to make both options easy if you follow the rules. Malicious compliance is to make cookie acceptance much easier than refusal. Lack of oversight doesn’t punish this. Now, the incentive is to make life hard for people who decline cookies. That results in more annoying pop-ups.
• andy_xor_andrew 2mo ago
  The article mentions you can always simply use a smaller, local, un-watermarked LLM to rephrase the original watermarked text. Which is true, sure. But if we're talking about deterministically taking some watermarked LLM output and having a function removeWatermark(text), it won't necessarily be "trivial" to remove, because the watermark function itself need not be public. Only the API that tests for the watermark need be public, right? Anthropic's magic watermark could be, like the article mentions, something like "every 7th semicolon has a N% chance to be a comma where N is the sum of the last X characters mod Y, and every character in the bit range q1...q2 has a Z% chance to..." etc etc etc. And if Anthropic controls those variables, it would be very difficult to determine the rule, even with some pretty advanced analysis (I would assume). And keep in mind, that example rule I mentioned is pretty naive, too. I expect the actual rule would be way more advanced and not so straightforward as "swap every <charX> for a <charY>"
  • cayleyh 2mo ago
    It could be, but common, we all know that Anthropic's watermark is the using "load bearing", "genuine", and "seam" 1000x more in the same paragraph than any human in history.
    • ljm 2mo ago
      [dead]
    • queenkjuul 2mo ago
      Worth surfacing that you're absolutely right
  • johnjwang 2mo ago
    There exist methods to detect what kind of watermarking tool that someone is using, and most of the big tools have specific signatures that you can look for. For Anthropic, it’s highly likely that the watermark is a SynthID type mark similar to the one that Sean is talking about (I actually ran the analysis here https://johnjwang.com/post/2026/08/12/how-claude-watermarking-probably-works/ https://johnjwang.com/post/2026/08/12/how-claude-watermarkin...). When we get confirmation of whether all models actually are watermarked, I think we’ll be even more confident. Of course it’s always possible that Anthropic has come up with a proprietary scheme, but I think it’s definitely harder to implement. I think the game will be a cat and mouse game similar to LinkedIn and other websites trying to block scrapers: each iteration makes it harder for someone to figure out the watermarking scheme, but likely not impossible
• clemlesne 2mo ago
  I done a C2PA implementation for raw text, if that can ease someone’s life: https://github.com/dualeai/c2patxt https://github.com/dualeai/c2patxt
• jerf 2mo ago
  "What would it even look like to sign ChatGPT outputs? There’s no artifact to pass around." It would look like a lot of little signatures on little bits of text, and then larger signatures on a collection of those chunks once the larger chunk exists. It's not that hard. It's just a lot of signatures. But this scheme could only ever prove that this bit of text was made by a given AI, and validate anything else ever included in the signature hasn't been tampered with. It's not hard to work up a scheme that proves (within reason) a text was generated no earlier than some date by incorporating some sort of information that could only have been known at that date so that could be validated. But this isn't even a step in the direction of proving that something was made by a human. And that's assuming the private keys stay private, which is its own tricky problem. If a private key ever leaks anything signed with it becomes invalidated.
  • zahlman 2mo ago
    > And that's assuming the private keys stay private, which is its own tricky problem. If a private key ever leaks anything signed with it becomes invalidated. We don't appear to be talking about cryptographic signing here. (That would never work for the problem because everyone expects unsigned text anyway.) We're talking about: > It’s basically a text steganography problem (concealing a secret code), made more difficult because the plaintext cannot be arbitrarily manipulated. As in, trying to force ChatGPT output to contain intentionally crafted ChatGPT-specific LLMisms that a human is unlikely to imitate, even one who reads a lot of ChatGPT output.
  • baby_souffle 2mo ago
    > It would look like a lot of little signatures on little bits of text, and then larger signatures on a collection of those chunks once the larger chunk exists. It's not that hard. It's just a lot of signatures. But how is this implemented? It's a few lines of code to implement a basic "identify and strip/replace any non printable ascii, unicode ..." or whatever. A screenshot/OCR will also do this. SO at the end of the day you're left with some dumb rules like "you used `load-bearing` more than once per 500 words, that's AI!"
    • recursivecaveat 2mo ago
• mmooss 2mo ago
  For SynthID and similar solutions, there is much I don't understand ... Here's what I grasp: The AI system scores each token and then selects tokens based on those scores. If we encode something in the token selection routine ('in order choose the 1st, 3rd, 1st, 5th, 2nd, then 1st highest scored tokens'), we can identify AI-generated text by comparing sample text (ST) to the expected text (ET) for that prompt. 1) How do we score the tokens for the ET without the original prompt? Even a Markov-like process needs to start somewhere. 2) To recreate ET don't we need to maintain, until the end of time, the AI state - entire model and code - at the time of ST output? 3) Doesn't #2 require maintaining all states for all AIs? Often you won't know when and from which AI system the ST might have been generated. What happens when an AI vendor goes out of business? 4) To recreate ET, don't we effectively have to rerun the prompt? Won't rerunning it for every verification increase most costs of AI output by an order of magnitude? Most of what AI vendors do would be ST validation.
  • pluto_modadic 2mo ago
    1. score by the past paragraph (this is also how the verifier checks without needing a GPU, it takes the previous length of text) 2. same answer 3. no, it just needs the previous text, private key, and the matrix math (CPU is fine) 4. no, see above.
    • mmooss 2mo ago
      Thanks ... 1. So we can't score the first paragraph (or similar-sized block), and not short texts? Not deal-breaker, but a limitation. 2. Doesn't the score vary by each AI system state - its model, programming, harness, etc.? Claude's output today doesn't match Gemini's, nor Claude from 2 years ago.
• eli 2mo ago
  Anthropic the other day announced future models "launched in the EU" will include watermarking https://support.claude.com/en/articles/16266773-how-claude-marks-ai-generated-content https://support.claude.com/en/articles/16266773-how-claude-m... I was assuming it was something like SynthID rather than just sneaky invisible unicode but it's hard to tell from the description.
• charlieyu1 2mo ago
  Good. Tracking and surveillance have no place in the modern world.
• deadbabe 2mo ago
  Society can overcome this problem by changing the way we think about text. Raw plain text should be banned, all text is cryptographically signed by the editor, gui element, or tool that created it.
  • bonoboTP 2mo ago
    You can ask an AI agent to type into Microsoft Word via computer use.
  • TheCoelacanth 2mo ago
    That sounds dystopian as fuck.
• simon84 2mo ago
  The goal of the AI act is not to determine if an "oh yeah!" comment was AI generated. The target is long papers that falsely claim human review and can have real significant consequences. E.g. research paper, law makers, lawyers, state policies, notaries,... These are much longer content and thus statistically they will disclose a better guess at AI generated content. Asking another AI to paraphrase will not erase the mark (which they are unaware about) but rather cumulatively add their own mark and make it easier to detect. The problem is not to use AI, but to endorse the responsibility of the content you (as a human) deliver and somehow make sure that fake-news, biased content or unverified output is detected as early as possible.
  • avaer 2mo ago
    You'd think that the humans being paid to review these things are actually reviewing them. Journals, laywers etc. are expensive. With the addition of AI, it should be easier than ever to review things on their merits. Maybe some part of it is that the deluge of slop is uncovering how poorly/sloppily these social institutions were working in the first place.
    • truckyday 2mo ago
      [dead]
      • avaer 2mo ago
        Anecdote: I've had at two friends publish "papers" just by slapping their names on something they had literally nothing to do with, out of pure nepotism. This wouldn't be helped by an AI watermark. What would help is if the reviewer used AI to look up the authors and see the authorship claims are dubious. The papers are still up. I do think the academic publishing field is corrupt, which is why I'm not convinced it should be on AI providers to help bail it out of doing its one job (verification and trust).
        • maleldil 2mo ago
          Verifying authorship is an ambiguous task. How much involvement is necessary to qualify for authorship? If someone reads the paper and gives some small feedback, is that enough? What if they were present in one meeting and raised a question that turned out not to be interesting? What if they have no clue about the work but helped with data validation? I'd argue all of these could justify authorship, even if they're just in the middle of the author list. At least in NLP, which can be seen in the generally high number of authors in papers.
• tzury 2mo ago
  If everyone is using ai (per reported revenues) and nobody like the outcome (banned here, dismissed there), then what is the future of AI?
  • krapp 2mo ago
    That is the future of AI. Everyone will use it whether they like it or not because there won't be an alternative.
  • Gigachad 2mo ago
    The internet will become entirely AI bots talking to each other with no humans writing or reading.
  • lofaszvanitt 2mo ago
    Just look at global warming, caused by our totally negligent behaviour.... AI will decimate people's thought processes and they essentially become zombie like. And it will take a few decades until they realize what have they done. It will be a painful experience to shoo off people from the heroin called AI in order for them to use their brains again.
• defen 2mo ago
  AI watermarks feel like they're approaching the problem from the wrong side - no matter what it will be possible to remove the watermark (Via manual rewriting, local LLLMs, etc). Instead it seems like we need "proof of human creation". And the only way I see that being possible is hardware-attested proof of keypresses. Which obviously has huge privacy implications, but how else would you actually know that a piece of content was produced by a human pressing keys on a keyboard? The proof would also need to include timestamps for the keypresses, so tell if someone is just copying from another window.
  • layman51 2mo ago
    That sounds like an interesting idea, but I think there's always weird edge cases. Like what if I pressed the keys to write an essay that was actually generated by an LLM. It would be hardware-attested, but I just transcribed most of my sentences from an LLM and maybe I even introduce my own typos or sentences because I am half-assing the transcription.
    • baby_souffle 2mo ago
      > Like what if I pressed the keys to write an essay that was actually generated by an LLM. It would be hardware-attested, but I just transcribed most of my sentences from an LLM and maybe I even introduce my own typos or sentences because I am half-assing the transcription. This is what's known as the "analogue loophole" and it's why DRM and the related "attest, strongly, at every link" schemes are ultimately destined to fail. Lots of people suggesting we need some sort of cryptographic signature that is unique and unforgeable from every camera; it's the only way to _know_ that something wasn't generated with an LLM! Just point the camera at a sufficiently high-resolution monitor...
    • defen 2mo ago
      You could do that, but a real human doesn't just type things out in one straight shot (modulo typos that are fixed "in-line"). There's going to be editing, pauses, copy-pasting, moving things around, etc. And at worst it puts a cap on how much text you can generate since at minimum it does need to be manually retyped.
• rustcleaner 2mo ago
  >watermarks Whatever happened to just delivering the best product or service? Why must tech be full of ninnying nannies that act against their users, "for their 'safety'‽"
  • dana-s 2mo ago
    Because the best product should still take care of some social accountability, Meta would not sell their shitty glasses without the light indicator and even with it, as it's trivial to hack there's large social pushback.
    • rustcleaner 2mo ago
      It is not the product's job or role to make the customer act in a socially pleasing manner. It seems to happen because a customer-sovereignty respecting competitor hasn't come in to clean up the moral crusader's lunch.
      • Espressosaurus 2mo ago
        Every technology, every action has a moral aspect to it. Just because you want to ignore it doesn't mean it's not there. Society then decides on what morals we want to uphold, or at least tolerate, vs. those we want to change. Laws are hard power to change behavior. Influence/goodwill/etc. are soft power ways to change behavior and bend organizations in ways they may not otherwise want to. Would you pay a pto keep grandma from eating dogfood if it wasn't the law that you have to put a portion of your earnings into Medicare and Social Security?
• fwip 2mo ago
  I don't think this is a good argument. There exist digital watermarking techniques for image and video that are imperceptible to humans but still survive cropping, rotation, resizing, recompression, or an analog round trip (photographing the image or pointing a camera at the video). These watermarks aren't, like, hiding in the low-bits of color information, they're spread among many perceptible details. It's not clear to me that it's impossible, or even especially difficult, to make something that survives a casual LLM paraphrase. Remember, all you need to encode is a single bit of info. There's a lot of space to redundantly encode that signal.
• rsynnott 2mo ago
  “Everyone will simply do fraud” - it is a bizarrely immoral world that the LLMs seem to have unleashed on us. Tech was kinda heading that way anyway, but our friends the magic robots really seem to have turbocharged it.
  • krapp 2mo ago
    Watermarks have always been a joke, but it is kind of crazy how quickly all of the rules went out the window when LLMs came along. If you'd told someone five years ago that people would just give "AI" root access to their systems and let it run unsupervised except for a list of rules that it sometimes just didn't follow - yeah there's just a nonzero chance that any device with access to your credit card will just buy a thousand eggs. No one knows why, it just happens sometimes even if you beg it not to - you'd be committed. I remember seeing watermarks in Stable Diffusion ages ago. We're just going to have to come to terms with the reality that AI generated anything will be indistinguishable from human generated anything soon. Counting fingers already doesn't work, most of the "tells" people think work with text are little better than reading tea-leaves. Watermarking anything is futile.
• phendrenad2 2mo ago
  I have a feeling this is really aimed at coding comments. I've noticed that Opus and Fable have started writing huge block comments all of a sudden.
  • marginalia_nu 2mo ago
    There's been a notable and well noted shift in Claude's vocabulary toward words that aren't quite appropriate, sometimes completely invented. Biggest claude smell at this moment is that it often uses these weird synonyms from fields like law and medicine in places where you'd expect highly specific well defined words in a technical text. From a SynthID (or analogous) perspective, it makes a lot of sense you'd see something like this especially in something like a code comment where there's a finite amount of things to say, and it should be said in a terse manner using well defined language. The room for poetic flourish is very small, so you get... 'substrate' or 'verdict' or 'firmament' or whatever else it's using these days.
  • queenkjuul 2mo ago
    Not only are they huge, they often don't even really make sense, or assume the reader has all the same context as the person who wrote the code to begin with. Basically 90℅ of my PR review comments now are "reword or delete this comment block, it's not helpful as-is"
• Waterluvian 2mo ago
  Maybe this is an information theory thing but can there exist a n“ I am human” shibboleth or is this concept fundamentally impossible? It feel that today you can generally convince someone with text alone that you’re human. Beepity boopity zip zap zoopity today’s AIs aren’t this loose and derpy. Here’s a fTypo and my secret stash of dashes ——-–. But one day even this won’t do, right?
  • StevenWaterman 2mo ago
    You'll have to just say something racist, homophobic, anti-Semitic, etc. More intelligence won't "fix" that because the labs don't want to fix that
    • Waterluvian 2mo ago
      This feels like a Warhammer 40k thing. “We weren’t actually space racists at heart. We just had to say those things to identify against the Necrons. But somewhere along the way the newer generations didn’t see the wink and the nod.”
    • queenkjuul 2mo ago
      Then we'll just think it's Grok
• nonethewiser 2mo ago
  Isnt this absurd? Say im brainstorming a resume bulet point. Its 15 words. I like it but want to condense it to a single line. I give it to the ai and tell it how much overflows and now it gives me back a simplified sentence and 11 words and some extra stenography constraint? What kind of rule could possibly not effect the quality of that output? Ok say i do that on 30% of bullet points. Karen the hiring manager is vehemently anti AI. She gives my resume to her AI scanner and what does she find? This not a rhetorical question. Will it treat the text as a whole and not find it? Does it scan every combination of contiguous terms? It could scan bullet points but i could generate in pairs of 2. What about novels?
  • ruilov 2mo ago
    Suppose I gave you a series of 1000 coin flips. I tell you they were generated by a fair coin. You’re suspicious, you think I hid a watermark in them. But you look at the sequence and 485 are heads, close enough to 50%. You look at the correlation between a coin flip and the next and it’s 0.0433. Close to zero. You do a bunch more stats and everything checks out. So you’re convinced. Then I tell you: calculate the average of every 3rd flip minus the average of every second. It should come out close to zero, and unlikely to be higher than +/- 30 but for my sequence it comes out to +89. Ok, that’s very unlikely. So from now on we share this secret. Whenever I generate sequences of coin flips they all came out with this particular statistic out of whack, but unless you’re looking for it, you can’t detect it. In fact, in the real world I use cryptography such that unless you know my secret key, the specific statistic in question is mathematically undetectable. Now use this sequence of coin flips to pick amongst next tokens in an LLM. It doesn’t change the distribution of words used in the LLM. It doesn’t change its writing style. In fact, unless you know the secret key you cannot detect the watermark. So it’s not that some words are used more often. That would be detectable. It’s just that if you translate back to a series of 0’s and 1’s the average of every 3rd bit minus the average of every 2nd is out of whack.
    • Terr_ 2mo ago
      I don't have the math for it, but I wonder how much text/words/tokens are needed before it become easy to embed an ID for authorship as well. There's got to be some number of bytes where it becomes both reliable and hard to find. "I'm sure glad the LLM was able to fix up my grammar and imagery in the anonymous treatise I made criticizing the authoritarian regime. Hold up, someone's knocking at my door..."
• aaron695 2mo ago
  [dead]
• giridharkannan 2mo ago
  https://declaude.org/watermarking/ https://declaude.org/watermarking/ did a good job in explaining how SynthID works. As per their blog, it feels like it will be difficult to remove watermarking on bigger text and the checking for watermarking is also not complex
  • WalterGR 2mo ago
    There’s some healthy discussion on the submission of that link: https://news.ycombinator.com/item?id=49292932 https://news.ycombinator.com/item?id=49292932
• sbszllr 2mo ago
  I've been working in the media and model IP space for quite many years. What this article misses a bit is the threat model for watermarking in general. Watermarking and fingerprinting have inherently weak security guarantees -- they rely a lot on security through obscurity, weak assumed adversaries to deliver. There are clear trade offs between true positives, false positives and maintaining the quality of the media. It's true for audio-visual media, models and their outputs alike. As much as I like to take shots at poor technical choices by corps and govs, this one is unjustified. Sure, inserting glyphs is bad but biased sampling is as good as it gets in 2026. EDIT: grammar
• Otterly99 2mo ago
  I don't really see how this would work anyway. Even if you find a way to 100% watermark any text, couldn't you just use a non-watermark model? I have a hard time believing every AI company on heart would comply. Hell, even if every AI company on earth decide to somehow apply watermarking to their next model, they would also need to apply it to all the previous version that they commercialize. Given that anybody could make a copy of an open source model right now and would be safe forever, this seems quite the lost cause to me.
  • theshrike79 2mo ago
    This is just managing the low-effort low hanging fruit. The students who, without a single neuron activating, will just copy paste questions to an AI web UI and paste the answers back. Or the mid-tier corporate people who file 50 page "reports" that are 100% AI bullshit. None of them will install a local model with no watermarking or bother looking up some Badonk AI service to do the same thing.
• Eddy_Viscosity2 2mo ago
  Is there a way to not only watermark that it was AI, but also which user generated it? Can it be used to fingerprint?
• thadk 2mo ago
  Would OpenPangram's LoRa fine-tuning training be able to pick up on the patterns of steganography left behind by LLM vendors? https://huggingface.co/pangram/editlens_Llama-3.2-3B https://huggingface.co/pangram/editlens_Llama-3.2-3B (pardon for reposting just now from off-front-page thread on https://declaude.org/watermarking/ https://declaude.org/watermarking/, just thought of question)