7 ms·
> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the
by simonw 2mo ago
> When a supported Claude model generates text, it weaves an imperceptible watermark directly into the text itself. You won’t see it, and it doesn’t change the meaning, quality, or readability of Claude’s response.
I'd like to know a lot more about how that works.
A lot of my interactions with Claude return pretty precise text. If I ask it to edit a project and refactor a specific function in several places I know exactly what I want to happen, it will NOT be OK if those refactors have some kind of weird pattern baked into their text to act as a watermark.
I guess this may be covered by this:
> Content generated by Claude may not carry a detectable mark if, for example: [...] The passage is very short, leaving too little text for a reliable signal;
- gajus 2mo agoMost likely watermark will be proportional to the input/output ratio, i.e. if you input a long document and ask to make edits, it will not attempt to watermark it. On the other hand, if you provide a tweet and ask it to write an article, that will include watermark. Just a guess (and yes, it feels flawed)
- stabbles 2mo agoIt will just thread some load-bearing seams through the paragraphs.
- baq 2mo ago> I'd like to know a lot more about how that works. Count load-bearing words using two different algorithms in a belt-and-braces fashion
- seamlessdev 2mo agoBelt, braces, and suspenders.
- isoprophlex 2mo agoDon't forget the suppositories
- quintu5 2mo agoThis is why I never use max effort! I’ll stick with my suspenders, thank you.
- jbs789 2mo agoFair - I should have been honest about the watermark.
- dd8601fn 2mo agoThat’s the real shape of the problem.
- r_lee 2mo agoOne thing worth flagging: those words are load-bearing
- Razengan 2mo agoYou’re absolutely right. Yo momma is doing a lot of heavy lifting here. Her load-bearing methods have the right shape.
- yedidmh 2mo agoI have to push back: you've found the seam.
- akozak 2mo agoMost likely this method https://arxiv.org/pdf/2301.10226 https://arxiv.org/pdf/2301.10226 (EDIT: and Google's SynthID paper which builds on it https://www.nature.com/articles/s41586-024-08025-4 https://www.nature.com/articles/s41586-024-08025-4)
- hannasanarion 2mo agoThat "just add a constant to the green logits" as a fix to the entropy problem is so elegant I love it.
- iamherrylok 2mo agoIf different model providers use different green logits, does that mean they can only tell if the text came from their own model?
- slickytail 2mo ago[dead]
- w_for_wumbo 2mo agoWhat happens if someone handwrites a Claude output, then someone uses that handwritten text as a reference. Now you've got a watermarked idea which may have no direct linkage to the usage of Claude.
- phainopepla2 2mo agoHow is that different from referencing digital text that someone copied and pasted from Claude?
- w_for_wumbo 2mo agoBecause there's an expectation of authenticity from the written word. If you've referenced something handwritten, you don't expect it to be the output of an LLM. Similarly, if you quote someone word-for-word, you wouldn't anticipate their words to be flagged as Claude content, but if someone memorized Claude output word-for-word. That would still be classified as a Claude output. Going forward you could categorize the influence of Claude on a population based off a percentage match between their spoken words with the LLM prose.
- dns_snek 2mo agoAre you worried about being accused of using LLMs to generate your work? As long as you don't plagiarize you have nothing to worry about.
- AlecSchueler 2mo agoWhat if I unknowingly read content written by Claude in various articles and it influences my own writing style?
- Cthulhu_ 2mo agoI'm not too sure about that, people making stuff have already gotten penalized by overzealous AI detectors, most recently Kurtzgesagt.
- 2mo ago
- mihaelm 2mo ago> have some kind of weird pattern baked into their text to act as a watermark. public abstract class BaseAnimalBeanFactoryGeneratedFromClaudeFactory
- siva7 2mo agoI can tell you how: Claude produces a huge wall of text with jargon ridden bullshit and invented terms no human subject matter expert would seriously use and overuse.
- COAGULOPATH 2mo ago>I'd like to know a lot more about how that works. My guess is that it works like Gemini's SynthID: by altering the logprobs of the next token. Like, for every 10th token, instead of outputting the most probable, it outputs the 17th most probable, or something. (Obviously it's way more complicated but I think conceptually this is how it works.) No human will notice this, but a classifier trained on Claude's output will. So it's not like the watermark is the words "le epic bacon" and Claude will output "le epic bacon" in everything. That would be extremely annoying (and easy to defeat).
- thunfischtoast 2mo agoThey still need to choose when to do that though. When I prompt the program to e.g. alter a bash script in a specific way or to recite a longer known text it can't go round and randomly exchange tokens. It has to somehow define what is a simple repeated text from a different origin and what is a novel generation.
- user43928 2mo agoI am wondering how that applies to newly generated code. Odd variable naming? Stylistic choices that are watermarked? Or as someone else noted further down in the comments, it could be more subtle: Between the first and second most likely choice, in certain positions it will consistently choose in a certain way.
- melvinroest 2mo ago> Odd variable naming? Stylistic choices that are watermarked? Whatever it is, I'm sure it's load-bearing.
- asdfsa32 2mo agoYou're absolutely right. But it is not just load-bearing, it is the load-bearing seams.
- silversmith 2mo ago
- miohtama 2mo agoMaybe there is a reason why Opus 5 produces such word salad conversations
- myko 2mo agoSo frustrating to use. And the comments generated by Claude today are unreadable garbage.
- miohtama 2mo agoBut hey, at least the government, will know and track you down, so we are all safe.
- andrewgleave 2mo agoYes. The irritating epigram / aphorism style it now uses is such a regression compared to previous Anthropic models. Probably is the case that this is due to watermarking - though hardly subtle if it is.
- sixtyj 2mo agoIt was quick :) … https://claudewatermarkremover.app/ https://claudewatermarkremover.app/
- shinryuu 2mo agoThough if pangram should be trusted, there are still statistical artifacts that tells you that a text LLM generated. I don't find that to be implausible.
- timpera 2mo agoAlas, Pangram should not be trusted.
- s_dev 2mo agoSo was it going down: "Neutralize engine is temporarily unavailable. Try again."
- arcfour 2mo ago> Honest note: Anthropic has not shipped a public Claude watermark detector yet. This tool uses rewrite-based neutralization — a meaning-preserving paraphrase with a non-Claude model — which is the attack path watermark research points to. Not affiliated with Anthropic. Well, they should have run their own AI slop website through their tool...
- nunez 2mo agoThis attack was actually pointed out in the watermarking paper linked above. The researchers added an instruction to the prompt that switches letters like a Caesar Cipher. It lowers the quality of the output from the LLM but alters the "red list" enough for a watermark detection tool to fail at detecting the watermark.
- fl0id 2mo agoalso their example for rewriting just completely changes it. might as well redo it in this case (with another model or by hand)
- nprateem 2mo agoLoad-bearing==claude
- infinite_spin 2mo agoMy guess is it will be similar to how Genius watermarked lyrics, using things like variants of punctuation https://www.pcmag.com/news/genius-we-caught-google-red-handed-stealing-lyrics-data https://www.pcmag.com/news/genius-we-caught-google-red-hande...
- toxik 2mo agoIn program code? Unlikely, surely¡
- DaiPlusPlus 2mo agoThat was actually the cause of an issue I had a couple of years ago: I had hand-typed JSON using my iPad into GitHub’s online text editor and Safari helpfully used “pretentious quotes” instead of "old-school quotes" - and the JSON library used by the program to read that file had relaxed parsing rules that accepted actual JS object literals without quoted property names; so the fancy-quotes were interpreted as part of the key-names. This took ages to figure out because when human-eyeballing the JSON file it looked perfectly fine in Notepad.
- kccqzy 2mo agoI don’t doubt your experience, but many people are intimately aware of the use of proper Unicode quote characters, in any reasonable font they choose. For me, one of the first things I learned when using LaTeX is how the quotes are transformed from the source to the typeset document; since then I’ve become extremely sensitive to the kind of quotes I see.
- deleted 2mo ago[deleted]
- infinite_spin 2mo agoConsider these naming options: > total = calculate(items) > result = calculate(items) > value = calculate(items) > amount = calculate(items) All of them are reasonable options. If we bias the model's output so that one of them is more likely than the others, then we can reconstruct that watermark if enough of these frames are present.
- invalidusernam3 2mo agoOff the top of my head I would have thought zero width characters (eg: U+200B, U+200C) making some unique identifier sprinkled in amongst the output. But obviously far from foolproof since they could simply be removed.
- wrsh07 2mo agoScott Aaronson talks about his project at OpenAI to do this^ You can carefully select which pseudorandom number generator (prng) you use to be able to id text of a certain length. I expect there is some performance characteristic you have to manage since you're doing this on every inference, but once you do that it doesn't change the output in any meaningful way (the prng is still a statistically valid prng, it just happens to let you check if the output used that prng) The point is that you can do this simply by swapping to a different RNG, which isn't noticeable to the end user, and while it changes the output, it's not any different from how using a different seed or being lumped in a different batch will change the output. ^ excerpt: > So then to watermark, instead of selecting the next token randomly, the idea will be to select it pseudorandomly, using a cryptographic pseudorandom function, whose key is known only to OpenAI. That won’t make any detectable difference to the end user, assuming the end user can’t distinguish the pseudorandom numbers from truly random ones. But now you can choose a pseudorandom function that secretly biases a certain score—a sum over a certain function g evaluated at each n-gram (sequence of n consecutive tokens), for some small n—which score you can also compute if you know the key for this pseudorandom function.
- EE84M3i 2mo agohttps://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823 is the source?
- wrsh07 2mo agoThat's what I excerpted, although I had seen it presented from his talk at Stony Brook
- denverllc 2mo agoI’ve always wondered how this works when we only observe the final output and not the internal state that’s used to generate the output. The LLM presumably generates f(input, RNG) but we only can observe f(RNG).
- Groxx 2mo ago
- kkukshtel 2mo agoI think a lot of the examples below are projecting more complicated options, but it could also be something just as simple as using a word with a hyphen in it every prime-numbered sentence. Or any other "puzzle-y" pattern.
- cush 2mo agoCertainly it can't watermark text with low entropy. If you're renaming a function using claude it won't be marked
- johnjwang 2mo agoI ran some tests from the watermarking literature: https://johnjwang.com/post/2026/08/12/how-claude-watermarking-probably-works/ https://johnjwang.com/post/2026/08/12/how-claude-watermarkin... Short version: it's likely a SynthID style statistical watermarking scheme that Anthropic is using. There's likely no hidden unicode and no large stable green/red scheme. Longer version: I found some really statistically significant outcomes when using Gloaguen et al.’s (https://www.sri.inf.ethz.ch/blog/probingsynthid https://www.sri.inf.ethz.ch/blog/probingsynthid) detection algorithm for SynthID, though unfortunately it seems like the models might be pre-biased to pass that test already. The difficult thing is that it's hard to conclusively know which models definitely already have watermarking enabled and if there are any models that don't have watermarking, which means that being able to get a highly positive statistical result is much harder. You can take a look at the full writeup, but many of the clues from Anthropic's blog post, plus negative results on many of the simpler tests, point to them using such a scheme. Of course, it's possible that they came up with a novel new scheme/algorithm, but amongst the available literature, the SynthID family of models is probably the best bet.