4 ms·
I would love to see what this looks like in practice. Especially in generated code. I assume this is more than insertion of non visible special unicode whitespa
by travisgriggs 2mo ago
I would love to see what this looks like in practice. Especially in generated code. I assume this is more than insertion of non visible special unicode whitespace characters, but more in the pattern of the text content itself?
- AtHeartEngineer 2mo agonon visible text is extremely easy to filter with a git hook, a post tool call hook, or just a script. I doubt they are doing that
- wolfy1993 2mo agoIIRC, watermarking text could be as simple as training the model to use specific words/phrases more frequently than what you would expect to find in human-written text, to the point where it's highly statistically improbable that it wasn't AI generated. I assume similar logic could apply to code in the form of functions/code styling. That's probably an over simplification. Also a solid defence that can be used against complaints about the way AI writes text.
- toufka 2mo agoI'm guessing - probably some textual variation on Benford's law? [1]. Trivial for compute, painful for a human. - "Ensure distribution of vowels is in >99th percentile of human work" - "Ensure the distribution of the letter "s" is within 99th percentile of human work" - "Ensure the distribution of the letter "L" is periodic with periodicity within 5% of 1/N characters. - "Ensure there is a cross-linguistic 'typo' (colour vs color) at 1/N words, where N: 1000 = Model1, 2000 = Model2, 3000 = Model3. - "Ensure the distribution of tense error is within 99th percentile of human work" If more than 3 dimensions have a score >99% percentile of human, let's call it watermarked... - 1) https://en.wikipedia.org/wiki/Benford%27s_law https://en.wikipedia.org/wiki/Benford%27s_law
- Computer0 2mo agoI would hate to have any of these rules effecting my output
- limbicsystem 2mo agoI see what you did there!
- Der_Einzige 2mo agoModels can't reliably follow instructions involving their own logprobs unless they can take agentic control and use quite sophisticated dynamic grammars/structures/constraints to force this behavior in one shot (which can be slow and the dynamic grammar modification feature isn't supported in closed model APIs for safety reasons) or repeated attempts at rewriting which is expensive/slow. Yes they can do this, but it's more likely closer to the original "red token, green token" paper: https://arxiv.org/abs/2301.10226 https://arxiv.org/abs/2301.10226 i.e. take half of your LLMs vocabulary, and upweight its probabilities by ~55% to the other half's ~45%, and scan for overuse of this half of all tokens. You can even choose a different half/slice for every individual user, for every individual action. You can implement this under the hood cheaply with logit-biasing.
- Terretta 2mo agoConsidering how weirdly detuned tokens selections have become in Anthropic's LLM prose in recent models, there is a chance this goes unnoticed in everyday use.
- lovetocode 2mo agoWell put. I wonder if this will inevitably increase usage costs because the model has to do more work to find a solution that meets watermarking standards.
- rcxdude 2mo agoIt essentially looks like the difference between two different runs of the model with the same prompt but different seeds. The watermark is essentially a small bias in the model such that when there's multiple different tokens that could conceivably follow the previous token, the model will only pick some subset of them (the subset is derived from a hash of the previous token). This bias can then be checked for statistically (without needing access to the model and without needing the whole prompt), and for longer text where there's enough freedom in word choice you can show that it would be vanishingly improbable to accidentally follow the rules in the watermark.