2 ms·

any watermarking ai researchers who can explain this? what if the llm should - repeat something verbatim (important in a compaction prompt) - there is just o

by tosh 2mo ago

any watermarking ai researchers who can explain this? what if the llm should - repeat something verbatim (important in a compaction prompt) - there is just one correct order of tokens for a somewhat long chain (a certain sequence of control signals) - provide a diff of 2 inputs without punctuation or whitespace wiggle room? how does the drifting work? does it postpone the drifting and drift stronger later? what if max_tokens is set to a low number? in what way does this not affect output quality?

talljeff68 2mo ago
The Anthropic annoucement addresses lots of this. They give an example of generating code where there is clearly a requirement to follow a specific syntax and therefore the watermarking will be much less effective and likely require much larger sample of generated content to build statistical confidence in the validity/existance of a watermark. For code, it is the generated comments that will be more likely/able to contribute to the watermark confidence. The key to understanding the watermark technology is to realize that the model is/was already using randomness to select among the top most probable tokens, often randomly picking between choices of [nearly] equal weight. The watermarking does NOT change the distribution of the random number generation nor does it affect the range of probabilities for which tokens are being considered. Instead, it only drives the sequence of the random numbers such that they form a cryptographically generated known ordering pattern that is determined by the secret key generating the pseudo-random sequence. As a result of the approach, assuming inference is being done with all other parameterization of token selection being the same, there should be NO impact on the output quality....the amount of variation of output is within sample of the variation in output that already exists run-to-run of the same prompt. FWIW too: Google has confirmed this experimentally as well through full scale tests and evaluation of online Gemini output in search result pages.

by tosh 2mo ago

talljeff68 2mo ago

The Anthropic annoucement addresses lots of this. They give an example of generating code where there is clearly a requirement to follow a specific syntax and therefore the watermarking will be much less effective and likely require much larger sample of generated content to build statistical confidence in the validity/existance of a watermark. For code, it is the generated comments that will be more likely/able to contribute to the watermark confidence. The key to understanding the watermark technology is to realize that the model is/was already using randomness to select among the top most probable tokens, often randomly picking between choices of [nearly] equal weight. The watermarking does NOT change the distribution of the random number generation nor does it affect the range of probabilities for which tokens are being considered. Instead, it only drives the sequence of the random numbers such that they form a cryptographically generated known ordering pattern that is determined by the secret key generating the pseudo-random sequence. As a result of the approach, assuming inference is being done with all other parameterization of token selection being the same, there should be NO impact on the output quality....the amount of variation of output is within sample of the variation in output that already exists run-to-run of the same prompt. FWIW too: Google has confirmed this experimentally as well through full scale tests and evaluation of online Gemini output in search result pages.