3 ms·
What they didn't put in the limitations or other sections (unless I missed it) is that it can only apply to larger creative text, not to structured or repeated
by viraptor 2y ago
What they didn't put in the limitations or other sections (unless I missed it) is that it can only apply to larger creative text, not to structured or repeated output. For example if you want to watermark generated code, you can't produce it as a diff to the existing file - the sampling changes will cause unwanted modifications.
Similar for things like "fix grammar in this long text" will have to tweak random words without a reason, because the existing text can't be 100% reproduced while injecting synth-id.
- jsenn 2y agoThis is discussed in the "Watermarking with Synth-ID Text" section right after they define the Score function: > There are two primary factors that affect the detection performance of the scoring function. The first is the length of the text x: longer texts contain more watermarking evidence, and so we have more statistical certainty when making a decision. The second is the amount of entropy in the LLM distribution when it generates the watermarked text x. For example, if the LLM distribution is very low entropy, meaning it almost always returns the exact same response to the given prompt, then Tournament sampling cannot choose tokens that score more highly under the g functions. In short, like other generative watermarks, Tournament sampling performs better when there is more entropy in the LLM distribution, and is less effective when there is less entropy.