3 ms·
What if the next token represents a wrong or low-quality answer, but would have only been picked 10% of the time, but now it's picked 20% of the time? Doesn't t
by throwuxiytayq 1mo ago
What if the next token represents a wrong or low-quality answer, but would have only been picked 10% of the time, but now it's picked 20% of the time? Doesn't that obviously decrease the model quality, even though "it might have picked that token anyway"?
- dan-robertson 1mo agoIt would be picked 10% of the time with watermarking. The randomness properties of the PRNG will be very similar to other random number generators, it is just chosen to be vulnerable to a particular cryptanalytic attack (that requires a private key known only to anthropic). I think of it like the Dual_EC_DRGB generator rather than a biased coin.
- throwuxiytayq 1mo agoIt is absolutely possible that it would not continue to be picked 10% of the time with a given fixed watermark key. The implementation literally labels tokens using a keyed hash and then modifies their scores. The entire point of the watermarking system is to bias certain tokens against others, and - as you would expect - this reportedly results in a reduced response diversity.
- deleted 1mo ago[deleted]
- wasabi991011 1mo agoWhat if the token represents a high-quality answer, but would have only been picked 80% of the time, but now it's picked 90% of the time? I'm not entirely sure (haven't read the original synthID proposal), but I believe that the re-weighing is set to make both your scenarios and mine equally likely, averaging out to net Zero effect on quality.