3 ms·

This article goes into it a little bit, but an interview with Scott Aaronson goes into some detail about how watermarking works[0]. He's a theoretical computer

by bko 2y ago

This article goes into it a little bit, but an interview with Scott Aaronson goes into some detail about how watermarking works[0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. It's really clever and apparently he has a working prototype in development. Some work arounds he hasn't figured out yet is asking for an output in language X and then translating it into language Y. But those may still be eventually figured out. I think watermarking would be a big step forward to practical AI safety and ideally this method would be adopted by all major LLMs. That part starts around 1 hour 25 min in. > Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi. https://axrp.net/episode/2023/04/11/episode-20-reform-ai-alignment-scott-aaronson.html https://axrp.net/episode/2023/04/11/episode-20-reform-ai-ali...

littlestymaar 2y ago
Sounds interesting, but it also sounds like something that could very well be circumvented by using a technique similar to speculative decoding: you use the censored model like you'd use the fast llm in speculative decoding, and you check whether the other model agrees with it or not. But instead of correcting the token every time both models disagree like you'd do with speculative decoding, you just need to change it often enough to mess with the watermark detection function (maybe you'd change every other mismatched token, or maybe one every 5 tokens would be enough to reduce the signal-to-noise ratio below the detection threshold). You wouldn't even need to have access to an unwatermarked model, the “correcting model” could even be watermaked itself as long as it's not the same watermarking function applied to both. Or am I misunderstanding something?
- jkhdigital 2y ago
  No you've got it right. Watermarks like this are trivial to defeat, which means they are only effective against lazy users like cheating college students and job applicants.
nicce 2y ago
I don't think that provable watermarking is possible in practice. The method you mention is clever, but before it can work, you would need to know the probability of the every other source which could also be used to generate the output for the same purpose. If you can claim that the probability of that model is much higher on that model than in any other place, including humans, then watermark might give some stronger indications. You would also need to define probability graph based on the output length. The longer the output, more certain you can be. What is the smallest amount of tokens that cannot be proved at all? You would also need include humans. Can you define that for human? All LLMs should use the same system uniformally. Otherwise, "watermaking" is doomed to be misused and not being reliable enough. False accusations will be take a place.
- A_D_E_P_T 2y ago
  I agree. I'd add that not only could human-written content fail the test -- it's also the case that humans will detect the word pairing, just as they detected "delve" and various other LLM tells. In time most forms of watermarking along those lines will seem like elements of an LLM's writing style, and will quickly be edited out by savvy users.
123yawaworht456 2y ago
>So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. hah, every single LLM already watermarks its output by starting the second paragraph with "It is important/essential to remember that..." followed by inane gibberish, no matter what question you ask.
- AlienRobot 2y ago
  I've always felt you'd be able to tell someone uses Reddit because they'll reply to a comment starting the sentence with "The problem is that..." Now LLMs are trained on Reddit users.
  - badsectoracula 2y ago
    Sounds like LLMs are trained on my posts because i tend to use both of those phrases :-P
nprateem 2y ago
Or just check whether text contains the word delve and it's most likely AI generated. I fucking hate that word now.

by bko 2y ago

littlestymaar 2y ago

Sounds interesting, but it also sounds like something that could very well be circumvented by using a technique similar to speculative decoding: you use the censored model like you'd use the fast llm in speculative decoding, and you check whether the other model agrees with it or not. But instead of correcting the token every time both models disagree like you'd do with speculative decoding, you just need to change it often enough to mess with the watermark detection function (maybe you'd change every other mismatched token, or maybe one every 5 tokens would be enough to reduce the signal-to-noise ratio below the detection threshold). You wouldn't even need to have access to an unwatermarked model, the “correcting model” could even be watermaked itself as long as it's not the same watermarking function applied to both. Or am I misunderstanding something?

jkhdigital 2y ago
No you've got it right. Watermarks like this are trivial to defeat, which means they are only effective against lazy users like cheating college students and job applicants.

nicce 2y ago

I don't think that provable watermarking is possible in practice. The method you mention is clever, but before it can work, you would need to know the probability of the every other source which could also be used to generate the output for the same purpose. If you can claim that the probability of that model is much higher on that model than in any other place, including humans, then watermark might give some stronger indications. You would also need to define probability graph based on the output length. The longer the output, more certain you can be. What is the smallest amount of tokens that cannot be proved at all? You would also need include humans. Can you define that for human? All LLMs should use the same system uniformally. Otherwise, "watermaking" is doomed to be misused and not being reliable enough. False accusations will be take a place.

A_D_E_P_T 2y ago
I agree. I'd add that not only could human-written content fail the test -- it's also the case that humans will detect the word pairing, just as they detected "delve" and various other LLM tells. In time most forms of watermarking along those lines will seem like elements of an LLM's writing style, and will quickly be edited out by savvy users.

123yawaworht456 2y ago

>So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. hah, every single LLM already watermarks its output by starting the second paragraph with "It is important/essential to remember that..." followed by inane gibberish, no matter what question you ask.

AlienRobot 2y ago
I've always felt you'd be able to tell someone uses Reddit because they'll reply to a comment starting the sentence with "The problem is that..." Now LLMs are trained on Reddit users.
- badsectoracula 2y ago
  Sounds like LLMs are trained on my posts because i tend to use both of those phrases :-P

nprateem 2y ago

Or just check whether text contains the word delve and it's most likely AI generated. I fucking hate that word now.