4 ms·

I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI sa

by bko 3y ago

I listened to a podcast with Scott Aaronson that I'd highly recommend [0]. He's a theoretical computer scientist but he was recruited by OpenAI to work on AI safety. He has a very practical view on the matter and is focusing his efforts on leveraging the probabilistic nature of LLMs to provide a digital undetectable watermark. So it nudges certain words to be paired together slightly more than random and you can mathematically derive with some level of certainty whether an output or even a section of an output was generated by the LLM. It's really clever and apparently he has a working prototype in development. Some work arounds he hasn't figured out yet is asking for an output in language X and then translating it into language Y. But those may still be eventually figured out. I think watermarking would be a big step forward to practical AI safety and ideally this method would be adopted by all major LLMs. That part starts around 1 hour 25 min in. [0] https://podcasts.apple.com/us/podcast/20-reform-ai-alignment-with-scott-aaronson/id1544393261?i=1000608689024 https://podcasts.apple.com/us/podcast/20-reform-ai-alignment...

Waterluvian 3y ago
I wonder if this will inevitably punish a small slice of students whose style matches these watermarks. It’s not like they can actually add a unique fingerprint. They can just add a statistically significant signal.
- sebzim4500 3y ago
  The scheme Aaronson is talking about is statistically sound and will not penalize a student who happens to have a particular style. Basically, you replace the random noise used in the sampling with a cryptographic strategy which biases certain n-tuples based on a pseudorandom number generator. It does not work with temperature = 0 but almost no one uses that. If the student was stupid enough to not modify the essay, you would be able to prove with 99.999% confidence that it was written by a particular LLM. If they modify it slightly then you will have a lower confidence, but they will still be caught.
  - Waterluvian 3y ago
    Thanks. This sounds compelling. I’ll look into this further for the details.
    - bko 3y ago
      I found the transcript. It starts around here: > Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi. https://axrp.net/episode/2023/04/11/episode-20-reform-ai-alignment-scott-aaronson.html https://axrp.net/episode/2023/04/11/episode-20-reform-ai-ali...
wintermutestwin 3y ago
Neat idea, but running your ChatCPT output through qillbot (or another paraphraser) would strip this watermark.
alchemist1e9 3y ago
His work is pointless as uncensored open source models are 6-8 months away from being best at everything. Eduction should be redesigned with LLMs at the center of it, not attempting to restrict it. They are an incredible learning tool.
- twobitshifter 3y ago
  Which OS models?
  - alchemist1e9 3y ago
    I believe uncensored Wizard Vicuna is one of the leaders right now. Remember the LoRA technique is creating many customized models. Meaning you load a different model depending on what you’re looking to generate. Even OpenAI knows their lead is shrinking very fast.
- bko 3y ago
  I don't know if these open source models will be better. The one's I've seen so far have been pretty terrible. OpenAI did a million little hacks to get the LLM to produce such good responses. Even google couldn't figure out a better way to train their model than to fit it to GPT responses. This may be a heretical opinion here, but open source doesn't always win esp if its an incredibly technical task and it costs a lot to train and host. Why didn't open source get us a reasonable search engine? Why isn't LibreOffice as good as Microsoft Office? Why do we pay for any software? I mean sure, you can always cheat. You can pay someone to write essays for you or take your tests. But watermarks would be a huge step forward to detect AI generated content.
kibwen 3y ago
> or even a section of an output was generated by the LLM A probabilistic approach like the one described cannot detect small-scale LLM usage. It takes a lot of text for the random "undetectable" word pairings to become statistically evident with any sort of confidence.
CyberDildonics 3y ago
digital undetectable watermark If it's undetectable, how does it function as a watermark?
rahimnathwani 3y ago
OP makes a good argument as to why this would be ineffective: ... even if we could definitively tell whether any given word was produced by ChatGPT, we still couldn’t prevent cheating. The ideas on the paper can be computer-generated while the prose can be the student’s own. No human or machine can read a paper like this and find the mark of artificial intelligence.

by bko 3y ago

Waterluvian 3y ago

I wonder if this will inevitably punish a small slice of students whose style matches these watermarks. It’s not like they can actually add a unique fingerprint. They can just add a statistically significant signal.

sebzim4500 3y ago
The scheme Aaronson is talking about is statistically sound and will not penalize a student who happens to have a particular style. Basically, you replace the random noise used in the sampling with a cryptographic strategy which biases certain n-tuples based on a pseudorandom number generator. It does not work with temperature = 0 but almost no one uses that. If the student was stupid enough to not modify the essay, you would be able to prove with 99.999% confidence that it was written by a particular LLM. If they modify it slightly then you will have a lower confidence, but they will still be caught.
- Waterluvian 3y ago
  Thanks. This sounds compelling. I’ll look into this further for the details.
  - bko 3y ago
    I found the transcript. It starts around here: > Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi. https://axrp.net/episode/2023/04/11/episode-20-reform-ai-alignment-scott-aaronson.html https://axrp.net/episode/2023/04/11/episode-20-reform-ai-ali...

wintermutestwin 3y ago

Neat idea, but running your ChatCPT output through qillbot (or another paraphraser) would strip this watermark.

alchemist1e9 3y ago

His work is pointless as uncensored open source models are 6-8 months away from being best at everything. Eduction should be redesigned with LLMs at the center of it, not attempting to restrict it. They are an incredible learning tool.

twobitshifter 3y ago
Which OS models?
- alchemist1e9 3y ago
  I believe uncensored Wizard Vicuna is one of the leaders right now. Remember the LoRA technique is creating many customized models. Meaning you load a different model depending on what you’re looking to generate. Even OpenAI knows their lead is shrinking very fast.
bko 3y ago
I don't know if these open source models will be better. The one's I've seen so far have been pretty terrible. OpenAI did a million little hacks to get the LLM to produce such good responses. Even google couldn't figure out a better way to train their model than to fit it to GPT responses. This may be a heretical opinion here, but open source doesn't always win esp if its an incredibly technical task and it costs a lot to train and host. Why didn't open source get us a reasonable search engine? Why isn't LibreOffice as good as Microsoft Office? Why do we pay for any software? I mean sure, you can always cheat. You can pay someone to write essays for you or take your tests. But watermarks would be a huge step forward to detect AI generated content.

kibwen 3y ago

> or even a section of an output was generated by the LLM A probabilistic approach like the one described cannot detect small-scale LLM usage. It takes a lot of text for the random "undetectable" word pairings to become statistically evident with any sort of confidence.

CyberDildonics 3y ago

digital undetectable watermark If it's undetectable, how does it function as a watermark?

rahimnathwani 3y ago

OP makes a good argument as to why this would be ineffective: ... even if we could definitively tell whether any given word was produced by ChatGPT, we still couldn’t prevent cheating. The ideas on the paper can be computer-generated while the prose can be the student’s own. No human or machine can read a paper like this and find the mark of artificial intelligence.