4 ms·
The scheme Aaronson is talking about is statistically sound and will not penalize a student who happens to have a particular style. Basically, you replace the
by sebzim4500 3y ago
The scheme Aaronson is talking about is statistically sound and will not penalize a student who happens to have a particular style.
Basically, you replace the random noise used in the sampling with a cryptographic strategy which biases certain n-tuples based on a pseudorandom number generator. It does not work with temperature = 0 but almost no one uses that.
If the student was stupid enough to not modify the essay, you would be able to prove with 99.999% confidence that it was written by a particular LLM. If they modify it slightly then you will have a lower confidence, but they will still be caught.
- Waterluvian 3y agoThanks. This sounds compelling. I’ll look into this further for the details.
- bko 3y agoI found the transcript. It starts around here: > Scott Aaronson: Exactly. In fact, we have a pseudorandom function that maps the N-gram to, let’s say, a real number from zero to one. Let’s say we call that real number ri for each possible choice i of the next token. And then let’s say that GPT has told us that the ith token should be chosen with probability pi. https://axrp.net/episode/2023/04/11/episode-20-reform-ai-alignment-scott-aaronson.html https://axrp.net/episode/2023/04/11/episode-20-reform-ai-ali...