4 ms·

To add to this a bit - I do think an embedded steganographic-type “signature” of chatgpt output could be enormously valuable - without it we can wave goodbye to

To add to this a bit - I do think an embedded steganographic-type “signature” of chatgpt output could be enormously valuable - without it we can wave goodbye to the efficacy of things like school assignments. Such a signature is very different than a typical cryptographic one we have today however. But I expect instead of embedding such a signature they will just save hashes of generated content and sell a billion dollar “checking” service to schools.

dinosaurdynasty 4y ago
https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823 (search for watermark) OpenAI did hire somebody who is investigating watermarking the output of models like GPT-3, so it's definitely possible.
- infotogivenm 4y ago
  Ah that makes sense “watermark” is the word I was fumbling for here (I kept coming up with “fingerprint”)
kwhitefoot 4y ago
School assignments should be what the student does to prepare for face to face evaluation; that is, they are a means not an end in themselves. We need to stop trying to get education on the cheap and put enough real, motivated, and properly trained humans in place to do it properly.
acdha 4y ago
I agree that it would be useful but also considering it adversarially I don’t think it’s practical because the people we most want to stop will strip it and in most cases they can submit samples until they pass just as spammers have been doing forever. There might be a few cases where e.g. maybe OpenAI would be willing to ping turnitin.com but it seems like that approach wouldn’t work in general and would inevitably result in false-positives with potentially serious consequences. I think we’ll end up seeing two changes: restricting things like school or hiring to favor in person, dynamic interactions - which is better in many ways but costs more and has downsides for people with social anxiety - and more pressure to make the internet less anonymous, especially if that’s linked to physical devices similar to what Cloudflare and Apple have been doing to prevent the bulk attacks (it wouldn’t help much with a cheating student). Depressingly, all of that seems expensive and likely to have substantial collateral damage.
nonameiguess 4y ago
How good is perceptual hashing for text? It seems trivial for a student with any modicum of intellect to delete a sentence and re-word a few others, same as you would do now after bullying a human nerd into writing your essay for you.

dinosaurdynasty 4y ago

https://scottaaronson.blog/?p=6823 https://scottaaronson.blog/?p=6823 (search for watermark) OpenAI did hire somebody who is investigating watermarking the output of models like GPT-3, so it's definitely possible.

infotogivenm 4y ago
Ah that makes sense “watermark” is the word I was fumbling for here (I kept coming up with “fingerprint”)

kwhitefoot 4y ago

School assignments should be what the student does to prepare for face to face evaluation; that is, they are a means not an end in themselves. We need to stop trying to get education on the cheap and put enough real, motivated, and properly trained humans in place to do it properly.

acdha 4y ago

I agree that it would be useful but also considering it adversarially I don’t think it’s practical because the people we most want to stop will strip it and in most cases they can submit samples until they pass just as spammers have been doing forever. There might be a few cases where e.g. maybe OpenAI would be willing to ping turnitin.com but it seems like that approach wouldn’t work in general and would inevitably result in false-positives with potentially serious consequences. I think we’ll end up seeing two changes: restricting things like school or hiring to favor in person, dynamic interactions - which is better in many ways but costs more and has downsides for people with social anxiety - and more pressure to make the internet less anonymous, especially if that’s linked to physical devices similar to what Cloudflare and Apple have been doing to prevent the bulk attacks (it wouldn’t help much with a cheating student). Depressingly, all of that seems expensive and likely to have substantial collateral damage.

nonameiguess 4y ago

How good is perceptual hashing for text? It seems trivial for a student with any modicum of intellect to delete a sentence and re-word a few others, same as you would do now after bullying a human nerd into writing your essay for you.