2 ms·

I think if we ran the GPT2 spotting model against anyone's comment history we'd find a couple comments that scored really high. I doubt it's actually related to

I think if we ran the GPT2 spotting model against anyone's comment history we'd find a couple comments that scored really high. I doubt it's actually related to your writing style. Alternative hypothesis is that it's not about how you're writing but where. HN appears to have been used to train GPT3.5. I don't know if it was used to train GPT2, but it might have done. So your comments might be in the training set.

ggm 4y ago
I already suspect my blog writings and standards work in IETF are, it wrote a pretty good CV covering letter/brief biography for me according to a relative (also a computer scientist) who tested it. He also noted it stuffed up his, claimed he'd written on work which was demonstrably untrue. RFCs and associated mailing lists are pretty widely distributed and certainly RFCs aren't ring-fenced from GPT. So if chatbot detectors are bad on people who are inputs to the ML can't that be added to the detector?
- maxbond 4y ago
  That's pretty cool. When I tried stuff like that, it was entirely based off of the prompt I gave it. I asked it to criticize my HN posts, and it came up with a story about how I was a charlatan; I asked it to summarize my views on Python exceptions in my HN posts, and it came up with a story about how I'm a world renowned Python expert. (I'm neither, but have written nothing noteworthy, so it's unsurprising.) It could maybe be added to the detector but I think we're going to discover "did this output come from this model?" is ultimately undecidable. My intuition is that this is somehow equivalent to the halting problem, but that's a wild guess, and I'm not an AI researcher or computer scientist (just a humble SWE). Something like, if you start generating prompts that might take you to a part of latent space where this output could have been produced, you don't know when to stop generating longer and longer prompts.

ggm 4y ago

I already suspect my blog writings and standards work in IETF are, it wrote a pretty good CV covering letter/brief biography for me according to a relative (also a computer scientist) who tested it. He also noted it stuffed up his, claimed he'd written on work which was demonstrably untrue. RFCs and associated mailing lists are pretty widely distributed and certainly RFCs aren't ring-fenced from GPT. So if chatbot detectors are bad on people who are inputs to the ML can't that be added to the detector?

maxbond 4y ago
That's pretty cool. When I tried stuff like that, it was entirely based off of the prompt I gave it. I asked it to criticize my HN posts, and it came up with a story about how I was a charlatan; I asked it to summarize my views on Python exceptions in my HN posts, and it came up with a story about how I'm a world renowned Python expert. (I'm neither, but have written nothing noteworthy, so it's unsurprising.) It could maybe be added to the detector but I think we're going to discover "did this output come from this model?" is ultimately undecidable. My intuition is that this is somehow equivalent to the halting problem, but that's a wild guess, and I'm not an AI researcher or computer scientist (just a humble SWE). Something like, if you start generating prompts that might take you to a part of latent space where this output could have been produced, you don't know when to stop generating longer and longer prompts.