8 ms·

Could you do the stupid thing and have openai et al offer an API that, given text (or a hash of some text) returns whether that hash was generated by the servic

Could you do the stupid thing and have openai et al offer an API that, given text (or a hash of some text) returns whether that hash was generated by the service? Some 3rd party companies specialize in cheat detection and pay for the API then get paid by schools or whoever to detect detect cheaters. Difficulties I can think of: * Getting ai companies to offer this. I don't think it comes with a downside for them really though. You couldn't use it to actually retrieve results in any way, and you wouldn't have prompts. * This only detects exact matches, people changing the output would defeat it. Fixable with some kind of fuzzy search that returns a 'distance to nearest response' but this obviously makes it more expensive and difficult to run, use, etc. * People could still run the model themselves, as models get better and more expensive maybe this becomes less of a problem. Or maybe models get small enough while still generating good output that it becomes more of a problem. Who knows. At least this would avoid AI grading AI issues

mkoubaa 3y ago
Yes but you insert one space and the checksum changes
- adriancooney 3y ago
  I wonder if comparing embeddings could work here? It might be more resilient to cosmetic changes.
dingledork69 3y ago
Locally ran models exist, and more will come and they'll become cheaper and easier to run.
- throwaway290 3y ago
  Even if ClosedAI doesn't get government to ban sales of powerful GPUs to the public. Homebrew models will not be of comparable quality in near future. Don't be fooled by how cheap ChatGPT is, it is not profitable.
  - MacsHeadroom 3y ago
    "Homebrew" Stable Diffusion beat DALL-E 2. There's zero reason this won't repeat for LLMs. All indications are that it will.
    - throwaway290 3y ago
      Sorry I don't think if you need to wait half an hour to see a result it's the same thing really. You don't need to worry about deluge of AI content if it can only come from a few crazy people with top of the line hardware spending days iterating on prompts. Especially if really good GPUs start requiring a license. At some point it's just faster to learn to write properly.
      - dingledork69 3y ago
        You are fooling yourself if you think the next generation of CPUs won't have ML coprocessors.
        throwaway290 3y ago
        Apple's neural engine has been around in phones and laptops for years. Not even close to what's required for offline chatgpt. What's your point? Which of us is fooling themselves? Average consumer hardware is not showing any signs of being capable of this in near term. Hype is hype, but use your own head. Don't forget that these homegrown models also require training data, scraped and cleaned, an individual or nonprofit can't do that.
        srslack 3y ago
        >an individual or nonprofit can't do that. https://laion.ai/ https://laion.ai/ https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b There's dozens if not hundreds more individuals and nonprofits doing these things.
        3y ago
ShamelessC 3y ago
OpenAI has released a model for that type of detection. It doesn’t work 100% of the time. The idea that you can just “hash all the AI generated answers and compare against them” is flawed because that problem is intractable.
sebzim4500 3y ago
Scott Aaronson (who is temporarily working for OpenAI) proposed a cryptographic scheme where the generated text can be watermarked without any decrease in quality. It has most of the same problems you list, except it is much more robust against small changes to the text. As of a few weeks ago, he mentioned that OpenAI had a working implementation and were discussing whether to start using it. I assume they'd tell people before they turn it on in prod, I see no advantage in secrecy.
- rjmunro 3y ago
  There's an interesting scheme described here: https://www.youtube.com/watch?v=XZJc1p6RE78 https://www.youtube.com/watch?v=XZJc1p6RE78 Essentially by biasing the LLM's token choices slightly you can infer with high probability later that the text was generated by it.
- dragonwriter 3y ago
  > As of a few weeks ago, he mentioned that OpenAI had a working implementation and were discussing whether to start using it. I assume they'd tell people before they turn it on in prod, I see no advantage in secrecy. Watermarking has zero-to-negative-value for the user of the generation service, but value for users of the detection service that the common vendor of both services will sell, so the only reason to announce that watermarking is active is because you are ready to sell the detection service leveraging it. Otherwise, its just a disincentive to some users to use the generation service with no upside.
  - sebzim4500 3y ago
    Sure, but that's true of lots of things OpenAI does. For example, they removed a bunch of functionality from their API for the new models, presumably for safety reasons.
- sieongioetnio 3y ago
  This will catch only the laziest and stupidest cheaters. Crafting an argument is the hard part of an essay. A student can rewrite an essay, leaving not a single word in place, but still be guilty of plagiarism because the argument is the same. This is hard to prove even with the two documents side by side. It is harder still when the source could be any of a million documents. Of course, many cheaters are that stupid and lazy. People still just copy and paste essays they found online.

mkoubaa 3y ago

Yes but you insert one space and the checksum changes

adriancooney 3y ago
I wonder if comparing embeddings could work here? It might be more resilient to cosmetic changes.

dingledork69 3y ago

Locally ran models exist, and more will come and they'll become cheaper and easier to run.

throwaway290 3y ago
Even if ClosedAI doesn't get government to ban sales of powerful GPUs to the public. Homebrew models will not be of comparable quality in near future. Don't be fooled by how cheap ChatGPT is, it is not profitable.
- MacsHeadroom 3y ago
  "Homebrew" Stable Diffusion beat DALL-E 2. There's zero reason this won't repeat for LLMs. All indications are that it will.
  - throwaway290 3y ago
    Sorry I don't think if you need to wait half an hour to see a result it's the same thing really. You don't need to worry about deluge of AI content if it can only come from a few crazy people with top of the line hardware spending days iterating on prompts. Especially if really good GPUs start requiring a license. At some point it's just faster to learn to write properly.
    - dingledork69 3y ago
      You are fooling yourself if you think the next generation of CPUs won't have ML coprocessors.
      - throwaway290 3y ago
        Apple's neural engine has been around in phones and laptops for years. Not even close to what's required for offline chatgpt. What's your point? Which of us is fooling themselves? Average consumer hardware is not showing any signs of being capable of this in near term. Hype is hype, but use your own head. Don't forget that these homegrown models also require training data, scraped and cleaned, an individual or nonprofit can't do that.
        srslack 3y ago
        >an individual or nonprofit can't do that. https://laion.ai/ https://laion.ai/ https://www.mosaicml.com/blog/mpt-7b https://www.mosaicml.com/blog/mpt-7b There's dozens if not hundreds more individuals and nonprofits doing these things.
        3y ago

ShamelessC 3y ago

OpenAI has released a model for that type of detection. It doesn’t work 100% of the time. The idea that you can just “hash all the AI generated answers and compare against them” is flawed because that problem is intractable.

sebzim4500 3y ago

Scott Aaronson (who is temporarily working for OpenAI) proposed a cryptographic scheme where the generated text can be watermarked without any decrease in quality. It has most of the same problems you list, except it is much more robust against small changes to the text. As of a few weeks ago, he mentioned that OpenAI had a working implementation and were discussing whether to start using it. I assume they'd tell people before they turn it on in prod, I see no advantage in secrecy.

rjmunro 3y ago
There's an interesting scheme described here: https://www.youtube.com/watch?v=XZJc1p6RE78 https://www.youtube.com/watch?v=XZJc1p6RE78 Essentially by biasing the LLM's token choices slightly you can infer with high probability later that the text was generated by it.
dragonwriter 3y ago
> As of a few weeks ago, he mentioned that OpenAI had a working implementation and were discussing whether to start using it. I assume they'd tell people before they turn it on in prod, I see no advantage in secrecy. Watermarking has zero-to-negative-value for the user of the generation service, but value for users of the detection service that the common vendor of both services will sell, so the only reason to announce that watermarking is active is because you are ready to sell the detection service leveraging it. Otherwise, its just a disincentive to some users to use the generation service with no upside.
- sebzim4500 3y ago
  Sure, but that's true of lots of things OpenAI does. For example, they removed a bunch of functionality from their API for the new models, presumably for safety reasons.
sieongioetnio 3y ago
This will catch only the laziest and stupidest cheaters. Crafting an argument is the hard part of an essay. A student can rewrite an essay, leaving not a single word in place, but still be guilty of plagiarism because the argument is the same. This is hard to prove even with the two documents side by side. It is harder still when the source could be any of a million documents. Of course, many cheaters are that stupid and lazy. People still just copy and paste essays they found online.