3 ms·

How long till OpenAI et al. offer a plagiarism detection service against this?

giobox 3y ago
OpenAI are already attempting, they have a free classifier you can try, but results aren't all that great: "Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases." https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-...
- none_to_remain 3y ago
  But they don't need a fancy AI classifier, they already have the text that ChatGPT outputs
  - thrtythreeforty 3y ago
    They'd be absolute fools not to be keeping a copy of everything it's ever said. How much text has it generated, ever? A couple of dozen terabytes? Maybe?
    - johntb86 3y ago
      According to https://help.openai.com/en/articles/7730893-data-controls-faq https://help.openai.com/en/articles/7730893-data-controls-fa... if you turn off history the conversation will be permanently deleted after 30 days.
- ghaff 3y ago
  Assuming you're actually going to do anything with the results, basically any meaningful number of false positives is a huge problem. If using an LLM (or appearing to have done so) to at least help write a college essay is considered plagiarism, that can easily be grounds for expulsion.
- George83728 3y ago
  > while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Utterly useless for teachers. You can't go around accusing 1 in 11 innocent students of plagiarism, and over the course of a semester you'd have virtually every student get falsely accused at least once. They may as well have not released this tool at all.
bequanna 3y ago
So far, the results aren’t great. People claiming to be able to reliably detect LLM generated content have been found to be selling snake oil.
deleted 3y ago
[deleted]

OpenAI are already attempting, they have a free classifier you can try, but results aren't all that great: "Our classifier is not fully reliable. In our evaluations on a “challenge set” of English texts, our classifier correctly identifies 26% of AI-written text (true positives) as “likely AI-written,” while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Our classifier’s reliability typically improves as the length of the input text increases." https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-...

none_to_remain 3y ago
But they don't need a fancy AI classifier, they already have the text that ChatGPT outputs
- thrtythreeforty 3y ago
  They'd be absolute fools not to be keeping a copy of everything it's ever said. How much text has it generated, ever? A couple of dozen terabytes? Maybe?
  - johntb86 3y ago
    According to https://help.openai.com/en/articles/7730893-data-controls-faq https://help.openai.com/en/articles/7730893-data-controls-fa... if you turn off history the conversation will be permanently deleted after 30 days.
ghaff 3y ago
Assuming you're actually going to do anything with the results, basically any meaningful number of false positives is a huge problem. If using an LLM (or appearing to have done so) to at least help write a college essay is considered plagiarism, that can easily be grounds for expulsion.
George83728 3y ago
> while incorrectly labeling human-written text as AI-written 9% of the time (false positives). Utterly useless for teachers. You can't go around accusing 1 in 11 innocent students of plagiarism, and over the course of a semester you'd have virtually every student get falsely accused at least once. They may as well have not released this tool at all.

bequanna 3y ago

So far, the results aren’t great. People claiming to be able to reliably detect LLM generated content have been found to be selling snake oil.

deleted 3y ago

[deleted]