3 ms·
This is plausible, but I can put some bounds on the scope of the conspiracy. OpenAI has already released their own AI text classifiers[0], and they are nowhere
by kmeisthax 3y ago
This is plausible, but I can put some bounds on the scope of the conspiracy. OpenAI has already released their own AI text classifiers[0], and they are nowhere near good enough to suggest the use of stylometric[1] watermarking. Furthermore, if such a watermark did exist, it would be detectable by anyone with a large enough corpus of known AI text. It probably would also be easier to detect (have lower false classification rates) than the unintentional stylo in ChatGPT output that most AI plagiarism detection is currently identifying.
I could see OpenAI archiving literally every line of text that their models generated, and then doing normal full-text search on their text corpus to remove their own output from it. But there's still one other flaw: the existence of other models.
GPT-3+ are not a monopoly. Google has BERT, Facebook has LLaMA, and the FOSS community has BLOOM, StableLM, and CerebrasGPT. While these models may or may not be GPT-4 quality, they will still be generating text without OpenAI's watermark, meaning that OpenAI won't know to filter for it. So even if they did plan to use watermarking as a way to invisibly scrape human content while polluting other companies' training sets, they wouldn't be able to succeed on such a plan.
[0] https://openai.com/blog/new-ai-classifier-for-indicating-ai-written-text https://openai.com/blog/new-ai-classifier-for-indicating-ai-...
[1] Stylometry is the process of fingerprinting word usage to determine authorship. This was used, for example, to unmask "Robert Galbraith" as the pen name of J.K. Rowling, before she made it too easy by just using her mystery novels as a way to rant about trans people and Twitter. More famously, the Unabomber was unmasked as Ted Kaczynski using the same means. Anonymity is a polite fiction and how you write is personally identifying information.