3 ms·
It is impossible to create detector for LLM generated writing. The LLM detector technology must beat the LLM at its own game and if it can we can easily create
by more_corn 3y ago
It is impossible to create detector for LLM generated writing. The LLM detector technology must beat the LLM at its own game and if it can we can easily create a better LLM. Anyone who claims they can do it is sorely mistaken or flat out lying.
- jmkd 3y agoBy the same logic I guess you could create one for previous generations of LLMs. "Looks like you need to update buddy...busted!" And across multiple industries a decent percentage of users will be using outdated versions...
- wongarsu 3y agoYour LLM detection method doesn't have to have a better understanding of human language than the LLM you're detecting, it just has to be able to identify patterns of speech typical of a particular LLM. Of course you have to repeat that work for each network and each significant update and finetune, but right now most people just use one of two ChatGPT versions. And of course you don't tell people how your LLM detector works, don't provide a public API, and preferably leave it ambiguous whether it even exists. That solves most issues with LLMs using your detection method to get better in future versions. That's counter to the business model of these LLM detection websites, but Google doesn't have a problem with keeping any potentially existing models inhouse and locked down.
- pixl97 3y agoReally all of it is bullshit if we don't know the true/false positive/negative rates. If you detect LLMs 100% of the time correctly, it is irrelevant if you also incorrectly detect human written text as LLM text 80% of the time. "Detect them all and let god sort them out" will eventually fail as a business plan.