3 ms·
I’m not saying whether I think you’re right or wrong, but AI text detectors don’t work and never will.
by devmor 2mo ago
I’m not saying whether I think you’re right or wrong, but AI text detectors don’t work and never will.
- richwater 2mo agoDoesn't stop charlatans from creating startups promising to make it work :)
- chis 2mo agoThis has been pretty well studied. Pangram is about as good at an expert human with a 98% detection rate and <2% false positive, for flagging AI generated text. https://arxiv.org/pdf/2501.15654 https://arxiv.org/pdf/2501.15654
- devmor 2mo agoI'm open to being convinced, but the study you've linked me is limited specifically to professionally written, proofread journalism from 8 publications over a span of a little over a single year. It's not exactly a wide breadth of varied text for such an all-encompassing task.
- chis 2mo agohttps://www.pangram.com/blog/third-party-pangram-evals https://www.pangram.com/blog/third-party-pangram-evals You can also just try it yourself I guess really what convinced me was how it perfectly agrees with my own judgement.
- devmor 2mo agoThe primary reason I am not convinced is because it (and several other tools) frequently classify my own writing as AI-generated. Especially my technical writeups. Now, I could just assume I'm an outlier within that small false-positive rate. But I find it far more likely that the efficacy of these tools is much lower than advertised and they rely on confirmation bias.