3 ms·
This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it. I tried a c
by nemomarx 2mo ago
This feels testable - you could go to fanfiction or similar sites with billions of words of writing from before 2016 or so and run them through it.
I tried a chapter just now and got human doing that, but I'm not invested enough to run a hundred samples today. But it sounds like it would be an alright way to audit it? I will confess I'm pretty skeptical you could ever eliminate false positives here though. I can often get an ai sense from some writing on my own but I doubt it would be better than 90% accurate, and "ai plus human editing" might screw with that anyway, stuff like that. I would have preferred we just never developed this kind of thing so I wouldn't have to guess.
- StilesCrisis 2mo agoIt's been done and showed up on HN recently. Older content was quite consistently marked as not-AI.
- philote 2mo agoThat still might work better with older texts. As AI-generated text gets more prevalent, I'm guessing people will start subconsciously adopting AI writing styles.
- nemomarx 2mo agoYeah, that's one of my questions. Everyone who talks to AI for too long seems to get worse at writing anyway, and humans mirror any form of conversation to some extent.
- subsistence234 2mo agoLLMS aren't the only thing that has changed over time in the way texts are written. if they used older texts as training data, to some extent pangram would just be an age classifier for writing style.