4 ms·
That's just writing. I frequently write like that. This insistence that certain stylistics patterns are "tell-tale" signs that an article was written by AI mak
by monooso 6mo ago
That's just writing. I frequently write like that.
This insistence that certain stylistics patterns are "tell-tale" signs that an article was written by AI makes no sense, particularly when you consider that whatever stylistic ticks an LLM may possess are a result of it being trained on human writing.
- gcr 6mo agoSee also: “I'm Kenyan. I Don't Write Like ChatGPT. ChatGPT Writes Like Me” by Marcus Olang', https://marcusolang.substack.com/p/im-kenyan-i-dont-write-like-chatgpt?triedRedirect=true https://marcusolang.substack.com/p/im-kenyan-i-dont-write-li... For what it’s worth, Pangram reports that Marcus’ article is 100% LLM-written: https://www.pangram.com/history/640288b9-e16b-4f76-a730-800080cb35f4 https://www.pangram.com/history/640288b9-e16b-4f76-a730-8000...
- croemer 6mo agoIn theory, wouldn't be too hard be to settle the question if whether he used ChatGPT to write it: get Olang to write a few paragraphs by hand, then have people judge (blindly) if it's the same style as the article. Which one sounds more like ChatGPT.
- embedding-shape 6mo agoThe times I've written articles, and those have gone through multiple rounds of reviews (by humans) with countless edits each time, before it ends up being published, I wonder if I'd pass that test in those cases. Initial drafts with my scattered thoughts usually are very different from the published end results, even without involving multiple reviewers and editors.
- jmalicki 6mo agoWhen people judge blindly, the are more likely to think the human is the AI and the AI is the human. 73% judged GPT 4.5 (edit: had incorrectly said 4o before)to be the human. https://arxiv.org/abs/2503.23674 https://arxiv.org/abs/2503.23674 Not only are people bad at judging this, but are directionally wrong.
- nothinkjustai 6mo agoThere is research showing the contrary that is far more convincing: > Our experiments show that annotators who frequently use LLMs for writing tasks excel at detecting AI-generated text, even without any specialized training or feedback. In fact, the majority vote among five such “expert” annotators misclassifies only 1 of 300 articles, significantly outperforming most commercial and open-source detectors we evaluated even in the presence of evasion tactics like paraphrasing and humanization. https://arxiv.org/html/2501.15654v2 https://arxiv.org/html/2501.15654v2
- croemer 6mo agoGreat find, I've submitted this preprint as a standalone item: https://news.ycombinator.com/item?id=47678270 https://news.ycombinator.com/item?id=47678270
- 360MustangScope 6mo agoI hate that I can’t write em dashes freely anymore without people accusing the writing of being AI generated. Even though they are perfect for usage in writing down thoughts and notes.
- croemer 6mo agoI have nothing against em dashes. As long as your writing is human, experienced readers will be able to tell it's human. Only less experienced ones will use all or nothing rules. Em dashes just increase the likelihood that the text was LLM generated. They aren't proof.
- brookst 6mo agoThat nuance is lost on the majority of anti-AI folks who’ve learned they get positive social reactions by declaring essentially everything to be AI written and condemnable. “An em dash… they’re a witch!”… “it’s not just X, it’s Y… they’re a witch!”
- andersonpico 6mo ago> anti-AI folks who’ve learned they get positive social reactions by declaring essentially everything to be AI written and condemnable. that's a strawman alright; all the comments complaining how they can't use their writing style without being ganged up on are positive karma from my angle, so I'm not sure the "positive social reactions" are really aligned with your imagination. Or does it only count when it aligns with your persecution complex?
- NetMageSCW 6mo agoYou have the same problem apparently. You think it’s okay to go witch hunting and accuse people with no real evidence.
- NetMageSCW 6mo agoEvidently there are no experienced readers who post AI accusations.
- croemer 6mo agoThese are just some of the good examples I found. My hunch that this is substantially LLM-generated is based on more than that. In my head it's like a Bayesian classifier, you look at all the sentences and judge whether each is more or less likely to be LLM vs human generated. Then you add prior information like that the author did the research using Claude - which increases the likelihood that they also use Claude for writing. Maybe your detector just isn't so sensitive (yet) or maybe I'm wrong but I have pretty high confidence at least 10% of sentences were LLM-generated. Yes, the stylistic patterns exist in human speech but RLHF has increased their frequency. Also, LLM writing has a certain monotonicity that human writing often lacks. Which is not surprising: the machine generates more or less the most likely text in an algorithmic manner. Humans don't. They wrote a few sentences, then get a coffee, sleep, write a few more. That creates more variety than an LLM can. Fun exercise: https://en.wikipedia.org/wiki/Wikipedia:AI_or_not_quiz https://en.wikipedia.org/wiki/Wikipedia:AI_or_not_quiz
- monooso 6mo agoHere's an alternative way of thinking about this... Someone probably expended a lot of time and effort planning, thinking about, and writing an interesting article, and then you stroll by and casually accuse them of being a bone idle cheat, with no supporting evidence other than your "sensitive detector" and a bunch of hand-wavy nonsense that adds up to naught.
- kenjackson 6mo agoWhile I agree with the sentiment, using AI to write the final draft of the article isn’t cheating. People may not like it, but it’s more a stylistic preference.
- oscaracso 6mo agoI am reminded of the Simpsons episode in which Principal Skinner tries to pass off the hamburgers from a near-by fast food restaurant for an old family recipe, 'steamed hams,' and his guest's probing into the kitchen mishaps is met with increasingly incredible explanations.
- butlike 6mo ago[dead]
- brookst 6mo agoI’m so glad the witch hunt has moved on to phrasing so I get less grief for my em dashes.
- nothinkjustai 6mo agoNo, it’s pretty obviously AI written. Not sure why you’re running so much interference for them…are you affiliated with this company?
- aionwikipedia 6mo agoThis is a myth. At least one study (Juzek/Ward) has shown that these stylistic patterns do not appear nearly as often either in well-known datasets of English language -- including datasets restricted to specific dialects of English. They don't even appear as often in text generated by raw language models. When they start showing up is after the model has undergone RLHF. Think of the Fermi paradox: if there are all these people who write like AI, then where are they? AI writing also tends to show these indicators over and over, consistently, over passages of text. It is very hard for humans, even if they are really familiar with AI writing, to be that consistent, and almost impossible for them to be that consistent for more than a sentence or two. Writing a long blog post by hand that is believably "AI-written" takes the amount of purposeful skill you'd need to forge an entire painting or ancient document. The problem is that people either look for the wrong things, look for obsolete things ("delve" is dead and modern LLMs have killed it), or extrapolate things from indicators that are extremely narrow and specific.