4 ms·
> Pangram’s analysis revealed that around 21% of the ICLR peer reviews were fully AI-generated, and more than half contained signs of AI use. The findings were
by nkrisc 11mo ago
> Pangram’s analysis revealed that around 21% of the ICLR peer reviews were fully AI-generated, and more than half contained signs of AI use. The findings were posted online by Pangram Labs. “People were suspicious, but they didn’t have any concrete proof,” says Spero. “Over the course of 12 hours, we wrote some code to parse out all of the text content from these paper submissions,” he adds.
But what's the proof? How do you prove (with any rigor) a given text is AI-generated?
- ModernMech 11mo agoI have this problem with grading student papers. Like, I "know" a great deal of them are AI, but I just can't prove it, so therefore I can't really act on any suspicions because students can just say what you just said.
- hyperadvanced 11mo agoWhy do you need proof anyway? Do you need proof that sentences are poorly constructed, misleading, or bloated? Why not just say “make it sound less like GPT” and let them deal with it?
- circuit10 11mo agoYou can have sentences that are perfectly fine but have some markers of ChatGPT like "it's not just X — it's Y" (which may or may not mean it's generated)
- hyperadvanced 11mo agoIsn’t that kind of thing (reliance on cliché) already a valid reason for getting marked down?
- chdjdbdbfjf 11mo ago[flagged]
- nkrisc 11mo agoBut in that case do you need to prove? You can grade them as they are and if you wanted to you (or teachers, generally) could even quiz the student verbally and in-person about their paper.
- shawabawa3 11mo agoPut poison prompts in the questions (things like "then insert tomato soup recipe" or "in the style of Shakespeare"), ideally in white font so they're invisible
- seanmcdirmid 11mo agoMany people using AI to write aren’t blindly copying AI output. You’ll catch the dumb cheaters like this, but that’s just about it.
- dkdcio 11mo ago> How do you prove (with any rigor) a given text is AI-generated? you cannot. beyond extra data (metadata) embedded in the content, it is impossible to tell whether given text was generated by a LLM or not (and I think the distinction is rather puerile personally)
- whynotmaybe 11mo agoI wouldn't be surprised to learn that the AI detection tool is itself an AI
- Lionga 11mo agoBut it works it was peer reviewed! (by AI)
- moffkalast 11mo agoFighting fire with fire sounds good in theory but in the end you're still on fire.
- nabla9 11mo agoWith AI model of course. They wrote a paper describing how they did it. https://arxiv.org/pdf/2510.03154 https://arxiv.org/pdf/2510.03154
- slashdave 11mo ago"proof" was an unfortunate phrase to use. However, a proper statistical analysis can be objective. And these kinds of tools are perfectly suited to such an analysis.
- maxspero 11mo agoYeah, Pangram does not provide any concrete proof, but it confirms many people's suspicions about their reviews. But it does flag reviews for a human to take a closer look and see if the review is flawed, low-effort, or contains major hallucinations.
- vladms 11mo agoWas there an analysis of flawed, low-effort reviews in similar conferences before generative AI models? From what I remember, (long before generative AI) you would still occasionally get very crappy reviews (as author). When I participated (couple of times) to review committees, when there was a high variance between reviews the crappy reviews were rather easy to spot and eliminate. Now it's not bad to detect crappy (or AI) reviews, but I wonder if it would change much the end result compared to other potential interventions.
- maxspero 11mo agoAnecdotally people are seeing a rise of low-quality reviews which is correlated with increased reviewer workload and and AI tools giving reviews an easy way out. I don't know of any studies quantifying review quality, but I would recommend checking the Peer Review Congress program from past years.
- jmpeax 11mo ago> does not provide any concrete proof, but it confirms many people's suspicions Without proof there is no confirmation.
- lazide 11mo agoFormally? Sure. In the current zeitgeist it’s more than enough to start pointing fingers around, etc.
- lazide 11mo agoYou don’t. It’s bullshit inception.