3 ms·
"proof" was an unfortunate phrase to use. However, a proper statistical analysis can be objective. And these kinds of tools are perfectly suited to such an anal
by slashdave 11mo ago
"proof" was an unfortunate phrase to use. However, a proper statistical analysis can be objective. And these kinds of tools are perfectly suited to such an analysis.
- maxspero 11mo agoYeah, Pangram does not provide any concrete proof, but it confirms many people's suspicions about their reviews. But it does flag reviews for a human to take a closer look and see if the review is flawed, low-effort, or contains major hallucinations.
- vladms 11mo agoWas there an analysis of flawed, low-effort reviews in similar conferences before generative AI models? From what I remember, (long before generative AI) you would still occasionally get very crappy reviews (as author). When I participated (couple of times) to review committees, when there was a high variance between reviews the crappy reviews were rather easy to spot and eliminate. Now it's not bad to detect crappy (or AI) reviews, but I wonder if it would change much the end result compared to other potential interventions.
- maxspero 11mo agoAnecdotally people are seeing a rise of low-quality reviews which is correlated with increased reviewer workload and and AI tools giving reviews an easy way out. I don't know of any studies quantifying review quality, but I would recommend checking the Peer Review Congress program from past years.
- jmpeax 11mo ago> does not provide any concrete proof, but it confirms many people's suspicions Without proof there is no confirmation.
- lazide 11mo agoFormally? Sure. In the current zeitgeist it’s more than enough to start pointing fingers around, etc.
- deleted 11mo ago[deleted]