6 ms·
To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy. And detection was not done with some sn
by bonoboTP 7mo ago
To be clear, as the article says, these authors were offered a choice and agreed to be on the "no LLMs allowed" policy.
And detection was not done with some snake oil "AI detector" but by invisible prompt injection in the paper pdf, instructing LLMs to put TWO long phrases into the review. They then detected LLM use through checking if both phrases appear in the review.
This did not detect grammar checks and touchups of an independently written review. The phrases would only get included if the reviewer fed the pdf to the LLM in clear violation to their chosen policy.
> After a selection process, in which reviewers got to choose which policy they would like to operate under, they were assigned to either Policy A or Policy B. In the end, based on author demands and reviewer signups, the only reviewers who were assigned to Policy A (no LLMs) were those who explicitly selected “Policy A” or “I am okay with either [Policy] A or B.” To be clear, no reviewer who strongly preferred Policy B was assigned to Policy A.
- mikkupikku 7mo agoIn that case, I hope these frauds have been banned for life.
- hodgehog11 7mo agoI was thinking this too, but I don't believe this is the case, and I feel like it would not be a good idea either. Most of these people are likely students; this should be a learning moment, but I don't think it is yet grounds for their entire academic career to be crippled by being unable to publish in a top-tier ML venue.
- mikkupikku 7mo agoIf this is tolerated, it sends exactly the wrong kind of message. The students, if they are, should be banned for life. Let them serve as an example for myriads of future students, this will be a better outcome in the long run. This didn't trip for people who were merely bouncing ideas off a LLM, they caught people who copy and pasted straight from their LLM.
- linkregister 7mo agoIt's not a fully consensus view, but a majority of sociologists agree that high severity deterrence has limited effectiveness against crime. Instead, certainty of enforcement is the most salient factor.
- matkoniecz 7mo ago> Instead, certainty of enforcement is the most salient factor. hodgehog11 is proposing effectively no enforcement
- jona-f 7mo agoBut the mob wants their kick.
- crimsoneer 7mo agoYup, precisely this. Doing something bad is rarely a rational commitment and cost of benefits. Likelihood and celerity of getting caught seem to be the driving factors.
- _flux 7mo agoBut this method is now spent, as if someone is determined on keep using LLM, this should be pretty easy to overcome. I suppose though new methods could be devised, but it's not "certainty" that they will catch them.
- jacquesm 7mo agoThat's not true. People still pick up USB sticks from the street, people still fall for scam phone calls and people still click on links in mail. Just because a method was successful once does not mean it was 'burned', none of these people will be checking each and every future pdf or passing it through a cleaner before they will do the same thing all over again and others are going to be 'virgin' and won't even be warned because this is not going to be widely distributed in spite of us discussing it here. If anything you can take this as proof that this method is more or less guaranteed to work.
- 7mo ago
- noduerme 7mo ago2% would be on the very low end of the number of people who lie, get caught, and become repeat offenders anyway.
- anonymousDan 7mo agoML reviewing is a total joke. Why do you have noob students reviewing a conference paper.
- ancillary 7mo agoI've been an AC (the person who manages the reviewing process and translates reviews into accept/reject decisions) at ICML and similar conferences a few times. In my experience, grad students tend to be pretty good reviewers. They have more time, they are less jaded, and they are keener to do a good job. Senior people are more likely to have the deep and broad field knowledge to accurately place a paper's value, but they are also more likely to write a short shallow review and move on. I think the worst reviews I've seen have been from senior people.
- maleldil 7mo agoIt's usually not "noob" students. Big conferences require reviewers to have at least one (usually more) published paper in major venues. For students, this usually means they went through the process of being the first author on a few papers.
- bonoboTP 7mo agoBecause someone has to do it. Conference submissions have ballooned as the field itself has ballooned. Whats your suggestion?
- marcosdumay 7mo agoIt's better if nobody does it than to send it to the randomizer.
- bonoboTP 7mo agoOk but you need peer reviewed publications to graduate with a PhD. And if you retort that the whole academic system is obsolete, well, it still carries a lot of prestige and legitimacy that makes politicians interested in maintaining it, so it's not going anywhere soon.
- bumby 7mo agoWhat makes you think these are mostly students? I may have missed that in the methodology
- nurettin 7mo agoWhat terrible deeds have you done to outburst so harshly?
- quinndupont 7mo agoIt’s an unethical, false choice. The reviewers are not perfectly rational agents that do free work, they have real needs and desires. Shame on ICML for exploiting their desperation.
- jojomodding 7mo agoIs it? The reviewers could simply have chosen a different option in a form field. While I understand that they were "forced" to review under reciprocal review, they still had other choices where I don't see coercion happening and that could have avoided the outcome for them.
- qbit42 7mo agoBanned for life is a stretch but the actual response is completely fine. They can just resubmit to the next conference. Words mean something, if you promise to uphold a contract and break it, there are consequences. The reviewers were free to select the policy which allows LLM use.
- mikkupikku 7mo ago[flagged]
- notrealyme123 7mo agoIn many cases authors and reviewers are not the same. In your first two publications to such venues you are not allowed to review yourself and need someone else. I think consequences are well deserved, but hopefully not on the authors cost (if innocent).
- jvanderbot 7mo agoI'm not sure what experience anyone in this thread has with grad level research as a student/author, but I can assure you that heads roll over this kind of thing. A professor's career is built on reputation, and that reputation is as strong as their students' (who do much of the "work" such as it is). It comes down to the professor, but this can be a career-ending moment for those students and I'm quite confident there were some very uncomfortable discussions as a result of this.
- echelon 7mo ago[flagged]
- amoss 7mo agoThis comment doesn't seem to fit the discussion at all? The discussion is not about humans using LLLs to write papers. It is about humans who agreed not to use LLVM in reviewing papers, then did exactly that.
- Cthulhu_ 7mo agoThere's a lot of irony in a defensive comment being written based on misreading / inattentive reading of a post about reviewing papers (requiring attentive reading).
- bjourne 7mo agoIt might be that paper authors required others not to use LLMs for reviewing their work. Then, by the rule of reciprocity, they shouldn't use LLMs for reviewing others work. The article is unclear on whether this implied reciprocity rule was explicitly stated or not.
- ameliaquining 7mo agoIt was. More details here: https://icml.cc/Conferences/2026/LLM-Policy https://icml.cc/Conferences/2026/LLM-Policy In particular: "Any reviewer who is an author on a paper that requires Policy A must also be willing to follow Policy A."
- rat9988 7mo agoBanned from doing free work?