3 ms·
One thing to note. They were quite conservative in their approach, so the only things that were rejected were from people who had agreed not to use an LLM and
by mijoharas 7mo ago
One thing to note.
They were quite conservative in their approach, so the only things that were rejected were from people who had agreed not to use an LLM and almost definitely did use an LLM (since they fed hidden watermarked instructions to the llm's).
This means the true number of people that used LLM's in their review (even in group A that had agreed not to) is likely higher.
Also worth noting, 10% of these authors used them in more than half of their reviews.
- grey-area 7mo agoYes for those in group B I'd suspect many were doing exactly what these cheaters in group A were doing - submitting the unaltered output of an LLM as their review.
- kombookcha 7mo agoThe rejection is based on the dishonesty of explicitly committing to standard A and then knowingly violating it, not on LLM use as such. I think that's pretty fair, considering that everyone could have just chosen B if they wanted to.
- grey-area 7mo agoSure, I'm just pointing out that the 2% headline figure is very conservative if not misleading as a far greater unknown number in group B will have done exactly the same (which I doubt ICML or those submitting papers actually want). This is probably a first step in clamping down on anyone doing this.
- kombookcha 7mo agoOh, I misread your post - that's fair!