4 ms·

IMHO (not a paper writer, but read a lot during my grad school years), the Genie is out of the bottle. The only way forward, as I see it, is using LLMs for revi

IMHO (not a paper writer, but read a lot during my grad school years), the Genie is out of the bottle. The only way forward, as I see it, is using LLMs for reviews also. Basically, filter all submitted papers with an LLM and ask it to summarize it, find the biggest weaknesses and main strong points, etc. that a human can then use to review the paper. Basically, LLM-as-a-reviewer . Personally, I would love to see a conference where people are explicitly encouraged to use LLMs for doing the work and writing the papers, and LLMs are used to review them too.

rsfern 7d ago
I think that’s a lot of risk of anchoring reviewer bias. I’d be more comfortable with a triaged review where the editor’s office uses models to score whether a human editor should evaluate a paper to potentially send out for review, then the editor makes their own assessment, and the reviewers continue to do their job unassisted
pwinnski 7d ago
The entire point of an academic paper is to add to the sum of human knowledge. How can an LLM trained on a subset of human knowledge possibly even begin to accurate evaluate such a paper? I trust an LLM to review that the language used in the paper is grammatically correct, but not to evaluate new information for accuracy.
- MikhailTal 7d ago
  This is very bad logic 1) Humans also are trained on a subset of human knowledge. 2)A lot of papers are just about experimenting something, and then applying simple stats. Eg empirical studies, around 1/3rd of published papers. Like, we tried this drug or did this experiment, from a sample size X here are the results. An expert is needed to maybe comment on the conclusion/hypothesis of the underlying suspected mechanism, but LLMs are still very useful on catching bad statistics or p hacking (so so common)
- zbyforgotp 6d ago
  Schmidthuber has an answer: https://arxiv.org/abs/0812.4360 https://arxiv.org/abs/0812.4360 For a more practical approach you need to use proxies: https://zby.github.io/commonplace/articles/what-an-automated-reviewer-should-measure/ https://zby.github.io/commonplace/articles/what-an-automated...
hyperjeff 7d ago
There should at least be a code of professional conduct where authors state the extent to which LLMs were used. (This would also help not wasting time by asking some “authors” about “their” paper.) Journals themselves should make policies about the extent to which they allow the use of LLMs. In some areas it might be considered more benign than in others.
- maleldil 7d ago
  There is. NLP conferences, like the ACL family (and the ARR) require you to disclose in the paper if you used LLMs for writing and coding, which ones and how. Whether every author is honest is a different matter.
dmos62 7d ago
Yes. I want reproducability, open-sourcing, accessibility, correctness, and most of all: usefulness. I don't care how it was written or reviewed, as long as some assurances regarding above things can be made, and I don't see why LLMs would get in the way of that. What can't be gotten rid of fast enough is the notion that having written something is meaningful on its own. Making something that looks right was a level above total novice: now it's the floor.
CJefferson 7d ago
Nope, I’ve tried this, it’s awful. For a start LLMs love LLM generated text, so you are boosting papers people never had any input in. Secondly, LLMs in my experience are good at small issues, but fail totally at the whole paper being obviously poorly constructed, or clearly fake.
maleldil 7d ago
This already happens. It's clear when a reviewer used an LLM, and it's very annoying for the authors that have to respond to what are usually low quality, superficial reviews.
- CrazyStat 7d ago
  To be fair, low quality superficial reviews were also not uncommon before LLMs…
- mlmonkey 7d ago
  Boy do I have news for you! :-D "low quality, superficial reviews" have always been around. Reviewing is most often an unpaid, thankless job and many times reviewers barely put in the effort.
  - __lm__ 6d ago
    There is still a difference. Previously, if the review was low quality and superficial, it was also quite short and easy to answer. Now I receive fully LLM-written reviews of 2-3 pages (or more) of superficial comments asking for tons of additional text and experiments that are almost always outside the scope of the work. Answering such a review requires a lot of work with absolutely no gain.
    - mlmonkey 6d ago
      I hear you. I just happened to submit a paper to Neurips this year, and the "reviewer" sent back so many questions, etc. that the character limit of the response could never accomodate my response. So I just addressed a few of the main points and called it a day. I am sure an LLM review could be made much shorter with some prompting.
      - maleldil 6d ago
        The character limit thing is real. Another problem is that even if you manage to address all of their points, they're not required to engage with that. They can just ask the LLM to write a basic response back and say they won't change the score.
kukkeliskuu 6d ago
Extend this beyond review. The most value we would get is from quality checking existing published papers.
zbyforgotp 6d ago
Here is my proposal (or actually our proposal - me and my agents): https://zby.github.io/commonplace/articles/what-an-automated-reviewer-should-measure/ https://zby.github.io/commonplace/articles/what-an-automated...
amdivia 6d ago
maybe but as a paper writer, the quality of reviews and reviewers have gone down because of LLM-as-a-reviewer too, because LLM reviews seem to regurgitate the limitations section of the paper, and are highly influenced by the way things are phrased in the paper rather than the actual substance. but I'm hopeful that some middle ground will be found in the future
- sgt101 6d ago
  I dunno man, I use LLM's to review things before I send them in and I often feel like I've been handcuffed to a chair, had a bright light shone in my face and had to account for all the mistakes I've made in my life. They can be brutal.
  - robwwilliams 6d ago
    Completely agree. I’ve had a Nature paper and a Nature Neuroscience paper published this year and the reviews from Opus were much more thorough and complete than those of the human reviewers. I still benefitted from the human reviewers, but now I consider an LLM review essential.

rsfern 7d ago

I think that’s a lot of risk of anchoring reviewer bias. I’d be more comfortable with a triaged review where the editor’s office uses models to score whether a human editor should evaluate a paper to potentially send out for review, then the editor makes their own assessment, and the reviewers continue to do their job unassisted

pwinnski 7d ago

The entire point of an academic paper is to add to the sum of human knowledge. How can an LLM trained on a subset of human knowledge possibly even begin to accurate evaluate such a paper? I trust an LLM to review that the language used in the paper is grammatically correct, but not to evaluate new information for accuracy.

MikhailTal 7d ago
This is very bad logic 1) Humans also are trained on a subset of human knowledge. 2)A lot of papers are just about experimenting something, and then applying simple stats. Eg empirical studies, around 1/3rd of published papers. Like, we tried this drug or did this experiment, from a sample size X here are the results. An expert is needed to maybe comment on the conclusion/hypothesis of the underlying suspected mechanism, but LLMs are still very useful on catching bad statistics or p hacking (so so common)
zbyforgotp 6d ago
Schmidthuber has an answer: https://arxiv.org/abs/0812.4360 https://arxiv.org/abs/0812.4360 For a more practical approach you need to use proxies: https://zby.github.io/commonplace/articles/what-an-automated-reviewer-should-measure/ https://zby.github.io/commonplace/articles/what-an-automated...

hyperjeff 7d ago

There should at least be a code of professional conduct where authors state the extent to which LLMs were used. (This would also help not wasting time by asking some “authors” about “their” paper.) Journals themselves should make policies about the extent to which they allow the use of LLMs. In some areas it might be considered more benign than in others.

maleldil 7d ago
There is. NLP conferences, like the ACL family (and the ARR) require you to disclose in the paper if you used LLMs for writing and coding, which ones and how. Whether every author is honest is a different matter.

dmos62 7d ago

Yes. I want reproducability, open-sourcing, accessibility, correctness, and most of all: usefulness. I don't care how it was written or reviewed, as long as some assurances regarding above things can be made, and I don't see why LLMs would get in the way of that. What can't be gotten rid of fast enough is the notion that having written something is meaningful on its own. Making something that looks right was a level above total novice: now it's the floor.

CJefferson 7d ago

Nope, I’ve tried this, it’s awful. For a start LLMs love LLM generated text, so you are boosting papers people never had any input in. Secondly, LLMs in my experience are good at small issues, but fail totally at the whole paper being obviously poorly constructed, or clearly fake.

maleldil 7d ago

This already happens. It's clear when a reviewer used an LLM, and it's very annoying for the authors that have to respond to what are usually low quality, superficial reviews.

CrazyStat 7d ago
To be fair, low quality superficial reviews were also not uncommon before LLMs…
mlmonkey 7d ago
Boy do I have news for you! :-D "low quality, superficial reviews" have always been around. Reviewing is most often an unpaid, thankless job and many times reviewers barely put in the effort.
- __lm__ 6d ago
  There is still a difference. Previously, if the review was low quality and superficial, it was also quite short and easy to answer. Now I receive fully LLM-written reviews of 2-3 pages (or more) of superficial comments asking for tons of additional text and experiments that are almost always outside the scope of the work. Answering such a review requires a lot of work with absolutely no gain.
  - mlmonkey 6d ago
    I hear you. I just happened to submit a paper to Neurips this year, and the "reviewer" sent back so many questions, etc. that the character limit of the response could never accomodate my response. So I just addressed a few of the main points and called it a day. I am sure an LLM review could be made much shorter with some prompting.
    - maleldil 6d ago
      The character limit thing is real. Another problem is that even if you manage to address all of their points, they're not required to engage with that. They can just ask the LLM to write a basic response back and say they won't change the score.

kukkeliskuu 6d ago

Extend this beyond review. The most value we would get is from quality checking existing published papers.

zbyforgotp 6d ago

Here is my proposal (or actually our proposal - me and my agents): https://zby.github.io/commonplace/articles/what-an-automated-reviewer-should-measure/ https://zby.github.io/commonplace/articles/what-an-automated...

amdivia 6d ago

maybe but as a paper writer, the quality of reviews and reviewers have gone down because of LLM-as-a-reviewer too, because LLM reviews seem to regurgitate the limitations section of the paper, and are highly influenced by the way things are phrased in the paper rather than the actual substance. but I'm hopeful that some middle ground will be found in the future

sgt101 6d ago
I dunno man, I use LLM's to review things before I send them in and I often feel like I've been handcuffed to a chair, had a bright light shone in my face and had to account for all the mistakes I've made in my life. They can be brutal.
- robwwilliams 6d ago
  Completely agree. I’ve had a Nature paper and a Nature Neuroscience paper published this year and the reviews from Opus were much more thorough and complete than those of the human reviewers. I still benefitted from the human reviewers, but now I consider an LLM review essential.