4 ms·

I'm a PhD and researcher who has worked in various fields, including at a national lab. I think AI systems like this could greatly help with peer review, espec

I'm a PhD and researcher who has worked in various fields, including at a national lab. I think AI systems like this could greatly help with peer review, especially as a first check before submitting a manuscript to a journal. That said, this particular system appears to focus on the wrong issues with peer review, in my opinion. I'll ignore the fact that an AI system is not a peer since another person already brought that up [1]. Even if this kind of system was a peer, the system appears to be checking superficial issues and not the deeper issues that many peer reviewer/referrers care about. I'll also ignore any security risks (other posts discuss that too). A previous advisor of mine said that a good peer review needs to think about one major/deep question when reviewing a manuscript: Does the manuscript present any novel theories, novel experiments, or novel simulations; or does it serve as a useful literature review? Papers with more novelty are inherently more publishable. This system does not address this major question and focuses on superficial aspects like writing quality, as if peer review is mere distributed editing and not something deeper. It is possible for even a well-written manuscript to lack any novelty, and novelty is what makes it worthy of publication. Moreover, many manuscripts have at best superficial literature reviews that name drop important papers and often mischaracterize their importance. It takes deep expertise in a subject to see how a work is novel and fits into the larger picture of a given field. This system does nothing to aid in that. Does it help identify what parts of a paper you should emphasize to prove its novelty? That is, does it help you find the "holes" in the field that need patching? Does it help show what parts of your literature review are lacking? A lot of peer review is kinda performative, but if we are going to create automated systems to help with peer review, I would like them to focus on the most important task of peer review: assessing the novelty of the work. (I will note that I have not tried out this particular system myself. I am basing my comments here on the documentation I looked at on GitHub and the information in this thread.) [1] https://news.ycombinator.com/item?id=44144672 https://news.ycombinator.com/item?id=44144672

yusina 1y ago
I would say the exact opposite. Let the machine do the easy/simple/boring stuff. Let the human peer reviewer do the big question. That's what the human is good at and excited about and it's what the machine will not be good at. The question is a philosophical one: Is this a good idea? Is it relevant? Is it important? This is highly subjective and needs folks in the field to build consensus about. Back in my PhD days, I'd have loved if a machine could have taken care of the simple stuff so humans could focus entirely on the big questions. (A machine could point to similar work though.)
- atrettel 1y ago
  You raise a good point overall. I was just trying to respond to the idea of it replacing a human entirely, as if the authors submit it to the system and a journal editor has to made the decision to publish it or not. I would love to focus more on the big picture stuff, but in my experience most peer reviews amount to "Could you phrase this different?" rather than "Is this a good idea?". I think the latter is a much better better question to ask.
- NoahZuniga 1y ago
  It makes sense to have the AI do the boring stuff, but don't frame it as a peer reviewer, because that's not what it is.
  - rjakob 1y ago
    noted.
rjakob 1y ago
Thanks for the thoughtful feedback. That’s very helpful. We didn’t think too deeply about the term “AI peer reviewer” and didn’t mean to imply it’s equivalent to human peer review. Based on your comments, we’ll stick to using “AI reviewer” going forward. Regarding security: there is an open-source version for those who want full control. The free cloud version is mainly for convenience and faster iteration. We don’t store manuscript files longer than necessary to generate feedback (https://www.rigorous.company/privacy https://www.rigorous.company/privacy), and we have no intention of using manuscripts for anything beyond testing the AI reviewer. On novelty. totally agree it’s a core part of good peer review. The current version actually includes agents evaluating originality, contribution, impact, and significance. It’s still v1 of course but we want to improve it. We'd actually love for critical thinkers like you to help shape it. If you're open to testing it with a preprint and sharing your thoughts on the feedback, that would be extremely valuable to us. Thanks again for engaging, we really appreciate it.
- Not4Hire 1y ago
  [flagged]
- atrettel 1y ago
  No worries, I appreciate that you took the time to read and respond! When I first read "Originality and Contribution" at [1], I actually assumed it was a plagiarism check. It did not occur to me until now that you were referring to novelty with that. Similarly, I assumed "Impact and Significance" referred to more about whether the subject was appropriate for a given journal or not (would readers of this journal find this information significant/relevant/impactful or should it be published elsewhere?). That's a question that many journals do ask of referees, independent of overall novelty, but I see how you mean a different aspect of novelty instead. I'm not opposed to testing your system with a manuscript of my own, but currently the one manuscript that I have approaching publication is still in the internal review stage at my national lab, and I don't know precisely when it will be ready for external submission. But I'll keep it in mind whenever it passes all of the internal checks. [1] https://github.com/robertjakob/rigorous/blob/main/Agent1_Peer_Review/README.md https://github.com/robertjakob/rigorous/blob/main/Agent1_Pee...
SubiculumCode 1y ago
My view, and how I conduct my peer reviews, is that I do not care to make decisions of whether a question is important/interesting or not. I feel like my job is to judge the paper on rigor, and whether it fails (purposely or from ignorance) to address/acknowledge the relevant literature.
- rjakob 1y ago
  We also provide feedback on rigor across 7 different categories: https://github.com/robertjakob/rigorous/tree/main/Agent1_Peer_Review https://github.com/robertjakob/rigorous/tree/main/Agent1_Pee...
godelski 1y ago
I'm also a PhD[0] and researcher who has worked in various fields, including national labs too (You DOE?) I mostly share the same sentiment, and I see a similar issue with the product. The current system is not in its current poor state due to lack of reviewers, it is due to lack of quality reviewers and arbitrary notions of "good enough for this venue." So I wanted to express a difference of opinion about what peer review should be (I think you'll likely agree). I don't think we are doing the scientific community any service by doing our current Conference/Journal based "peer review". The truth is that you cannot verify a paper by reading it. You can falsify it, but even that is difficult. The ability to determine novelty and utility is also a crapshoot, where we have a long history illustrating how bad we are at this. Several Nobel prize worthy works have been rejected multiple times due to "obviousness", "lack of novelty", and "clearly wrong." All three apply to the paper that led to the 2001 Nobel Prize in Economics[1]! The truth of the matter is that peer review is done in the lab. It is done through replication, reproduction, and the further development of ideas. What we saw around LK-99[2] was more quality and impactful peer review than what any reader for a venue could provide. The impact existed long before any of those works were published in venues. I think this came down to forgetting the purpose of journals. They were there when we didn't have tools like ArXiV, OpenReview, or even GitHub. Journals were primarily focused on solving the logistic problem of distribution. So I consider all those technical works, "preprints", and blog posts around LK-99 replications as much of a publication as anything else. The point is that we are communicating with our peers. There's always been prestige around certain venues, but primarily people did not publish to them. The other venues checked for plagiarism, factual errors, and any obvious errors. Otherwise, they continued with publication. This silly notion of acceptance rates just creates a positive feedback loop which is overburdening the system (highly apparent in ML conferences). The notions of novelty and impact are highly noisy (as demonstrated in multiple NeurIPS studies and elsewhere), making the process far more random than acceptable. I don't think this is all that surprising. It is quite easy to poke flaws in any work you come across. It does not take a genius to figure out limitations of works (often they're explicitly stated!). The result of this is obvious, and is what most researchers end up doing: resubmit elsewhere and try your luck again. Maybe the papers are improved, maybe they aren't, mostly the latter. The only thing this accomplishes is an exponentially increasing number of paper submissions and slowing down of research progress as we spend time reformatting and resubmitting which should instead be spent researching. The distribution of quality review comments seems to have high variance, but I can say that early in my PhD they mainly resulted in me making my works worse as I chased their comments rather than just trying to re-roll and move on. In this sense, I don't think there's a "lack of reviewer" problem, so much as we have an acceptance threshold problem with an arbitrary metric. I think we should check for critical errors, check for plagiarism, and then just make sure the work is properly communicated. The rest is far more open to interpretation and not even us experts are that good at it. [0] Well my defense is in a week... [1] https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons [2] https://en.wikipedia.org/wiki/LK-99 https://en.wikipedia.org/wiki/LK-99

yusina 1y ago

I would say the exact opposite. Let the machine do the easy/simple/boring stuff. Let the human peer reviewer do the big question. That's what the human is good at and excited about and it's what the machine will not be good at. The question is a philosophical one: Is this a good idea? Is it relevant? Is it important? This is highly subjective and needs folks in the field to build consensus about. Back in my PhD days, I'd have loved if a machine could have taken care of the simple stuff so humans could focus entirely on the big questions. (A machine could point to similar work though.)

atrettel 1y ago
You raise a good point overall. I was just trying to respond to the idea of it replacing a human entirely, as if the authors submit it to the system and a journal editor has to made the decision to publish it or not. I would love to focus more on the big picture stuff, but in my experience most peer reviews amount to "Could you phrase this different?" rather than "Is this a good idea?". I think the latter is a much better better question to ask.
NoahZuniga 1y ago
It makes sense to have the AI do the boring stuff, but don't frame it as a peer reviewer, because that's not what it is.
- rjakob 1y ago
  noted.

rjakob 1y ago

Thanks for the thoughtful feedback. That’s very helpful. We didn’t think too deeply about the term “AI peer reviewer” and didn’t mean to imply it’s equivalent to human peer review. Based on your comments, we’ll stick to using “AI reviewer” going forward. Regarding security: there is an open-source version for those who want full control. The free cloud version is mainly for convenience and faster iteration. We don’t store manuscript files longer than necessary to generate feedback (https://www.rigorous.company/privacy https://www.rigorous.company/privacy), and we have no intention of using manuscripts for anything beyond testing the AI reviewer. On novelty. totally agree it’s a core part of good peer review. The current version actually includes agents evaluating originality, contribution, impact, and significance. It’s still v1 of course but we want to improve it. We'd actually love for critical thinkers like you to help shape it. If you're open to testing it with a preprint and sharing your thoughts on the feedback, that would be extremely valuable to us. Thanks again for engaging, we really appreciate it.

Not4Hire 1y ago
[flagged]
atrettel 1y ago
No worries, I appreciate that you took the time to read and respond! When I first read "Originality and Contribution" at [1], I actually assumed it was a plagiarism check. It did not occur to me until now that you were referring to novelty with that. Similarly, I assumed "Impact and Significance" referred to more about whether the subject was appropriate for a given journal or not (would readers of this journal find this information significant/relevant/impactful or should it be published elsewhere?). That's a question that many journals do ask of referees, independent of overall novelty, but I see how you mean a different aspect of novelty instead. I'm not opposed to testing your system with a manuscript of my own, but currently the one manuscript that I have approaching publication is still in the internal review stage at my national lab, and I don't know precisely when it will be ready for external submission. But I'll keep it in mind whenever it passes all of the internal checks. [1] https://github.com/robertjakob/rigorous/blob/main/Agent1_Peer_Review/README.md https://github.com/robertjakob/rigorous/blob/main/Agent1_Pee...

SubiculumCode 1y ago

My view, and how I conduct my peer reviews, is that I do not care to make decisions of whether a question is important/interesting or not. I feel like my job is to judge the paper on rigor, and whether it fails (purposely or from ignorance) to address/acknowledge the relevant literature.

rjakob 1y ago
We also provide feedback on rigor across 7 different categories: https://github.com/robertjakob/rigorous/tree/main/Agent1_Peer_Review https://github.com/robertjakob/rigorous/tree/main/Agent1_Pee...

godelski 1y ago

I'm also a PhD[0] and researcher who has worked in various fields, including national labs too (You DOE?) I mostly share the same sentiment, and I see a similar issue with the product. The current system is not in its current poor state due to lack of reviewers, it is due to lack of quality reviewers and arbitrary notions of "good enough for this venue." So I wanted to express a difference of opinion about what peer review should be (I think you'll likely agree). I don't think we are doing the scientific community any service by doing our current Conference/Journal based "peer review". The truth is that you cannot verify a paper by reading it. You can falsify it, but even that is difficult. The ability to determine novelty and utility is also a crapshoot, where we have a long history illustrating how bad we are at this. Several Nobel prize worthy works have been rejected multiple times due to "obviousness", "lack of novelty", and "clearly wrong." All three apply to the paper that led to the 2001 Nobel Prize in Economics[1]! The truth of the matter is that peer review is done in the lab. It is done through replication, reproduction, and the further development of ideas. What we saw around LK-99[2] was more quality and impactful peer review than what any reader for a venue could provide. The impact existed long before any of those works were published in venues. I think this came down to forgetting the purpose of journals. They were there when we didn't have tools like ArXiV, OpenReview, or even GitHub. Journals were primarily focused on solving the logistic problem of distribution. So I consider all those technical works, "preprints", and blog posts around LK-99 replications as much of a publication as anything else. The point is that we are communicating with our peers. There's always been prestige around certain venues, but primarily people did not publish to them. The other venues checked for plagiarism, factual errors, and any obvious errors. Otherwise, they continued with publication. This silly notion of acceptance rates just creates a positive feedback loop which is overburdening the system (highly apparent in ML conferences). The notions of novelty and impact are highly noisy (as demonstrated in multiple NeurIPS studies and elsewhere), making the process far more random than acceptable. I don't think this is all that surprising. It is quite easy to poke flaws in any work you come across. It does not take a genius to figure out limitations of works (often they're explicitly stated!). The result of this is obvious, and is what most researchers end up doing: resubmit elsewhere and try your luck again. Maybe the papers are improved, maybe they aren't, mostly the latter. The only thing this accomplishes is an exponentially increasing number of paper submissions and slowing down of research progress as we spend time reformatting and resubmitting which should instead be spent researching. The distribution of quality review comments seems to have high variance, but I can say that early in my PhD they mainly resulted in me making my works worse as I chased their comments rather than just trying to re-roll and move on. In this sense, I don't think there's a "lack of reviewer" problem, so much as we have an acceptance threshold problem with an arbitrary metric. I think we should check for critical errors, check for plagiarism, and then just make sure the work is properly communicated. The rest is far more open to interpretation and not even us experts are that good at it. [0] Well my defense is in a week... [1] https://en.wikipedia.org/wiki/The_Market_for_Lemons https://en.wikipedia.org/wiki/The_Market_for_Lemons [2] https://en.wikipedia.org/wiki/LK-99 https://en.wikipedia.org/wiki/LK-99