3 ms·
I don't disagree with the proposed idea of paying for review, but I would prefer also to have guardrails to ensure a good review. I would be willing to pay for
by bumby 3y ago
I don't disagree with the proposed idea of paying for review, but I would prefer also to have guardrails to ensure a good review. I would be willing to pay for a good review because it makes the paper/investigation better. But let's face it: under the current paradigm, there are also a lot of really bad reviews. It's one thing when it's apparent that a reviewer doesn't understand something because of a lack of clarity in the writing. But it's also extremely frustrating when it's obvious the reviewer hasn't even bothered to carefully read the manuscript.
Under a payment paradigm, we need mechanisms to limit the incentive to maximize throughput as a means of getting the most pay and instead maximize the review quality. I assume there'd be good ways to do that, but I don't know what those would be.
- heikkilevanto 3y agoSo, we just need a meta-review to review the reviews. At a cost, of course. And in order to keep that honest, we need a meta-meta-review...
- notatoad 3y agowe need twitter community notes for science
- currymj 3y agomany CS conferences have something literally called a "meta-review" and then there are further senior people who read and oversee the meta-reviews. it stops there though.
- abhgh 3y agoUnfortunately the state of meta-review is similar to that of reviews. Rarely delves deeper, mostly acts as a summarizer for the independent reviews.
- MichaelZuo 3y agoWho picks the meta reviewers?
- bumby 3y agoOr possibly just a way to review the reviewers. This opens itself up to competitor bias, though, so it would need to be thought out in a way to minimize that.
- rhelz 3y agochuckle recall, a paper is given to 3 or 4 reviewers. No need for hierarchies of reviewers; it’s more like a jury; if all the reviewers more or less come to the same conclusion we can have a high confidence that the decision is the correct one. Under the proposed plan, if one of the reviewers gave a review which was radically different, or otherwise obviously slap-dash job, payment could be withheld and another reviewer commissioned.