5 ms·
So, we just need a meta-review to review the reviews. At a cost, of course. And in order to keep that honest, we need a meta-meta-review...
by heikkilevanto 3y ago
So, we just need a meta-review to review the reviews. At a cost, of course. And in order to keep that honest, we need a meta-meta-review...
- notatoad 3y agowe need twitter community notes for science
- currymj 3y agomany CS conferences have something literally called a "meta-review" and then there are further senior people who read and oversee the meta-reviews. it stops there though.
- abhgh 3y agoUnfortunately the state of meta-review is similar to that of reviews. Rarely delves deeper, mostly acts as a summarizer for the independent reviews.
- MichaelZuo 3y agoWho picks the meta reviewers?
- bumby 3y agoOr possibly just a way to review the reviewers. This opens itself up to competitor bias, though, so it would need to be thought out in a way to minimize that.
- rhelz 3y agochuckle recall, a paper is given to 3 or 4 reviewers. No need for hierarchies of reviewers; it’s more like a jury; if all the reviewers more or less come to the same conclusion we can have a high confidence that the decision is the correct one. Under the proposed plan, if one of the reviewers gave a review which was radically different, or otherwise obviously slap-dash job, payment could be withheld and another reviewer commissioned.