3 ms·
Yes, remove all the blinders. Of course, in practice, this has the problem that it is already hard to get reviewers, because nobody has time anymore. Removing
by auggierose 4y ago
Yes, remove all the blinders.
Of course, in practice, this has the problem that it is already hard to get reviewers, because nobody has time anymore. Removing the anonymity of reviewers leads to the judge becoming open to judgement themselves, and many would not like that. Furthermore, there might be severe repercussions if the reviewed is more powerful than the reviewer, especially in authoritarian societies. But maybe one should just exclude these societies from peer review, and let them do their own thing.
I think peer review has to become a market place. Let everyone choose for themselves which paper they would like to review, and there is both positive and negative credit for both reviewers and authors. The pool of reviewers of a paper shares 25% of the credit that the authors of the paper get. This way you are incentivised to review even outlier papers, because if they are successful you get a substantial amount of credit for them. How the credit is distributed among the reviewers should also incorporate a time factor, so that reviewers rushing to a paper that is already successful don't get nearly as much credit as a lone early reviewer.
- collyw 4y agoPeer review is done for free, maybe it needs to be compensated.
- auggierose 4y agoThat's why I use the term "credit". It needs to have real-world value of some sort. I don't think just paying people is the solution, that would just mean the richer you are, the more reviews you get, and the better your reviews are. You can argue they should be paid via a neutral source of money, but there is no such thing.
- suoduandao2 4y agoIf the problem of gaming the system could be gotten around, something very similar to internet comments could work - anyone can post a public review of anyone's work, in which case reputation would become the currency. Successfully criticizing someone with higher reputation than yourself would gain more status than praising them, but also come with more risk. It would also make it a lot less meaningful to get those 'not new ideas lol' comments that others on this thread have complained of. I quite like the fact that whatever someone says on the internet now, the first comment is usually the best-argued counterpoint. It seems like a dynamic that could be brought to science quite well. I may not have the time to parse an entire paper in a field unfamiliar to me, but if I come across it and the top comment is 'there's a math error in table 6 that throws paragraph 4 into doubt', I can certainly verify that for myself. That sort of 'peer' review would make science much more accessible to the interested layperson.
- zoolily 4y agoThere are efforts to give reviewers credit for their work, like Publons.
- currere 4y agoI can imagine some unintended consequences from this. If I'm a hard-going reviewer, people might choose to avoid me, and I'll be compensated less. Conversely, if I'm easy-going, people might find ways to get their papers to me. I suppose there can be some other layer of reviewer meta-review to account for this, much like the role 'acceptance rate' has come to have for journals and conferences. But, following Goodhart's law, even that has come to be gamed now that a meaning - a proxy for journal prestige or quality - has been placed on it.