3 ms·

> Peer review is a base level quality check Ideally this is true. At the worst though, you'll end up with a peer reviewer that gets pedantic or one who's a me

> Peer review is a base level quality check Ideally this is true. At the worst though, you'll end up with a peer reviewer that gets pedantic or one who's a member of a citation cartel [0] that demands that you cite his/her friends papers unnecessarily as a quid-pro-quo leading to publication. It's difficult to say how common this is in practice since academia is generally terrible about self-reflection, but I observed concerning evidence of this when I was still in science. In some fields at least, there's ample evidence [1] that peer review provides inadequate QC controls for methods sections, making it literally impossible to replicate results due to entire processing steps being omitted. This is why post-publication services such as PubPeer have become more and more relevant, since it has become obvious as time has elapsed how woefully inadequate the initial screening process is. [0] https://academia.stackexchange.com/questions/81261/reviewer-recommending-citation-of-certain-papers-of-specific-group-of-authors-ma https://academia.stackexchange.com/questions/81261/reviewer-... [1] https://www.ncbi.nlm.nih.gov/pubmed/22796459 https://www.ncbi.nlm.nih.gov/pubmed/22796459

Certhas 8y ago
It's good to keep in mind that reviewers do not make the decision. They recommend the decision. The Editor, who knows the identity of the reviewer, can see if a reviewer is asking for unreasonable citations. So while this definitely does happen (and to different degrees in different fields) there are some checks and balances, too.
- neuromantik8086 8y ago
  True, and your overall point (that it is better than the alternative; i.e. viXra) stands. It's really one of those things where ymmv depending on the field- I just happened to notice a somewhat disheartening tendency for those sorts of horse trading practices to creep in, which is ultimately one of the major reasons why I left science. I could be naive, but my impression was that there are almost too many degrees of freedom in the peer review process that could allow for this kind of chicanery, and that it would almost be preferable to have criteria for acceptance laid out in advance (i.e., exactly what steps, metrics, and citable articles were necessary) and use those the way an actuary would. In some ways I almost think that practices like preregistration serve as a better first-pass indicator of whether or not research is publishable.
  - Certhas 8y ago
    Completely agreed. I think the system is ripe for an overhaul, and has many problematic aspects. We'll see how it goes, after all it doesn't look too bad with the open access revolution. I personally would like more use of pre-registration (there is something about that on the front-page right now. But also I think deanonymzing peer review and publishing the reviews along with the papers could be very interesting. Obvious drawbacks as well of course but I think it has potentially massive upsides.

Certhas 8y ago

It's good to keep in mind that reviewers do not make the decision. They recommend the decision. The Editor, who knows the identity of the reviewer, can see if a reviewer is asking for unreasonable citations. So while this definitely does happen (and to different degrees in different fields) there are some checks and balances, too.

neuromantik8086 8y ago
True, and your overall point (that it is better than the alternative; i.e. viXra) stands. It's really one of those things where ymmv depending on the field- I just happened to notice a somewhat disheartening tendency for those sorts of horse trading practices to creep in, which is ultimately one of the major reasons why I left science. I could be naive, but my impression was that there are almost too many degrees of freedom in the peer review process that could allow for this kind of chicanery, and that it would almost be preferable to have criteria for acceptance laid out in advance (i.e., exactly what steps, metrics, and citable articles were necessary) and use those the way an actuary would. In some ways I almost think that practices like preregistration serve as a better first-pass indicator of whether or not research is publishable.
- Certhas 8y ago
  Completely agreed. I think the system is ripe for an overhaul, and has many problematic aspects. We'll see how it goes, after all it doesn't look too bad with the open access revolution. I personally would like more use of pre-registration (there is something about that on the front-page right now. But also I think deanonymzing peer review and publishing the reviews along with the papers could be very interesting. Obvious drawbacks as well of course but I think it has potentially massive upsides.