4 ms·
The review process is noisy and subject to much social engineering, lobbying, editorial PR, and quid pro quo if also serving as reviewer. Reviews are inherentl
by robwwilliams 4y ago
The review process is noisy and subject to much social engineering, lobbying, editorial PR, and quid pro quo if also serving as reviewer.
Reviews are inherently noisy because there are usually no more than three (over-taxed, unpaid) reviewers. But papers are now often highly multidisciplinary. Luck of the draw applies with force at n = 3 or even n = 10.
Reviews DO improve quality of final product, especially in the better journals. But there are still huge gaps in reviewing due to the underlying assumptions about “good” science.
For example, the majority if experimental studies using mice and rats use only a single genotype or strain of animal and are therefore essentially n = 1. My inclination is to reject a great majority of such studies without serious review because they are just fancy case reports.
So if you lucked out and did NOT get me as a reviewer you may be good to go at Science, Nature, or Cell. If you did not luck out then you will have to fight with the editor-in-chief to overcome my fundamental dismissal of case reports in experimental biology.
I support the eLife direction. We have squandered 20 years already and the hegemony gets stronger. Eisen is a bold publication experimentalist.
- ethbr0 4y agoDo review comments get publicly shared under the previous model? I'm curious if a large part of this push-back is "I don't want to be held publicly responsible for my review comments..." (the over-taxed, unpaid ones) ... which leads inexorably back to the place publishers don't want to go -- reviewers deserve to be well-compensated, for doing a time-intensive, hard job well. And especially because they create the bulk of a curated journal's value.
- f6v 4y agoNature publishes review history for some. Also, F1000 follows this model, although not officially a journal.
- matthewdgreen 4y agoI don’t work in your area, but my observation is: you seem to have a strong opinion about the validity of certain experimental work. Moreover, your opinion does not appear to be broadly shared by your fellow reviewers, and (here I make a second presumption) also by many experimental scientists submitting work to these venues. I don’t know if your opinion is valid or incorrect - that is incidental to the discussion - but I sure wish that reviewers would agree on basic standards in advance, rather than surprising scientists with disagreements about them after researchers have done enormous amounts of work and spent public funds. The situation you’re describing (basically, having a debate about standards in the wrong place) seems very harmful.