4 ms·
There are a few reasons why peer review is failing, but may not be obvious to outsiders. First, to be selected as a peer reviewers for any reputable publicatio
by fngs13 9y ago
There are a few reasons why peer review is failing, but may not be obvious to outsiders.
First, to be selected as a peer reviewers for any reputable publication, one needs to have a credible publication record in the field (e.g. 5-10 publication in the peer review literature in the same or a closely related field in the last 2-5 years).
Second, the handling editor needs to be aware of who your work and consider it relevant before being selected. Inherent in this step is some knowledge of where you trained, you collaborators/coauthors, your stature in the field, along with any other potential bias (e.g., age, sex, nationality. etc).
Third, if elected as one of three potential reviewers for a manuscript, you must be willing to volunteer two to four hours of your time to handle the task, usually within a few weeks of the invitation. Inherent in this step is also the potential interest level of the manuscript, you opinion of the authors and their past work, the quality of writing, the quality of the publication, your relationship to the editor and editorial board and any potential conflicts of interest that may arise because of your past and present work.
Fourth, peer-reviewers are volunteers. The work is not credited and certainly not 'credit-worthy" in an academic sense (it doesn't help in tenure or promotion decisions. The reason that anyone accepts a peer-review invitation is to gain some insights into what potential competitors/collaborators are working on, before the work is made public in the literature. Alternatively, it is done as a favor to an editor. Peer-reviewing is critical to ensure that the quality of science remains high, but every moment spent on the task is one that is not spent working on one's own publications, grants, proposals or other work for which they are either paid or must secure fund to remain competitive.
Fifth, the reward for doing a good job is receiving more invitation to do more peer reviews. Simply put, most editors are insufficiently aware of others working in the field beyond their own small network of colleagues, so they go back to those they know. This works until an editor has no friends or colleagues left and the run away from him/her at meetings.
Sixth, the sheer volume of submission into the scholarly literature has increased at a rate that is about twice the historical rate, largely drive by open access publishers and the demand for more publications/year/academic scientist (note that scientists in the private sector do not waste their time chasing after meaningless publications). In the last year or so, the number of published articles based on industry around 1.5 - 1.75 M. When you factor in the average rejection rate (around 40%), the need for three peer reviewers/article, the number of usable reviews/number of invitations (typically 8-10), your get an idea of the scope of the problem facing publishers.
Are there solutions. Yes. Automation, when applied in the right places, works. Tools are already available for publisher to prescreen manuscript for completeness and accuracy at the time of submission and to alleviate much of the grunt work that is pushed to peer-reviewer (because the work for free, remember?). There are also tools that are coming to market that help to identify credible peer-reviewers that editors may not be aware of because they are simply outside of their network of contacts. Those same tools can be used to ensure that the reviewers are selected to review manuscripts that are likely to be relevant to them, of interest, and reasonably well written 9or at least comprehensible). On the other hand, AI tools to screen manuscript that are describing new things are highly unlikely to work, because there will be nothing on which the algorithms would be based. This takes real intelligence and insight that comes with years of deep reading and work in a field. Even if a AI based peer review system could be developed for one field based on modeling after a human expert, it is unlikely that the technology could be readily generalized. Block chain? How would that work? What happens when there are conflicting opinions?
The other solution would be for journal to impose much stricter requirements on publishing (real quality, no just "good enough, which is the emerging trend) and to impose limits on each academic scientist (e.g. no more than 100 articles/life). Other possible solutions include increasing the cost of publication. To some extent this is already done with journals in the sciences, where prestige publications can demand a premium article processing fee. Also, many funding agencies and academic institutions discount articles published in lower tiered journals, so perhaps some level of natural selection might apply. The other point to consider is if an increasing portion of the research budget is being expended to publish in open access journals, we may eventually reach the point in which there is nothing left in the budget to actually pay for the salaries, stipends, and research and the peer review problem will disappear.
- GeorgeCarlin123 9y ago"It's a Big Club, and You Ain't in It" --George Carlin