6 ms·

I think about this problem a lot because the whole biomedical publishing industry is so backwards. One of the reasons why scientists are currently dependent on

I think about this problem a lot because the whole biomedical publishing industry is so backwards. One of the reasons why scientists are currently dependent on the journal industry is because the journals control peer review. Unfortunately, instead of merely reviewing manuscripts for correctness, the journals link peer review to "impact," which is a prospective measure of importance (i.e. whether they are likely to get many citations). Every journal you submit your manuscript to will put it through a new round of peer review looking for both correctness and impact. Massive amounts of time are wasted by working scientists and journal editors attempting to prospectively "grade" manuscripts on how important they will be (along with checking the correctness of the manuscript). Upon rejection for either correctness or impact, authors resubmit their manuscript to a lower tier journal (+/- revision) and wait for the next round of reviews. Rinse/repeat. Eventually, assuming the paper is not awful, it will be published. How could we do it differently? One idea is to disentangle impact from correctness and eliminate the journals from peer review. Scientists already review each others manuscripts pro-bono, so here's a proposal: Scientists submit manuscripts to bioRxiv as "pre-prints" for a nominal fee (say $1000) and request peer review. An editor selects 3-4 potential reviewers and reviewers are paid a nominal fee (~$150) to perform the review. The money paid by authors covers the reviewer fees and editors salaries. After peer review, authors can decide whether they want to submit their articles for additional peer review and fancy publication in a journal. Impact is thereafter judged only by how many citations a paper gets, not which glossy magazine it gets published in.

deleted 9y ago
[deleted]
currymj 9y ago
that is more or less the PLoS One or Scientific Reports model, right? you pay the fee, it gets peer reviewed for correctness but not impact, and then published open access.
- rgejman 9y ago
  Yes to PLoS One. I think Scientific Reports (as a NPG journal) still tries to be somewhat selective, but I'm not sure. My proposal is a bit different because it establishes a preliminary and public review process that is separate from "publication." You cannot take your PLoS One article and now get it published at Nature or Science. But you could do that if your paper was "reviewed" at bioRxiv, meaningfully improved and corrected, and then submitted for publication elsewhere.
new299 9y ago
It's true, but the review process (while not the publisher) does add value. It's really useful to highlight the gaps in an authors work and suggest additional analysis they could do to support the work and make it more useful. Or even highlight areas where the work is not reproducible (algorithms not sufficiently well described, data not available etc).
- rgejman 9y ago
  Agreed. That's why I'm arguing that peer reviews should be handled separately from publication or any consideration of impact.
Fomite 9y ago
You and one of the child posts are both putting "Pay the Fee" before "Peer review and a publication decision". In my experience, anyone asking for money up front pre-review is running a predatory journal.
- rgejman 9y ago
  The fee is designed to offset the real cost of (1) a person making an informed decision about what reviewers to send the manuscript to and (b) the real cost of a reviewer spending 3-4 hours on the manuscript. That ought to be an upfront cost and the results posted publicly.
  - Fomite 9y ago
    The problem with that is two-fold: 1. You've turned peer review into a service - which means I am going to start expecting things for my money. I occasionally get bad or unhelpful reviewers - do I have the right to complain? 2. Unless you can supplant the prestige of journals, who will undoubtedly have their own system, all you've actually done is added a $1000 tax to pre-print services. Beyond that, I was mostly addressing the perception issue - "Asks for money upfront", rightly or not, is already cemented in academic culture as a place you should not send your work to. That's an uphill battle.
    - rgejman 9y ago
      1. Agreed this is a problem, but it's not unsolvable. A pilot project of peer review as a service could ferret out these issues and come up with solutions. Here's a couple of ideas: (a) let authors give feedback on reviews for editors to consider when assigning this reviewer to future manuscripts, (b) require public disclosure of who reviewed the paper (this presents other problems, but should at least be considered), (c) editors can bring on 4th or 5th reviewers as needed if authors/editors/reviewers feel it is warranted (after seeing all 3 initial reviews) 2. Time is worth more than money. Remember it's not only the PI but also the postdocs and grad students who are not productive while they work on review/resubmission related work. I bet many labs would be glad to pay this amount of money for a single review process. The goal would be for single-review-per-manuscript to be able to replace journal reviews in most cases. Sure, some top flight journals may require additional review--and that's fine--but those should eventually be outliers, not the norm. I don't agree about the perception issue for a reputable source which is providing a tangible product. The authors end up with a peer reviewed manuscript that is citable. Call it biorXivGold or something like that and you're done. ^^ This red team/blue team (peer review) on this idea is a microcosm of the best part of science :)

deleted 9y ago

[deleted]

currymj 9y ago

that is more or less the PLoS One or Scientific Reports model, right? you pay the fee, it gets peer reviewed for correctness but not impact, and then published open access.

rgejman 9y ago
Yes to PLoS One. I think Scientific Reports (as a NPG journal) still tries to be somewhat selective, but I'm not sure. My proposal is a bit different because it establishes a preliminary and public review process that is separate from "publication." You cannot take your PLoS One article and now get it published at Nature or Science. But you could do that if your paper was "reviewed" at bioRxiv, meaningfully improved and corrected, and then submitted for publication elsewhere.

new299 9y ago

It's true, but the review process (while not the publisher) does add value. It's really useful to highlight the gaps in an authors work and suggest additional analysis they could do to support the work and make it more useful. Or even highlight areas where the work is not reproducible (algorithms not sufficiently well described, data not available etc).

rgejman 9y ago
Agreed. That's why I'm arguing that peer reviews should be handled separately from publication or any consideration of impact.

Fomite 9y ago

You and one of the child posts are both putting "Pay the Fee" before "Peer review and a publication decision". In my experience, anyone asking for money up front pre-review is running a predatory journal.

rgejman 9y ago
The fee is designed to offset the real cost of (1) a person making an informed decision about what reviewers to send the manuscript to and (b) the real cost of a reviewer spending 3-4 hours on the manuscript. That ought to be an upfront cost and the results posted publicly.
- Fomite 9y ago
  The problem with that is two-fold: 1. You've turned peer review into a service - which means I am going to start expecting things for my money. I occasionally get bad or unhelpful reviewers - do I have the right to complain? 2. Unless you can supplant the prestige of journals, who will undoubtedly have their own system, all you've actually done is added a $1000 tax to pre-print services. Beyond that, I was mostly addressing the perception issue - "Asks for money upfront", rightly or not, is already cemented in academic culture as a place you should not send your work to. That's an uphill battle.
  - rgejman 9y ago
    1. Agreed this is a problem, but it's not unsolvable. A pilot project of peer review as a service could ferret out these issues and come up with solutions. Here's a couple of ideas: (a) let authors give feedback on reviews for editors to consider when assigning this reviewer to future manuscripts, (b) require public disclosure of who reviewed the paper (this presents other problems, but should at least be considered), (c) editors can bring on 4th or 5th reviewers as needed if authors/editors/reviewers feel it is warranted (after seeing all 3 initial reviews) 2. Time is worth more than money. Remember it's not only the PI but also the postdocs and grad students who are not productive while they work on review/resubmission related work. I bet many labs would be glad to pay this amount of money for a single review process. The goal would be for single-review-per-manuscript to be able to replace journal reviews in most cases. Sure, some top flight journals may require additional review--and that's fine--but those should eventually be outliers, not the norm. I don't agree about the perception issue for a reputable source which is providing a tangible product. The authors end up with a peer reviewed manuscript that is citable. Call it biorXivGold or something like that and you're done. ^^ This red team/blue team (peer review) on this idea is a microcosm of the best part of science :)