4 ms·

The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valuel

by scottfr 12y ago

The fundamental issue with the current peer review system is that most academic research is "valueless" to those involved in the system. I use the term "valueless" in a very careful and specific meaning. It does not mean the work is without merit or doesn't have large impact or benefits to society, rather it simply means the quality of the work has no value to the peer reviewers or to the journal itself. * The peer reviewers are donating their time and generally getting nothing in return from reading the work other than seeing what other developments are going in in the field. * The journal gets nothing from work itself. They are solely in the business of collecting author fees and fees from library. In this context, I think it is accurate to describe most work as "valueless" to those involved in the system. This naturally leads to sloppy peer review and publishing practices. What I would like to see is a system that makes all players have skin in the game for the quality of the work. How to do this? One way would be a new kind of journal where everything published has to be patentable. The journal would pay for the patenting costs and the split any potential licensing fees with the authors/funding institutions in some way. Peer reviewers would be paid and/or get a small cut of the patent. This would make everyone in the process have "skin in the game" in the value and quality of the work being published. I don't know if the economics of such a model would work, but I think it would be a fascinating experiment.

protonfish 12y ago
Whoa, you had a nice analysis and then went off the rails with the patenting idea. Maybe a simpler solution to having some skin in the game is to pay the reviewer and/or have a way to view a reviewer's history.
niels_olson 12y ago
Um, you're talking about patenting the Higgs boson, the structure of DNA, the speed of light. There is academic merit in being a reviewer. Being a reviewer for the New England Journal of Medicine would surely favor any tenure committee or grant application. Perhaps one could argue that's hard to perceive, but I think it's safe to say any PhD sees the value. The journals, on the other hand, have abided by very loose business practices with exorbatant margins enabled to a great degree by the perception of the academic honor system, for a long time. Their business models and methods are thankfully coming under much deserved scrutiny. Should researchers be honorable people? Yes. Will you always find dishonorable behavior in a set of 100 people, over a lifetime? Yes.
- danieltillett 12y ago
  Having been through this there is no value in being a reviewer when it comes to tenure or grants. There is value in being an editor, but being a reviewer is worth zero to your career.
Matumio 12y ago
I thought one of the main points of academia was that it isn't centered around making money.
GFK_of_xmaspast 12y ago
There is, or should be, more to academia than just the pursuit of money.
- scottfr 12y ago
  Of course. The primary goal of academia is the advancement of human knowledge. Unfortunately, the current ecosystem of journals is to a significant extent failing at that. Possibly introducing a clear [0] profit motive into that system would result in it working more efficiently and better serving academia's primary goal. [0] I say "clear" because a lot of the current system of journals is primarily driven by profit motives that are only slightly veiled. There are notable exceptions to this, e.g. PLoS, Journal of Statistical Software, etc...
  - untilHellbanned 12y ago
    The researchers need to profit in order for the incentives to truly align. It's like an incredibly poor man's spotify and we all know how that works for musicians.
czr80 12y ago
No. Peer reviewers get: early access to articles of interest, the feeling of contributing to the community they belong to, the ability to demonstrate their knowledge, the ability to reciprocate other people for reviewing their own articles, and so on. These are not financial rewards, but they are not less strong for that (arguably even stronger). Journals are heavily incentivized to maintain standards because otherwise good authors will stop submitting to them and librarians will stop subscribing (of course only the first incentive holds for OA journals). The fact that system sometimes fails is not proof that it is a bad system - all human systems have a failure rate.
untilHellbanned 12y ago
Onarbor, https://onarbor.com https://onarbor.com, is doing precisely what you describe. It's publishing and funding. I'm one of its creators so would be grateful if you'd be interested in talking more. tim@onarbor.com.
pcrh 12y ago
The standards required for patentability are quite different from scientific novelty or interest. Paying reviewers is not a bad idea, as journals that pay will both be more likely to attract quality reviewers, and it also creates a contract between the reviewer and the journal. Such a contract could include requirements for the standard of reviewing. If a paper is later found to have been poorly reviewed the journal can seek redress from the reviewers, further encouraging them to do their job properly.
dalke 12y ago
So if I have an idea that might be patentable, the journal will handle all the costs for me? Including maintenance fees and the cost of appeals if the patent is rejected? Sweet! Someone else takes the brunt of the $10,000+ in patent filing costs, and there's no downside for me! In other words, your proposed journal just managed to increase the cost of publication by over an order of magnitude.

by scottfr 12y ago

protonfish 12y ago

Whoa, you had a nice analysis and then went off the rails with the patenting idea. Maybe a simpler solution to having some skin in the game is to pay the reviewer and/or have a way to view a reviewer's history.

niels_olson 12y ago

Um, you're talking about patenting the Higgs boson, the structure of DNA, the speed of light. There is academic merit in being a reviewer. Being a reviewer for the New England Journal of Medicine would surely favor any tenure committee or grant application. Perhaps one could argue that's hard to perceive, but I think it's safe to say any PhD sees the value. The journals, on the other hand, have abided by very loose business practices with exorbatant margins enabled to a great degree by the perception of the academic honor system, for a long time. Their business models and methods are thankfully coming under much deserved scrutiny. Should researchers be honorable people? Yes. Will you always find dishonorable behavior in a set of 100 people, over a lifetime? Yes.

danieltillett 12y ago
Having been through this there is no value in being a reviewer when it comes to tenure or grants. There is value in being an editor, but being a reviewer is worth zero to your career.

Matumio 12y ago

I thought one of the main points of academia was that it isn't centered around making money.

GFK_of_xmaspast 12y ago

There is, or should be, more to academia than just the pursuit of money.

scottfr 12y ago
Of course. The primary goal of academia is the advancement of human knowledge. Unfortunately, the current ecosystem of journals is to a significant extent failing at that. Possibly introducing a clear [0] profit motive into that system would result in it working more efficiently and better serving academia's primary goal. [0] I say "clear" because a lot of the current system of journals is primarily driven by profit motives that are only slightly veiled. There are notable exceptions to this, e.g. PLoS, Journal of Statistical Software, etc...
- untilHellbanned 12y ago
  The researchers need to profit in order for the incentives to truly align. It's like an incredibly poor man's spotify and we all know how that works for musicians.

czr80 12y ago

No. Peer reviewers get: early access to articles of interest, the feeling of contributing to the community they belong to, the ability to demonstrate their knowledge, the ability to reciprocate other people for reviewing their own articles, and so on. These are not financial rewards, but they are not less strong for that (arguably even stronger). Journals are heavily incentivized to maintain standards because otherwise good authors will stop submitting to them and librarians will stop subscribing (of course only the first incentive holds for OA journals). The fact that system sometimes fails is not proof that it is a bad system - all human systems have a failure rate.

untilHellbanned 12y ago

Onarbor, https://onarbor.com https://onarbor.com, is doing precisely what you describe. It's publishing and funding. I'm one of its creators so would be grateful if you'd be interested in talking more. tim@onarbor.com.

pcrh 12y ago

The standards required for patentability are quite different from scientific novelty or interest. Paying reviewers is not a bad idea, as journals that pay will both be more likely to attract quality reviewers, and it also creates a contract between the reviewer and the journal. Such a contract could include requirements for the standard of reviewing. If a paper is later found to have been poorly reviewed the journal can seek redress from the reviewers, further encouraging them to do their job properly.

dalke 12y ago

So if I have an idea that might be patentable, the journal will handle all the costs for me? Including maintenance fees and the cost of appeals if the patent is rejected? Sweet! Someone else takes the brunt of the $10,000+ in patent filing costs, and there's no downside for me! In other words, your proposed journal just managed to increase the cost of publication by over an order of magnitude.