13 ms·
The peer review industry: implausible and outrageous
- SoylentOrange 8y agoOpen access, as is mentioned in the article, is both important and difficult to achieve. As a PhD student or associate professor, you work on a strict schedule to publish or perish. The only acceptable publication venues are the 4-6 respectable journals in your field. You have no time or power to push for open access. Only large research institutions like (I’m familiar only with the CS research community) MSR, Google Research, Facebook Research, ... as well as already tenured professors can really force the issue. I work at a smaller industrial research lab. We try to push for open access whenever we publish, but publishing in non open access journals is important to maintaining our perceived prestige, which in turn is important in hiring (especially when we poach from academia). I publish pretty regularly and try to be selective, but ultimately the push has to come from an institutional level such as universities or large industrial research labs to say “we will only publish in open access journals”. This is especially true in the US where EU style legislation is a long way off.
- skeptic_69 8y agodo you put your work in the arxiv?
- cbkeller 8y agoNot the OP, but I've started putting all my first-author papers in the EarthArXiv [1]. The original ArXiv only accepts preprints from a few fields, but the Open Science Framework has recently made a federation of preprint servers that fills many of the gaps, and can be searched in one place [2]. [1] https://eartharxiv.org/ https://eartharxiv.org/ [2] https://osf.io/preprints/ https://osf.io/preprints/
- ma2rten 8y agoLarge industrial research labs have the same problem. They need to maintain their prestige for recruiting purposes.
- drb91 8y agoAt the same time, this seems to select the worst of the candidates: those who care about prestige.
- Vinnl 8y ago> Only large research institutions like (I’m familiar only with the CS research community) MSR, Google Research, Facebook Research, ... as well as already tenured professors can really force the issue. Well, they can to a certain extent, by submitting their work to open access (preferably fair OA [1]) journals, thus increasing their prestige. I think a more important role here lies with funders, though, and more specifically the people that decide who to fund (although these often are researchers as well). By currently strongly relying on journal brand names as indicators of "excellence", they're providing the incentive that keeps/forces academics to publish in traditional, paywalled or high-APC journals. They need to find other ways to decide whom to give funding to. (And perhaps have less of a focus on an ill-defined idea of "excellence" as well.) Disclaimer: I'm part of a just-launched initiative that aims to provide such an alternative. I'd love to hear people's thoughts about this though. [1] https://www.fairoa.org/ https://www.fairoa.org/
- deleted 8y ago[deleted]
- deleted 8y ago[deleted]
- dpc_pw 8y agoThe way academia works makes absolutely no sense. And then people wonder "why do so many people are <<anti-scientific>>?". Well, maybe because it's all done by a priest-caste cartel, shielded from reality in ivory towers, and then presented to the public in a similar way religion once was forced on peasants (taxes included). Academia is the modern priesthood: usurping the right to all knowledge, in bed with the state, corrupted to the bones, fighting heretics.
- maxxxxx 8y ago"Academia is the modern priesthood: usurping the right to all knowledge, in bed with state, corrupted to the bones." That goes way too far. There are plenty of areas where great work is being done. But yes, there are some where bad work is done. And a lot of the people that talk about science in public have an agenda and they get heard over the people who do real science.
- dpc_pw 8y ago> There are plenty of areas where great work is being done. You could say the same stuff about the historical priesthood. For ages, knowledge, and science were cultivated by priests. They were the ones that could read, write, they hand-copied books, they advised rules, helped peasants, took care of sick, poor and so an so. Which doesn't invalidate my critique about the corruption. The mechanisms are exactly the same.
- maxxxxx 8y agoYou just have to be careful not to reject all scientists. When I listen to my neighbor he rejects all climate science (because they all have an agenda) but trusts the deniers who are a very loud, very small minority. This make no sense.
- kevin_thibedeau 8y agoHuman psychology. Negative statements are given greater weight. You neighbor is happily living in the matrix and sees no need to observe reality with objective reason when the negative pundits give him regular doses of satisfying outrage to feel good about.
- apo 8y agoWhat to do about the entrenched scientific publishing-funding complex has been a concern since the first full text of journal articles started appearing in the 1990s. The response to date has been for funding agencies (nonprofits and governments) to require publication in open access journals as a condition of funding research. This has happened in both the US and EU. However, I'm pessimistic this approach will address the problem, and it could well create a number of unintended consequences. For one, publishers that have introduced OA models can charge thousands of dollars to the research teams publishing the work. That could mean fewer publications, and more information silos. The article hints at, but doesn't fully describe another, more pressing problem. Peer review is in trouble. The number of papers the average researcher is being asked to review is increasing while the abundance of papers means the few bother to read them. I suspect the solution lies somewhere else entirely. Post-publication peer review is one option, and Fighare has built an interesting model that could be a component of a future system: https://figshare.com https://figshare.com Regardless, the problem is cultural, not technical. The crazy economic incentives described in the article work because everyone involved has a (sometimes perverse) incentive to keep it in place. Until that changes, it will be mostly status quo ante.
- buboard 8y ago> the number of papers the average researcher is being asked to review is increasing The quality of review becomes proportional to the impact factor of the journal. And given that the high impact journals have not substantially expanded the number of articles they publish, it might mean that there are now more candidate reviewers. What has become substantially more burdensome is the number of funding applications that (esp. young) investigators have to submit per year. http://www.pnas.org/content/112/2/313 http://www.pnas.org/content/112/2/313 > Post-publication peer review is one option, and Fighare There are interesting proposals. E.g. elifesciences proposes to publish everything, under the condition that the reviews will be published eponymously along with the paper
- lsh 8y agono proposals, the review feedback and author response to the review are published. Here is an example: https://elifesciences.org/articles/35082#SA2 https://elifesciences.org/articles/35082#SA2 Peer review at elife works slightly differently. The author(s) are not subjected to multiple rounds with multiple reviewers, instead the review feedback from the peer reviewers is aggregated by the editorial staff before being sent back to the author. There may be multiple rounds of this however.
- euske 8y agoI just realized that this analogy of the academic publishing industry is also quite similar to that of the search engine industry. Most web pages are provided to the public free of charge, but they typically don't obtain them directly (technically they can, but it's often too tedious). A search engine takes them and redistribute to the public. They do it by using the metrics mostly provided by the (unpaid) public: PageRank. And they make a hefty profit for doing that. They also amass a lot of power. Both industries are essentially doing the same thing: content curation. The difference is that search engines don't get complaints like academic publishers. I think the reason is that the academic publishing is much simpler and actually more transparent than search engines; you can easily see the inner workings and the flow of money. On the other hand, people don't really understand how Google makes so much money. We're effectively blinded by the complexity.
- jungturk 8y agoThe search engines have sidestepped grief because of their third-party pays model - advertisers fund their content curation and discovery expenses and the system retains open access at no (direct, monetary) cost to the consumers. The peer review process might be more expensive/intensive than PageRank for all I know, so perhaps journal publishers are doing much more work than search engines. Perhaps if journal publishers used an ad-supported model they wouldn't be demonized so readily since they'd be incentivized for open access since they're selling eyeballs I'm not advocating, just working with the parent's comparison.
- atq2119 8y ago> The peer review process might be more expensive/intensive than PageRank for all I know, so perhaps journal publishers are doing much more work than search engines. The peer review process is labor intensive, but to add insult to injury, publishers aren't the ones doing it and they don't pay reviewers. It's all done for free by researchers. At best publishers add some proof reading services, but in many cases not even that.
- confused_guy123 8y agoWhat I don't understand is why peer reviewers agree to perform uncompensated labor on behalf of these gargantuan, highly profitable publishers. What is the benefit to them, other than the warm glow of fulfilling one's professional obligations?
- zmb_ 8y agoOften it is a favor to a colleague that is serving as the (also unpaid) editor to get a line on their CV. Given that the fields are highly specialized, you tend to work with the same small community of people (writing articles and getting project funding together, reviewing each other's articles, etc.), so it pays to be well-liked among your peers.
- verylongname 8y agoIts very hard to turn down a request to review a paper you are qualified to comment on from an editor you know, or who regularly handles your papers. Editor X from Journal Y sends you a paper which you are well-qualified to review. You've submitted 5 papers to Journal Y in the last few years, some of them handled by Editor X. Are you really going to turn down Editor X's request? What if the paper is written by Author Z, whose work you are familiar with and whose papers you read anyway. Why not review it? And so on.
- jonathanstrange 8y agoMostly it's the latter, out of a sense of duty. But there are other motives. Regular reviews give you some perspective that you otherwise wouldn't get. For example, you see what mistakes others make and that can help you improve your own papers. Another motivation for me is sometimes that I don't want the paper to be reviewed by someone less qualified in my areas of specialization.
- 08-15 8y agoEver noticed that the person who appeals to your sense of duty is always your boss who doesn't want to give you a raise?
- 8y ago
- titzer 8y ago> (4 paragraph analogy of implausible and outrageous scheme of bakeries which is total fiction). > ...Yet the story is broadly analogous to what happens in the publication of academic journals. While I think there are lots of problems with the academic journal system, please, please, please do not reason from analogy like this article does. This is a complex topic that deserves being reasoned about directly. Presenting an analogy which is obviously artificial just primes readers with a false, oversimplified narrative. Don't do this. This is highly misleading, and the bias that results from the priming is corrosive to clear reasoning.