3 ms·

Hi! Author here. Preprint review as it is being discussed here is post-publication. The preprint is shared first, and review layered on top of it later. Click

Hi! Author here. Preprint review as it is being discussed here is post-publication. The preprint is shared first, and review layered on top of it later. Click through to the paper[1] I'm responding to and give it a read. But, also, prepublication review doesn't need to be "reintroduced". It's still the standard for the vast majority of scholarship. By some estimates there are around 5 million scholarly papers published per year. There are only about 10 - 20 million preprints published total over the past 30 years since Arxiv's introduction. There are a bunch of layered institutions and structures that are maintaining it as the standard. I don't have data for it to hand, but my understanding is that the vast majority of preprints go on to be published in a journal with pre-publication review. And as far as most of the institutions are concerned, papers aren't considered valid until they have published in a journal with prepublication review. There is a significant movement pushing for the adoption of preprint review as an alternative to journal publishing with the hope that it can begin a change to this situation. The idea is that preprint review offers a similar level of quality control as journal review (which, most reformers would agree is not much) and could theoretically replace it in those institutional structures. That would, at least invert the current process: with papers being shared immediately and review coming later after the results were shared openly. [1] https://journals.plos.org/plosbiology/article?id=10.1371/journal.pbio.3002502 https://journals.plos.org/plosbiology/article?id=10.1371/jou...

adastra22 3y ago
Thanks for your charitable response to a reader who quite obviously misunderstood what you were writing about. You might want to update your article to make that clear, as "preprint review" isn't a term I had ever encountered before, and a simple reading of your article doesn't obviously indicate that you were talking about post-publication review. (Note that outside of academia, upload to arXiv is publication, so yes your "preprint review" would be a post-publication review.)
- dbingham 3y ago
  No worries. I was really writing for an audience of academics and, in particular, people involved in the various open science and publication reform movements. Sharing here was a bit of an after thought, so it wasn't written for an audience unfamiliar with the intricacies of academic publishing. It's a complicated enough space as it is that this is already a 4-5 article series with the potential to grow, even when I'm writing with the assumption of quite of a bit of knowledge. But yeah, "preprint review" is considered post-publication review both in and outside of academia. There are nuances to what is consider "publication" in academia. A preprint is not a "Version of Record", meaning it doesn't count towards tenure and promotion. The movement pushing for preprint review is attempting to layer review on top of already public preprints in the hopes that reviewed preprints can begin to count as VORs. It's unclear whether that will work. Some models, like eLife's, seem more promising than others. But eLife got a ton of backlash when they switched to their new reviewed preprint model, so it remains to be seen whether it will work in the long run.
MichaelZuo 3y ago
Quality control was handled fine enough by editors for literally all output on the planet pre 1970s. There’s nothing physically stopping that paradigm from returning now that we have the internet, other than the fact that there’s only a few thousand (?) such editors. If somehow there were fifty thousand such editors, then the whole peer review system would be completely unnecessary. Of course not enough people want to pay for that many editors, but that doesn’t stop a partial adoption by those willing to do so via some arrangements.
- dbingham 3y ago
  There are nearly 50,000 commercial journals and a long tail of non-commercial journals each with teams of editors. There are probably hundreds of thousands of people currently serving as editors. The issue isn't editorial bandwidth, it's that peer review is currently built into the promotion and tenure structure for academics, who produce the vast majority of scholarship and thus dictate the shape of the scholarly publishing system. Academics have to publish the papers in peer reviewed journals for it to count towards tenure and promotion. And in fact, they are limited to a small set of journals that are deemed high quality enough for their fields. These journals are chosen by tenure and promotion committees composed of their senior peers and school administration. There are over 1000 R1 and R2 universities worldwide, each with hundreds of departments each with their own tenure and promotion committees. So changing the system is a massive collective action problem.
  - MichaelZuo 3y ago
    I’m including only actually competent, full time editors, with sufficiently high reputation that their decisions will be taken seriously. There’s definitely not 50000 of those. A huge number of journals by numerical count, along with their ‘editors’, are literally laughed at in many fields. As you’ve mentioned, trying to expand the actually reputable number by 10x, 20x, etc… is a huge problem. Hence it has to be paid for, quite highly paid for, otherwise the coordination problem is probably impossibly difficult.

adastra22 3y ago

Thanks for your charitable response to a reader who quite obviously misunderstood what you were writing about. You might want to update your article to make that clear, as "preprint review" isn't a term I had ever encountered before, and a simple reading of your article doesn't obviously indicate that you were talking about post-publication review. (Note that outside of academia, upload to arXiv is publication, so yes your "preprint review" would be a post-publication review.)

dbingham 3y ago
No worries. I was really writing for an audience of academics and, in particular, people involved in the various open science and publication reform movements. Sharing here was a bit of an after thought, so it wasn't written for an audience unfamiliar with the intricacies of academic publishing. It's a complicated enough space as it is that this is already a 4-5 article series with the potential to grow, even when I'm writing with the assumption of quite of a bit of knowledge. But yeah, "preprint review" is considered post-publication review both in and outside of academia. There are nuances to what is consider "publication" in academia. A preprint is not a "Version of Record", meaning it doesn't count towards tenure and promotion. The movement pushing for preprint review is attempting to layer review on top of already public preprints in the hopes that reviewed preprints can begin to count as VORs. It's unclear whether that will work. Some models, like eLife's, seem more promising than others. But eLife got a ton of backlash when they switched to their new reviewed preprint model, so it remains to be seen whether it will work in the long run.

MichaelZuo 3y ago

Quality control was handled fine enough by editors for literally all output on the planet pre 1970s. There’s nothing physically stopping that paradigm from returning now that we have the internet, other than the fact that there’s only a few thousand (?) such editors. If somehow there were fifty thousand such editors, then the whole peer review system would be completely unnecessary. Of course not enough people want to pay for that many editors, but that doesn’t stop a partial adoption by those willing to do so via some arrangements.

dbingham 3y ago
There are nearly 50,000 commercial journals and a long tail of non-commercial journals each with teams of editors. There are probably hundreds of thousands of people currently serving as editors. The issue isn't editorial bandwidth, it's that peer review is currently built into the promotion and tenure structure for academics, who produce the vast majority of scholarship and thus dictate the shape of the scholarly publishing system. Academics have to publish the papers in peer reviewed journals for it to count towards tenure and promotion. And in fact, they are limited to a small set of journals that are deemed high quality enough for their fields. These journals are chosen by tenure and promotion committees composed of their senior peers and school administration. There are over 1000 R1 and R2 universities worldwide, each with hundreds of departments each with their own tenure and promotion committees. So changing the system is a massive collective action problem.
- MichaelZuo 3y ago
  I’m including only actually competent, full time editors, with sufficiently high reputation that their decisions will be taken seriously. There’s definitely not 50000 of those. A huge number of journals by numerical count, along with their ‘editors’, are literally laughed at in many fields. As you’ve mentioned, trying to expand the actually reputable number by 10x, 20x, etc… is a huge problem. Hence it has to be paid for, quite highly paid for, otherwise the coordination problem is probably impossibly difficult.