6 ms·

Admittedly I work outside academia (though my wife is a tenured professor) and thus have some unavoidable naïveté about how the culture and community work, but

Admittedly I work outside academia (though my wife is a tenured professor) and thus have some unavoidable naïveté about how the culture and community work, but I’m always confused about why universities aren’t just sponsoring the organizational costs of coordinating the review process, and establishing community-driven open journals to replace the rent-seeking corporate journals. What am I missing about the benefits of the status quo keep it so entrenched?

yread 8y ago
Universities could be in conflict of interest - organizing review of competing (or worse its own) research
- apetresc 8y ago
  Isn't the peer review process blind, anyway? University affiliation would presumably be one the scrubbed fields, no?
  - cli 8y ago
    The process is blind, but it's fairly easy to know who the author is just due to the topic at hand. At least for specific subfields.
    - Mediterraneo10 8y ago
      Agreed. Within my own field, there are less than 10 people who are presently working on the same topics as I do. Consequently, we are constantly asked to review each other’s papers, and it is pretty trivial to tell who the author is, either because of the views and interests shown in the paper, or because of language use (i.e. you can tell that the author is a native speaker of the language in which the paper was written, or from their unidiomatic skills in it you can often guess at their native language). The hard part in this is remaining silent and pretending you don’t know anything about this, when you meet your peers at conferences.
    - njarboe 8y ago
      It is not blind for most science fields (I was not aware of any that were blind before reading this. PhD in Earth Science). What fields do you know of where journals send out papers for review with the author and his/her affiliation removed?
      - jpeanuts 8y ago
        I've seen it exactly once (in a numerical analysis journal). Authors were blacked out. This was undermined almost immediately when the authors wrote in the introduction "... following previous work of the present authors [1]...". Even if all these clues could be effectively removed, the nature of the research work, tools used, writing style, all unambiguously indicate authorship to someone in the field. I think for this reason double-blind review is not popular. Which is not to say that it would be great if it were possible...
      - cli 8y ago
        Nature offers a double-blind option for peer review: https://www.nature.com/authors/policies/peer_review.html?&$NMW_TRANS$=ext https://www.nature.com/authors/policies/peer_review.html?&$N... See the "anonymity" section.
  - suchow 8y ago
    Whether peer review is blinded depends on the field. Some of the major ML conferences like NIPS are blinded, but in general most journals are not. The reviewers see the authors and affiliations and the editors see everything.
  - mrybczyn 8y ago
    Many fields are small enough, with infrequent enough papers, that experts will be able to tell the author from style and content alone...
  - Vinnl 8y ago
    Blinding is not as effective as you'd think: http://blogs.plos.org/absolutely-maybe/2017/10/31/the-fractured-logic-of-blinded-peer-review-in-journals/ http://blogs.plos.org/absolutely-maybe/2017/10/31/the-fractu...
- titanix2 8y ago
  They already are organizing conferences, which for some fields is a major way of sharing research so it wouldn’t be a problem.
- PeterisP 8y ago
  Just as with the established journals, organizing it isn't that influential and relevant compared to actual editorial board; and it would be likely or even expected that the board of a publication organized by some university would have just a couple members from that university and dozens of respected scientists from outside. It's generally not considered conflict of interest, as the boards transcend single institutions and their members tend to be individuals in the stage of career where their individual reputations that matter more than the reputation of the institution they "represent".
alexbeloi 8y ago
>What am I missing about the benefits of the status quo keep it so entrenched? The entrenchment is due to perceived (and cultivated) prestige, less so about tangible benefits. Prestigious journals have good brand value, they spend money cultivating it and they extract earnings from it. Researchers and administrators trade on journal brand value for career advancements. They benefit from a highly prestige-stratified journal catalog as a tool for differentiating themselves from their peers, it's much easier to justify a promotion based on 'X publications to prestigious journal A, B, C' than 'bunch of people in the community think this is a great result', especially to a non-expert.
- xtracto 8y ago
  This is good for blockchain... In all seriousness, I have always thought that Blockchain could pose a good solution for the issue of peer-reviewing scientific papers. It provides a descentralized infrastructure with "tokens" that can be transferred to provide "prestige" to papers and authors.
  - blocked_again 8y ago
    Why not use a centralized karma system like hacker news does?
    - arthur_pryor 8y ago
      because then you wouldn't get a chance to say "blockchain"
      - SpeciesInvader 8y ago
        You could call it hn blockchain, but only if you paint it mauve...
  - arthur_pryor 8y ago
    ...right, but then you have to figure out explicit metrics for determining what "prestige" is, and you have to explicitly implement the mechanics of it. i think this is a worthwhile research project of its own (or many research projects). but the mapping is very much non-trivial. i think the amount that people complain about how poorly the usual social media ontology ("friends", "likes", etc) maps to the "real"-world equivalent shows that this isn't something were you can just be like "we'll implement a first-pass of the blockchain thing now, and we'll refine the granularity of our prestige tokens later". thanks, but no thanks. which is all to say, like so many of the things people want to use a block chain for, the hard part is the politics and the real-world nuance, not the immutable ledger. the immutable ledger is not trivial. but compared to the other stuff, it's not all that hard (and is certainly more solved, at this point).
gowld 8y ago
The academy is a primary example status quo and gatekeeping. Any change to the system is risky to them.
Tomminn 8y ago
The way to think about this is an article in Nature is like winning a tiny Nobel prize, with no prize money. You can make whatever other sensible publication outlet you like, but it's hard to create something that has the prestige of winning a tiny Nobel prize. That's Nature's value proposition. Brand, history and exclusivity. This is why you can't get too angry at the scientists who continue to publish in it. If you kill yourself to figure something out worthy of a tiny Nobel prize, are you really not going to claim it? It's not like claiming that tiny Nobel prize is gonna hurt the dissemination of your work. The higher profile of your work will easily compensate for the fact people have to jump through a hoop to read it. This is why I love sci-hub. It just destroys the hoop. It doesn't need to figure out how to replace the brand, history and exclusivity of publications like Nature. Along with government-mandated open source laws, it's one of the only realistic ways to actually disrupt this industry.
- Vinnl 8y ago
  I don't think Sci-Hub breaks the loop. Researchers still need to publish with the mini Nobel prizes to advance their career in a highly competitive field, and governments/funders still don't want to dictate where researchers get to publish their work, so in effect, what happens is that governments mandate Open Access and pay the same old publishers significant (and disproportionate) amounts of money to make the single articles they fund openly available. In other words, the costs of research still don't go down, and the government-mandates were happening without Sci-Hub as well...
  - Tomminn 8y ago
    At the university I'm at, every time the library dropped its subscription to a particular journal there used to be a strong backlash from academics at the university. Since sci-hub, the library has been able to drop a bunch of journals and for some mysterious reason none of the academics seem to give as much of a shit anymore. Obviously, no-one has no idea why..
mirimir 8y ago
That's basically how major journals got started, long ago. Most were essentially not-for-profit. But then, since the 70s, they've been acquired by large publishers. And those new owners have been monetizing those journals' prestige and market share. Given the inertia of the academic community, they've had a long run at that. But the end is coming, I think.
- Vinnl 8y ago
  The Guardian had a great longread describing that history: https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science https://www.theguardian.com/science/2017/jun/27/profitable-b... And I'm afraid that, even though a lot is changing, the disproportionate monetising isn't yet done for.
hannob 8y ago
A lot of people want to do exactly this. The problem is that changing an existing culture that is deep in everything how this system works is difficult. This goes way beyond the mere question of publication, it's still pretty common to hire people based on high impact factor journal publications etc. You're faced with a system that's controlled by the people who made their career in this system. They're not the most qualified to improve the failures of that system. Changing it is possible, but it's hard and it's a slow process.
- dv_dt 8y ago
  One hopes it will change faster than the typical "one a death at a time".
  - Vinnl 8y ago
    Well, the Open Access movement has been going at it for more than two decades now, so don't get your hopes up... (That said, it's definitely not like nothing has changed! There's a lot changing at the moment, and it's not quite sure what the system is going to look like. Traditional publishers aren't looking too bad yet, though.)
MaupitiBlue 8y ago
Why not adopt the legal profession's approach and have grad students operate the journals? I.e. Harvard Law Review?

yread 8y ago

Universities could be in conflict of interest - organizing review of competing (or worse its own) research

apetresc 8y ago
Isn't the peer review process blind, anyway? University affiliation would presumably be one the scrubbed fields, no?
- cli 8y ago
  The process is blind, but it's fairly easy to know who the author is just due to the topic at hand. At least for specific subfields.
  - Mediterraneo10 8y ago
    Agreed. Within my own field, there are less than 10 people who are presently working on the same topics as I do. Consequently, we are constantly asked to review each other’s papers, and it is pretty trivial to tell who the author is, either because of the views and interests shown in the paper, or because of language use (i.e. you can tell that the author is a native speaker of the language in which the paper was written, or from their unidiomatic skills in it you can often guess at their native language). The hard part in this is remaining silent and pretending you don’t know anything about this, when you meet your peers at conferences.
  - njarboe 8y ago
    It is not blind for most science fields (I was not aware of any that were blind before reading this. PhD in Earth Science). What fields do you know of where journals send out papers for review with the author and his/her affiliation removed?
    - jpeanuts 8y ago
      I've seen it exactly once (in a numerical analysis journal). Authors were blacked out. This was undermined almost immediately when the authors wrote in the introduction "... following previous work of the present authors [1]...". Even if all these clues could be effectively removed, the nature of the research work, tools used, writing style, all unambiguously indicate authorship to someone in the field. I think for this reason double-blind review is not popular. Which is not to say that it would be great if it were possible...
    - cli 8y ago
      Nature offers a double-blind option for peer review: https://www.nature.com/authors/policies/peer_review.html?&$NMW_TRANS$=ext https://www.nature.com/authors/policies/peer_review.html?&$N... See the "anonymity" section.
- suchow 8y ago
  Whether peer review is blinded depends on the field. Some of the major ML conferences like NIPS are blinded, but in general most journals are not. The reviewers see the authors and affiliations and the editors see everything.
- mrybczyn 8y ago
  Many fields are small enough, with infrequent enough papers, that experts will be able to tell the author from style and content alone...
- Vinnl 8y ago
  Blinding is not as effective as you'd think: http://blogs.plos.org/absolutely-maybe/2017/10/31/the-fractured-logic-of-blinded-peer-review-in-journals/ http://blogs.plos.org/absolutely-maybe/2017/10/31/the-fractu...
titanix2 8y ago
They already are organizing conferences, which for some fields is a major way of sharing research so it wouldn’t be a problem.
PeterisP 8y ago
Just as with the established journals, organizing it isn't that influential and relevant compared to actual editorial board; and it would be likely or even expected that the board of a publication organized by some university would have just a couple members from that university and dozens of respected scientists from outside. It's generally not considered conflict of interest, as the boards transcend single institutions and their members tend to be individuals in the stage of career where their individual reputations that matter more than the reputation of the institution they "represent".

alexbeloi 8y ago

>What am I missing about the benefits of the status quo keep it so entrenched? The entrenchment is due to perceived (and cultivated) prestige, less so about tangible benefits. Prestigious journals have good brand value, they spend money cultivating it and they extract earnings from it. Researchers and administrators trade on journal brand value for career advancements. They benefit from a highly prestige-stratified journal catalog as a tool for differentiating themselves from their peers, it's much easier to justify a promotion based on 'X publications to prestigious journal A, B, C' than 'bunch of people in the community think this is a great result', especially to a non-expert.

xtracto 8y ago
This is good for blockchain... In all seriousness, I have always thought that Blockchain could pose a good solution for the issue of peer-reviewing scientific papers. It provides a descentralized infrastructure with "tokens" that can be transferred to provide "prestige" to papers and authors.
- blocked_again 8y ago
  Why not use a centralized karma system like hacker news does?
  - arthur_pryor 8y ago
    because then you wouldn't get a chance to say "blockchain"
    - SpeciesInvader 8y ago
      You could call it hn blockchain, but only if you paint it mauve...
- arthur_pryor 8y ago
  ...right, but then you have to figure out explicit metrics for determining what "prestige" is, and you have to explicitly implement the mechanics of it. i think this is a worthwhile research project of its own (or many research projects). but the mapping is very much non-trivial. i think the amount that people complain about how poorly the usual social media ontology ("friends", "likes", etc) maps to the "real"-world equivalent shows that this isn't something were you can just be like "we'll implement a first-pass of the blockchain thing now, and we'll refine the granularity of our prestige tokens later". thanks, but no thanks. which is all to say, like so many of the things people want to use a block chain for, the hard part is the politics and the real-world nuance, not the immutable ledger. the immutable ledger is not trivial. but compared to the other stuff, it's not all that hard (and is certainly more solved, at this point).

gowld 8y ago

The academy is a primary example status quo and gatekeeping. Any change to the system is risky to them.

Tomminn 8y ago

The way to think about this is an article in Nature is like winning a tiny Nobel prize, with no prize money. You can make whatever other sensible publication outlet you like, but it's hard to create something that has the prestige of winning a tiny Nobel prize. That's Nature's value proposition. Brand, history and exclusivity. This is why you can't get too angry at the scientists who continue to publish in it. If you kill yourself to figure something out worthy of a tiny Nobel prize, are you really not going to claim it? It's not like claiming that tiny Nobel prize is gonna hurt the dissemination of your work. The higher profile of your work will easily compensate for the fact people have to jump through a hoop to read it. This is why I love sci-hub. It just destroys the hoop. It doesn't need to figure out how to replace the brand, history and exclusivity of publications like Nature. Along with government-mandated open source laws, it's one of the only realistic ways to actually disrupt this industry.

Vinnl 8y ago
I don't think Sci-Hub breaks the loop. Researchers still need to publish with the mini Nobel prizes to advance their career in a highly competitive field, and governments/funders still don't want to dictate where researchers get to publish their work, so in effect, what happens is that governments mandate Open Access and pay the same old publishers significant (and disproportionate) amounts of money to make the single articles they fund openly available. In other words, the costs of research still don't go down, and the government-mandates were happening without Sci-Hub as well...
- Tomminn 8y ago
  At the university I'm at, every time the library dropped its subscription to a particular journal there used to be a strong backlash from academics at the university. Since sci-hub, the library has been able to drop a bunch of journals and for some mysterious reason none of the academics seem to give as much of a shit anymore. Obviously, no-one has no idea why..

mirimir 8y ago

That's basically how major journals got started, long ago. Most were essentially not-for-profit. But then, since the 70s, they've been acquired by large publishers. And those new owners have been monetizing those journals' prestige and market share. Given the inertia of the academic community, they've had a long run at that. But the end is coming, I think.

Vinnl 8y ago
The Guardian had a great longread describing that history: https://www.theguardian.com/science/2017/jun/27/profitable-business-scientific-publishing-bad-for-science https://www.theguardian.com/science/2017/jun/27/profitable-b... And I'm afraid that, even though a lot is changing, the disproportionate monetising isn't yet done for.

hannob 8y ago

A lot of people want to do exactly this. The problem is that changing an existing culture that is deep in everything how this system works is difficult. This goes way beyond the mere question of publication, it's still pretty common to hire people based on high impact factor journal publications etc. You're faced with a system that's controlled by the people who made their career in this system. They're not the most qualified to improve the failures of that system. Changing it is possible, but it's hard and it's a slow process.

dv_dt 8y ago
One hopes it will change faster than the typical "one a death at a time".
- Vinnl 8y ago
  Well, the Open Access movement has been going at it for more than two decades now, so don't get your hopes up... (That said, it's definitely not like nothing has changed! There's a lot changing at the moment, and it's not quite sure what the system is going to look like. Traditional publishers aren't looking too bad yet, though.)

MaupitiBlue 8y ago

Why not adopt the legal profession's approach and have grad students operate the journals? I.e. Harvard Law Review?