8 ms·

I would love a world where publishing papers is cheap and reading papers, particularly taxpayer-funded ones, is free. However, I'd also like a job and none of

I would love a world where publishing papers is cheap and reading papers, particularly taxpayer-funded ones, is free. However, I'd also like a job and none of the "top tier" biomedical journals are open access. While I'd like to think that the quality of my work would speak for itself, the imprimatur that one gets from Nature, Science, Cell, and Neuron is worth quite a bit when applying for jobs and grants. Unless the big names in academia--and the search committees they sit on--publicly and credibly commit to this, you're essentially asking people (many of whom are on short-term contracts making $40,000 a year) to take an enormous career risk.

Fomite 10y ago
This. The biggest and most meaningful way to do this would be for people sitting on search committees and tenure/promotion committees to say publicly that candidates wouldn't be punished for deliberately publishing in OA journals.
- JorgeGT 10y ago
  Nobody punishes you for publishing in OA journals, what matters is the IF and JCR rank of the journal, not if it's OA or not.
  - mattkrause 10y ago
    If you find an OA journal with cachet matching Nature or Science, I would be happy to submit stuff there. As far as I know, there's nothing even particularly close. In other words, I doubt anyone is explicitly penalized for submitting to PLoS Biology instead of Journal of Neuroscience. However, there is an implicit penalty for not submitting to Nature/Science/Cell. [As an aside, I really hate the obsession with IF and all these other "quantitative" measures of research quality. They're all incredibly ad-hoc, someone easily game-able, and yet otherwise-smart people take them so seriously.]
    - throwaway729 10y ago
      Yes, they are pointless and arbitrary metrics. Competition for positions in academia has gotten so bad that it doesn't matter. Like Google rejecting perfectly good candidates just because they can, but in Academia even south central Kansastucky state university has enough excellent applicants that arbitrary metrics do more good than harm.
    - JorgeGT 10y ago
      At least in my country, I'm thankful for these quantitative measurements, that difficult corrupt practices: Oh, I assure you he's a great researcher, just not in any measurable way. He's my nephew you say? Well, that's not the point...
      - mattkrause 10y ago
        Couldn't you just require people to recuse themselves from hiring decisions involving relatives? Particularly for early career people, I suspect most of these "quantitative" measurements are basically reflecting where and wtih whom you worked.
        Fomite 10y ago
        There's also less nepotistic problems. For example, I've moved between different sides of the same overall field several times. That means, for any given reader of my CV, only about half the "impressive" papers read to them as impressive using most subjective measures.
        JorgeGT 10y ago
        > Couldn't you just require people to recuse themselves from hiring decisions involving relatives? They used to have this nice trick where you hired the son of your friend and your friend hired your son. But at least now there are some non-subjective requirements to be met before anyone can get admitted.
      - angry_octet 10y ago
        While this is important, in many fields their are cliques of related researchers (1) on the grants committees and (2) on the journal committees. These people know who works on which problems, so blind review is a farce. Also, the concept of IF being a proxy for rigorous review and work quality is weak at best. Just read a few pages of http://retractionwatch.com/ http://retractionwatch.com/
  - hannob 10y ago
    Impact factors and similar metrics are part of the problem. These are metrics that support flashy research, not good research. If anything the Impact Factor is a sign of sloppy research (there's a lot of evidence for that, e.g. the higher the IF the more likely there's a retraction).
    - JorgeGT 10y ago
      But it's not an easy problem. How do you grade from 0 to 10 good research from a candidate?
    - dagw 10y ago
      there's a lot of evidence for that, e.g. the higher the IF the more likely there's a retraction The higher the IF, the more likely it is the researcher trying to do something 'hard' and the more people will scrutinise the results. These two factors alone will lead to more retractions. Someone publishing something completely uninteresting in a journal no one reads will never be in a position to have to retract their paper, no matter how flawed, because no one will notice and no one will care.
  - Fomite 10y ago
    But given the IF and ranks of OA journals, that's effectively the same thing. What you'd have to do is essentially give OA journals a handicap. Not to mention the opportunity cost associated with OA fees, which can add up if you're talking about lab startup.
- a3_nm 10y ago
  The problem is the following: if there is no established OA journal in the field, you can't distinguish the candidate who "deliberately" publishes good quality work in a non-selective OA journal, from the candidate who publishes low-quality work in the same non-selective OA journal. The only way to tell is to read the papers and judge for yourself, and this is the root of the problem: evaluating candidates based on the number of papers and the impact factor (or some analogous metric) of the venues is much, much easier for committees, than actually reading the papers.
  - brownbat 10y ago
    In a very rough analogy, this is why we have film critics, because we can't trust other institutions in the industry (studios, individual theaters, directors) as a guarantee of quality. If a panel of respected tenured professors in the field ranked the best works in open access journals each month, or if there was a high profile yearly prize for best open access papers of the year, that could help disaggregate reputational effects from the publication itself. Bootstrapping this sort of thing is hard. I'm sure Nobels didn't signal as strongly in year one as they do now.
  - jonathanstrange 10y ago
    Exactly, this nonsense of mostly just counting publications must stop. If at all, there is only a very light correlation (or perhaps even a negative one, who knows?) between the number of publication and the quality of a scientist's research. And what's so crazy about it is that everybody knows this. No one has ever been called a great scientist, because he published many articles. Such scientists may be called prolific, but that doesn't necessarily reflect on the quality of the content. The same for counting quotations, by the way. This may be different in STEM fields, but I'm in the humanities. In the humanities bad papers are sometimes quoted as often as brilliant papers, because everybody wants to correct the authors. I've seen more than one colleague literally make his career by writing a polemic and obviously wrong book (books=often improper peer review) and then sneaking a few apologetic articles in which they half-retracted their claims past the reviewers. What's worse, I know postdocs who are actively looking for such untenable topics that will surely stir up a reaction. We're talking about theses, conjectures or claims that nobody defended previously, because they are so obviously wrong or flawed. I've also met people at the postdoc level who literally replied "Who cares, it's one more publication" when I raised doubts about their main claim in an article and mentioned that it might not 'work' for purely formal reasons (but adding that I'd have to check, to be polite). I'm sure it's a bit less drastic in CS and the STEM fields, but in the humanities publication counting almost enforces mediocrity and the publication of uninteresting, pointless papers with vastly overlapping topics - and it's completely obvious that it does so. The situation is particularly frustrating, because the system makes it harder to publish good science. Good science needs larger studies and/or better formal underpinnings, both of which take more time and more effort than duplicating fake papers about fake problems. Well, enough ranting for today, you get the point.
dv_dt 10y ago
It is interesting to separate the review from the storage/access portions of scientific publishing. Bind the prestige to the review board, not the publisher. Once you have open access, there's nothing stopping academics from forming an independent review board organization which then provide some form of stamps of approval; paper A from It seems like it would be a good organizational excuse to stay in touch with peers and exchange information on the field anyway...
- mattkrause 10y ago
  Have you heard of Faculty Of 1000 (f1000.com)? It sorta does that. The "faculty" submit 1-2 paragraph blurbs describing papers. The blurbs often point out the strengths and weaknesses of each paper and give a brief description of how it fits into the broader literature. If you can find a couple of people on there with similar research interests, it can be a great way to get up to speed on something.
BeetleB 10y ago
>However, I'd also like a job and none of the "top tier" biomedical journals are open access. Isn't it a requirement that all NIH funded work must be published as open access?
- JorgeGT 10y ago
  Open, but not immediately, I believe. Journals have an embargo period of 1-3 years, and after that you can put the accepted paper on a free repository. Your manuscript can usually be posted for free as soon as you want.
- Fomite 10y ago
  It is, but it's after a time. It's not actually weakly enforced - we've been told we can't take credit for publications on grant reports if they're not in PMC.
  - mattkrause 10y ago
    Don't you need to include the PMC ID number for each paper on the progress report/renewal? I vaguely remember doing something like this for our last one....
    - Fomite 10y ago
      Yep - you need the ID number, and if its not there, it doesn't count.
- dekhn 10y ago
  yes but it's only weakly enforced,and I believe there's an embargo peroid
- mattkrause 10y ago
  Not quite. You have to deposit a copy of the manuscript in PubMedCentral within a year of publication. Some journals let you deposit the final pdf (or even do it for you). Others restrict you to an unformatted "final draft" (i.e., the last thing you submitted to the journal). However, this is still tagged with the source journal, so it doesn't really get you around the career-related issues. Public Access guidelines: https://publicaccess.nih.gov/policy.htm https://publicaccess.nih.gov/policy.htm
JumpCrisscross 10y ago
> none of the "top tier" biomedical journals are open access Is this solely a product of network effects? Or could there be some truth to the top journals' ability to organise the review process?
- mattkrause 10y ago
  Pretty much. The review process seems to be about the same across most journals. In fact, there's probably a niche for some journals that have an actual editorial process where an editor actually...edits the prose. Nature/Science/Cell are popular because everyone recognizes their "brand" and associates them with interesting, high-impact science. Thus, it's easy for anyone "Hey, we should fund/hire her because she's got a bunch of Nature papers!" It's not miles from the old "Nobody got fired for buying IBM" bit. In contrast, it's unlikely that one person--let alone more-on the committee has carefully read your J. Soc. Whatever paper and says "This is well done! Get him instead!"
  - ProblemFactory 10y ago
    It's not just a subjective well-known brand. Nature/Science/Cell have a prestigious brand, so many scientists try to get their best work published there. The journals get first pick of new research to choose from, and as a result getting published there really is a good indicator that your work is impactful and high quality. It's a network effect that can be broken only if enough funding & promotion committees, or top scientists in a field take a stand together. Something I have wondered though: What if we separate the "measure of quality and impact" from getting published in the journals? Something like "shadow journals", that doesn't publish and edit any articles itself. Its reviewers would pick the top 30 new papers published in their field each month, and just publish links to them. If this would get established, it could be used by funding and promotion committees, and the brand of the original journals would matter less.
  - tnecniv 10y ago
    > Pretty much. The review process seems to be about the same across most journals. In fact, there's probably a niche for some journals that have an actual editorial process where an editor actually...edits the prose. Some journals do this. I've had PhD student friends inform me that when they've published to some (well-regarded) journals, after being accepted, they spent months going back and forth with the editor polishing the article.
antennas 10y ago
It's a prisoner's dilemma. If you (researcher) defect (publish in Elsevier journals), you perpetuate Elsevier's business, a shitty situation. But it's not as bad as cooperating (not publishing), because there's a chance that everyone else defects and you screw up your career, a really shitty situation. So out of self-interest everyone defects and perpetuates Elsevier's business.
- mizzao 10y ago
  More of a public goods problem (n-player PD).
jmnicholson 10y ago
This is a great point. We need to provide researchers with options that don't kill their career prospects. One idea, which we're working on at Authorea (authorea.com) is to help researchers write their documents online so they can be easily published as preprints. This way open access can be achieved in a manner that goes beyond a PDF or Word doc, and researchers can still play the game so long as it exists. Some more info here: https://www.authorea.com/123679/d8cErAb_olPC5CvcWWwsig https://www.authorea.com/123679/d8cErAb_olPC5CvcWWwsig
Certhas 10y ago
I think the best leverage is with the funding bodies. Declare that all publicly funded papers need to be OA, and this goes away in an instant. That said, these news made my life a lot easier right now. I was considering whether to accept an invitation to review for an Elsevier journal (which I've usually avoided). Now I'm fairly sure I will not.
pelario 10y ago
Agreeing with you, and also answering the GP: The issue I see is that "Academia" is not the same as "the scientists", there is a larger community, that includes institutions, processes, administrative people, etc. So, yes, for instance, I would love that we all published our papers in, say, arxive, an we were evaluated by the actual impact of our publications (~ PageRank). But that is not how the funding organizations, hiring committees, etc, are going to do it. How to do the transition from the current state of things to a better one, and not to commit career suicide? That is the question, and that is what those boycotts are aiming to, I believe.
tomjen3 10y ago
Sounds like we need to make it illegal to use government founds to publish in a non open access journal.
visarga 10y ago
> However, I'd also like a job and none of the "top tier" biomedical journals are open access Scientific domains with open publishing, like machine learning, have a very fast pace and amazing evolution. Won't biology as a field be affected severely by all this closeness? I think in the long run the whole field is suffering.
arthur2e5 10y ago
> particularly taxpayer-funded ones In the US there is a "Fair Access to Science and Technology Research Act of 2015" (S. 779, H.R.1477) that requires federal agencies to have a "public access policy" if they receive extramural research expenditures of over $100,000,000. https://act.eff.org/action/keep-publicly-funded-research-public https://act.eff.org/action/keep-publicly-funded-research-pub...

Fomite 10y ago

This. The biggest and most meaningful way to do this would be for people sitting on search committees and tenure/promotion committees to say publicly that candidates wouldn't be punished for deliberately publishing in OA journals.

JorgeGT 10y ago
Nobody punishes you for publishing in OA journals, what matters is the IF and JCR rank of the journal, not if it's OA or not.
- mattkrause 10y ago
  If you find an OA journal with cachet matching Nature or Science, I would be happy to submit stuff there. As far as I know, there's nothing even particularly close. In other words, I doubt anyone is explicitly penalized for submitting to PLoS Biology instead of Journal of Neuroscience. However, there is an implicit penalty for not submitting to Nature/Science/Cell. [As an aside, I really hate the obsession with IF and all these other "quantitative" measures of research quality. They're all incredibly ad-hoc, someone easily game-able, and yet otherwise-smart people take them so seriously.]
  - throwaway729 10y ago
    Yes, they are pointless and arbitrary metrics. Competition for positions in academia has gotten so bad that it doesn't matter. Like Google rejecting perfectly good candidates just because they can, but in Academia even south central Kansastucky state university has enough excellent applicants that arbitrary metrics do more good than harm.
  - JorgeGT 10y ago
    At least in my country, I'm thankful for these quantitative measurements, that difficult corrupt practices: Oh, I assure you he's a great researcher, just not in any measurable way. He's my nephew you say? Well, that's not the point...
    - mattkrause 10y ago
      Couldn't you just require people to recuse themselves from hiring decisions involving relatives? Particularly for early career people, I suspect most of these "quantitative" measurements are basically reflecting where and wtih whom you worked.
      - Fomite 10y ago
        There's also less nepotistic problems. For example, I've moved between different sides of the same overall field several times. That means, for any given reader of my CV, only about half the "impressive" papers read to them as impressive using most subjective measures.
      - JorgeGT 10y ago
        > Couldn't you just require people to recuse themselves from hiring decisions involving relatives? They used to have this nice trick where you hired the son of your friend and your friend hired your son. But at least now there are some non-subjective requirements to be met before anyone can get admitted.
    - angry_octet 10y ago
      While this is important, in many fields their are cliques of related researchers (1) on the grants committees and (2) on the journal committees. These people know who works on which problems, so blind review is a farce. Also, the concept of IF being a proxy for rigorous review and work quality is weak at best. Just read a few pages of http://retractionwatch.com/ http://retractionwatch.com/
- hannob 10y ago
  Impact factors and similar metrics are part of the problem. These are metrics that support flashy research, not good research. If anything the Impact Factor is a sign of sloppy research (there's a lot of evidence for that, e.g. the higher the IF the more likely there's a retraction).
  - JorgeGT 10y ago
    But it's not an easy problem. How do you grade from 0 to 10 good research from a candidate?
  - dagw 10y ago
    there's a lot of evidence for that, e.g. the higher the IF the more likely there's a retraction The higher the IF, the more likely it is the researcher trying to do something 'hard' and the more people will scrutinise the results. These two factors alone will lead to more retractions. Someone publishing something completely uninteresting in a journal no one reads will never be in a position to have to retract their paper, no matter how flawed, because no one will notice and no one will care.
- Fomite 10y ago
  But given the IF and ranks of OA journals, that's effectively the same thing. What you'd have to do is essentially give OA journals a handicap. Not to mention the opportunity cost associated with OA fees, which can add up if you're talking about lab startup.
a3_nm 10y ago
The problem is the following: if there is no established OA journal in the field, you can't distinguish the candidate who "deliberately" publishes good quality work in a non-selective OA journal, from the candidate who publishes low-quality work in the same non-selective OA journal. The only way to tell is to read the papers and judge for yourself, and this is the root of the problem: evaluating candidates based on the number of papers and the impact factor (or some analogous metric) of the venues is much, much easier for committees, than actually reading the papers.
- brownbat 10y ago
  In a very rough analogy, this is why we have film critics, because we can't trust other institutions in the industry (studios, individual theaters, directors) as a guarantee of quality. If a panel of respected tenured professors in the field ranked the best works in open access journals each month, or if there was a high profile yearly prize for best open access papers of the year, that could help disaggregate reputational effects from the publication itself. Bootstrapping this sort of thing is hard. I'm sure Nobels didn't signal as strongly in year one as they do now.
- jonathanstrange 10y ago
  Exactly, this nonsense of mostly just counting publications must stop. If at all, there is only a very light correlation (or perhaps even a negative one, who knows?) between the number of publication and the quality of a scientist's research. And what's so crazy about it is that everybody knows this. No one has ever been called a great scientist, because he published many articles. Such scientists may be called prolific, but that doesn't necessarily reflect on the quality of the content. The same for counting quotations, by the way. This may be different in STEM fields, but I'm in the humanities. In the humanities bad papers are sometimes quoted as often as brilliant papers, because everybody wants to correct the authors. I've seen more than one colleague literally make his career by writing a polemic and obviously wrong book (books=often improper peer review) and then sneaking a few apologetic articles in which they half-retracted their claims past the reviewers. What's worse, I know postdocs who are actively looking for such untenable topics that will surely stir up a reaction. We're talking about theses, conjectures or claims that nobody defended previously, because they are so obviously wrong or flawed. I've also met people at the postdoc level who literally replied "Who cares, it's one more publication" when I raised doubts about their main claim in an article and mentioned that it might not 'work' for purely formal reasons (but adding that I'd have to check, to be polite). I'm sure it's a bit less drastic in CS and the STEM fields, but in the humanities publication counting almost enforces mediocrity and the publication of uninteresting, pointless papers with vastly overlapping topics - and it's completely obvious that it does so. The situation is particularly frustrating, because the system makes it harder to publish good science. Good science needs larger studies and/or better formal underpinnings, both of which take more time and more effort than duplicating fake papers about fake problems. Well, enough ranting for today, you get the point.

dv_dt 10y ago

It is interesting to separate the review from the storage/access portions of scientific publishing. Bind the prestige to the review board, not the publisher. Once you have open access, there's nothing stopping academics from forming an independent review board organization which then provide some form of stamps of approval; paper A from It seems like it would be a good organizational excuse to stay in touch with peers and exchange information on the field anyway...

mattkrause 10y ago
Have you heard of Faculty Of 1000 (f1000.com)? It sorta does that. The "faculty" submit 1-2 paragraph blurbs describing papers. The blurbs often point out the strengths and weaknesses of each paper and give a brief description of how it fits into the broader literature. If you can find a couple of people on there with similar research interests, it can be a great way to get up to speed on something.

BeetleB 10y ago

>However, I'd also like a job and none of the "top tier" biomedical journals are open access. Isn't it a requirement that all NIH funded work must be published as open access?

JorgeGT 10y ago
Open, but not immediately, I believe. Journals have an embargo period of 1-3 years, and after that you can put the accepted paper on a free repository. Your manuscript can usually be posted for free as soon as you want.
Fomite 10y ago
It is, but it's after a time. It's not actually weakly enforced - we've been told we can't take credit for publications on grant reports if they're not in PMC.
- mattkrause 10y ago
  Don't you need to include the PMC ID number for each paper on the progress report/renewal? I vaguely remember doing something like this for our last one....
  - Fomite 10y ago
    Yep - you need the ID number, and if its not there, it doesn't count.
dekhn 10y ago
yes but it's only weakly enforced,and I believe there's an embargo peroid
mattkrause 10y ago
Not quite. You have to deposit a copy of the manuscript in PubMedCentral within a year of publication. Some journals let you deposit the final pdf (or even do it for you). Others restrict you to an unformatted "final draft" (i.e., the last thing you submitted to the journal). However, this is still tagged with the source journal, so it doesn't really get you around the career-related issues. Public Access guidelines: https://publicaccess.nih.gov/policy.htm https://publicaccess.nih.gov/policy.htm

JumpCrisscross 10y ago

> none of the "top tier" biomedical journals are open access Is this solely a product of network effects? Or could there be some truth to the top journals' ability to organise the review process?

mattkrause 10y ago
Pretty much. The review process seems to be about the same across most journals. In fact, there's probably a niche for some journals that have an actual editorial process where an editor actually...edits the prose. Nature/Science/Cell are popular because everyone recognizes their "brand" and associates them with interesting, high-impact science. Thus, it's easy for anyone "Hey, we should fund/hire her because she's got a bunch of Nature papers!" It's not miles from the old "Nobody got fired for buying IBM" bit. In contrast, it's unlikely that one person--let alone more-on the committee has carefully read your J. Soc. Whatever paper and says "This is well done! Get him instead!"
- ProblemFactory 10y ago
  It's not just a subjective well-known brand. Nature/Science/Cell have a prestigious brand, so many scientists try to get their best work published there. The journals get first pick of new research to choose from, and as a result getting published there really is a good indicator that your work is impactful and high quality. It's a network effect that can be broken only if enough funding & promotion committees, or top scientists in a field take a stand together. Something I have wondered though: What if we separate the "measure of quality and impact" from getting published in the journals? Something like "shadow journals", that doesn't publish and edit any articles itself. Its reviewers would pick the top 30 new papers published in their field each month, and just publish links to them. If this would get established, it could be used by funding and promotion committees, and the brand of the original journals would matter less.
- tnecniv 10y ago
  > Pretty much. The review process seems to be about the same across most journals. In fact, there's probably a niche for some journals that have an actual editorial process where an editor actually...edits the prose. Some journals do this. I've had PhD student friends inform me that when they've published to some (well-regarded) journals, after being accepted, they spent months going back and forth with the editor polishing the article.

antennas 10y ago

It's a prisoner's dilemma. If you (researcher) defect (publish in Elsevier journals), you perpetuate Elsevier's business, a shitty situation. But it's not as bad as cooperating (not publishing), because there's a chance that everyone else defects and you screw up your career, a really shitty situation. So out of self-interest everyone defects and perpetuates Elsevier's business.

mizzao 10y ago
More of a public goods problem (n-player PD).

jmnicholson 10y ago

This is a great point. We need to provide researchers with options that don't kill their career prospects. One idea, which we're working on at Authorea (authorea.com) is to help researchers write their documents online so they can be easily published as preprints. This way open access can be achieved in a manner that goes beyond a PDF or Word doc, and researchers can still play the game so long as it exists. Some more info here: https://www.authorea.com/123679/d8cErAb_olPC5CvcWWwsig https://www.authorea.com/123679/d8cErAb_olPC5CvcWWwsig

Certhas 10y ago

I think the best leverage is with the funding bodies. Declare that all publicly funded papers need to be OA, and this goes away in an instant. That said, these news made my life a lot easier right now. I was considering whether to accept an invitation to review for an Elsevier journal (which I've usually avoided). Now I'm fairly sure I will not.

pelario 10y ago

Agreeing with you, and also answering the GP: The issue I see is that "Academia" is not the same as "the scientists", there is a larger community, that includes institutions, processes, administrative people, etc. So, yes, for instance, I would love that we all published our papers in, say, arxive, an we were evaluated by the actual impact of our publications (~ PageRank). But that is not how the funding organizations, hiring committees, etc, are going to do it. How to do the transition from the current state of things to a better one, and not to commit career suicide? That is the question, and that is what those boycotts are aiming to, I believe.

tomjen3 10y ago

Sounds like we need to make it illegal to use government founds to publish in a non open access journal.

visarga 10y ago

> However, I'd also like a job and none of the "top tier" biomedical journals are open access Scientific domains with open publishing, like machine learning, have a very fast pace and amazing evolution. Won't biology as a field be affected severely by all this closeness? I think in the long run the whole field is suffering.

arthur2e5 10y ago

> particularly taxpayer-funded ones In the US there is a "Fair Access to Science and Technology Research Act of 2015" (S. 779, H.R.1477) that requires federal agencies to have a "public access policy" if they receive extramural research expenditures of over $100,000,000. https://act.eff.org/action/keep-publicly-funded-research-public https://act.eff.org/action/keep-publicly-funded-research-pub...