8 ms·

Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. This may be less accentuated in computer science

Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. This may be less accentuated in computer science, where most research is presented at conferences and preprints are often available on the arXiv, but it's a very important factor in many other disciplines. It's also the primary reason why researchers don't boycott these journals: most of them are simply not in a position where they can afford to without seriously damaging their career prospects. See also https://en.wikipedia.org/wiki/Price_of_anarchy https://en.wikipedia.org/wiki/Price_of_anarchy - a system can get stuck in a terrible state where everyone's best move is to keep going. A much better equilibrium could be constructed (Price of Stability), but how do we get there?

heavenlyblue 9y ago
You don't have to boycott anything - if a university starts a new journal and their respective academics start publishing with it they would have a public plausible deniability on why they're also publishing in this journal. That's a no-loss situation.
- matt4077 9y ago
  The problem isn't that you're somehow branded a 'rebel' if you refuse to publish with Elsevier journals. It's that, when you're applying somewhere, they'll rank you by SUM(paper_i * impact_factor(journal_i)). That score will suffer, and nobody will care about your reasons. You're also underestimating the independence of researches at a university. They're not employees in the usual sense. In fact, the best method to stop them from publishing at some journal Y would be for the administration to tell them to publish in Y.
  - codesnik 9y ago
    that easy? "You should publish only in Elsevier journals!"
  - tankenmate 9y ago
    Also, in some counties (the UK is a prime example) the rules governing "impact" is largely dictated and/or influenced by central government. So not only do you need to change just your local institution but a whole country's institutions at a go; a multi-year political fiefdom protecting shambles at best.
    - sleavey 9y ago
      That "impact" scoring system is utterly useless. I hope it doesn't survive the next non-Tory government.
  - heavenlyblue 9y ago
    See [https://news.ycombinator.com/item?id=15007958 https://news.ycombinator.com/item?id=15007958]. Their impact factor is now 2.450 vs 1.848. Of course I am sure it's way harder to do this for far more conservative and larger industries. At the same time it may be just the case of starting small.
- dragandj 9y ago
  You mean a new journal for every scientific field? That's helluva lot of journals. Then, you need to attract people to send the papers there. There are countless local journals that mainly attract local authors now, and those are a joke when they are not outright junk. To publish even a half decent journal, you need to attract researchers from all corners of the world to send you their best work. How?
  - heavenlyblue 9y ago
    Isn't there already a journal for every scientific field, or are you speaking of only a part of the industries?
- perilunar 9y ago
  Many papers have multiple authors from different universities/institutions. It would be difficult if just one was boycotting the top journals in the field.
- rocqua 9y ago
  Those academics would still have less-prestigious publications, which is a net-loss for both them and their university. A solution might be dual publishing, but I believe some of the bigger journals require some form of exclusivity.
- mjn 9y ago
  If done collectively that has worked before. In 2001, almost the entire editorial board of the Springer journal Machine Learning resigned [1] in order to lend their support to a new open-access journal, Journal of Machine Learning Research, which quickly supplanted the former as the top journal in the field. It helped that this included a lot of senior people in the field (Stuart Russell, Geoffrey Hinton, Leslie Kaelbling, etc.) who would be the ones judging ML hiring and tenure cases at many universities. [1] Resignation letter: http://www.jmlr.org/statement.html http://www.jmlr.org/statement.html
  - Vinnl 9y ago
    That only works if the editorial board "flips", though - unfortunately, universities mostly aren't able to start journals with reputable editorial boards. I'm currently leaning towards finding a way to convince large numbers of boards to flip as the most viable path to proper open access.
Aaargh20318 9y ago
> Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. This is very weird to me. It seems to me like a variant of the 'argument from authority' fallacy. Publications should be judged on their content, not on who owns the printing press.
- pfortuny 9y ago
  Oh, yes. The problem is not rasearch, the problem is the Administration and its permanent quest for "objective" (whatever that means) rating criteria...
- ptero 9y ago
  If I'm reading a paper in Nature I can assume that the editors and reviewers did first checks on content (BS can still get through, but it is rare). Self-published, unknown journal gives me none of that. Thus when making hiring decisions it does help to overweight established journals. And most tenure tracks in academia make mistakes costly. I do not like the current system, but established, well curated journals do provide some benefits in academia hiring. They do damage, too, but to a different group. My 2c.
  - lucasverra 9y ago
    Would it a good use case for a blockchain validation system ? Decentralized (scientific) content reviewer ? Pay scarce resource (attention odf the well educated/trained scientist) with some new coin "SCI-coin" ?
  - bobcostas55 9y ago
    >BS can still get through, but it is rare The data[0] suggests the exact opposite: the more prestigious the journal, the more bad science it attracts. [0] http://bjoern.brembs.net/2016/01/even-without-retractions-top-journals-publish-the-least-reliable-science/ http://bjoern.brembs.net/2016/01/even-without-retractions-to...
    - Ar-Curunir 9y ago
cabalamat 9y ago
> Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. Maybe there could be a policy of only counting articles in open-access journals. > It's also the primary reason why researchers don't boycott these journals: most of them are simply not in a position where they can afford to without seriously damaging their career prospects. Then that's where the log-jam is.
- Vinnl 9y ago
  > Maybe there could be a policy of only counting articles in open-access journals. Unfortunately it's not a policy, it's the easy option for those evaluating scientist, which is why they do it. Sometimes there's a policy to ignore the reputation (or actually, the flawed metric that's supposed to represent it called "Impact Factor"), but that's not widespread. Academics unfortunately have more to concern themselves with than spreading Open Access.
IshKebab 9y ago
> the reputation of journals where applicants published Yes which is why ordinary individuals can't just start a journal. But governments and universities come with already-established reputation. If Cambridge University started their own open access biology journal (or whatever) and encouraged their scientists to contribute I don't think it would be hard to imagine it succeeding. Have any universities actually tried?
- lorenzhs 9y ago
  To have a successful journal, you don't need a particular university's researchers to publish in it, even if it's a very reputable university. You need everybody working in the field wanting to publish there. All the supposedly really simple solutions suggested in this thread don't work. It's not like academics haven't thought about it.
  - IshKebab 9y ago
    Sure, that's why you get a particular university's researchers to publish in it. That's exactly how you start making it so that everyone in the field wants to publish there. They may have thought about it but has anyone actually tried it? I'm genuinely asking. Edit: It seems they have and it worked. See mjn's answer below.
- Vinnl 9y ago
  Their scientists have careers that span beyond those universities, so "encouraging them to contribute" is easier said than done. (Also, their scientists have very diverse disciplines and thus many, many different journals to contribute to - so each individual journal still wouldn't have enough good scientists contributing to carry them.)
jancsika 9y ago
So why not divide and conquer? 1. University-scope undergrad journals 2. University-scope grad journals 3. University-scope science journals 4. University-scope humanities journals 5. Regional-scope science journals 6. Regional-scope humanities journals 7. Nation-scope obscure science journals 8. Nation-scope obscure humanities journals 9. Nation-scope science journals. 10. Nation-scope humanities journals. 11. Nature. You start with the people who haven't been educated to rationalize the racket, in journals that have the least amount of reputability in the first place. (Ignoring fake journals, journal mills, etc.) They are also more likely to understand the underlying technology and probably do a lot of the implementation work, too. Same for grads. Then you get the journals that only function within the scope of a single university-- i.e., they are used to publish and gain notoriety within the university but are of only limited value outside of it. Probably best to start with a high-fallutin' uni whose stature could be used to convinced them the publicity of being the first mover is reason enough for the risk. Then regional journals, where again the notoriety is relatively low and personal relationships are leveraged more for notoriety/credibility. (For example, an anthropological journal that covers a small region.) Because these academic communities probably already can recite the names of all the current scholars in the field, moving the research content isn't as disruptive. In each case the science journals should go first. Humanities have a natural envy of science's verifiability/falsifiability, so they'll quickly follow whatever the scientists do. Edit: clarification

heavenlyblue 9y ago

You don't have to boycott anything - if a university starts a new journal and their respective academics start publishing with it they would have a public plausible deniability on why they're also publishing in this journal. That's a no-loss situation.

matt4077 9y ago
The problem isn't that you're somehow branded a 'rebel' if you refuse to publish with Elsevier journals. It's that, when you're applying somewhere, they'll rank you by SUM(paper_i * impact_factor(journal_i)). That score will suffer, and nobody will care about your reasons. You're also underestimating the independence of researches at a university. They're not employees in the usual sense. In fact, the best method to stop them from publishing at some journal Y would be for the administration to tell them to publish in Y.
- codesnik 9y ago
  that easy? "You should publish only in Elsevier journals!"
- tankenmate 9y ago
  Also, in some counties (the UK is a prime example) the rules governing "impact" is largely dictated and/or influenced by central government. So not only do you need to change just your local institution but a whole country's institutions at a go; a multi-year political fiefdom protecting shambles at best.
  - sleavey 9y ago
    That "impact" scoring system is utterly useless. I hope it doesn't survive the next non-Tory government.
- heavenlyblue 9y ago
  See [https://news.ycombinator.com/item?id=15007958 https://news.ycombinator.com/item?id=15007958]. Their impact factor is now 2.450 vs 1.848. Of course I am sure it's way harder to do this for far more conservative and larger industries. At the same time it may be just the case of starting small.
dragandj 9y ago
You mean a new journal for every scientific field? That's helluva lot of journals. Then, you need to attract people to send the papers there. There are countless local journals that mainly attract local authors now, and those are a joke when they are not outright junk. To publish even a half decent journal, you need to attract researchers from all corners of the world to send you their best work. How?
- heavenlyblue 9y ago
  Isn't there already a journal for every scientific field, or are you speaking of only a part of the industries?
perilunar 9y ago
Many papers have multiple authors from different universities/institutions. It would be difficult if just one was boycotting the top journals in the field.
rocqua 9y ago
Those academics would still have less-prestigious publications, which is a net-loss for both them and their university. A solution might be dual publishing, but I believe some of the bigger journals require some form of exclusivity.
mjn 9y ago
If done collectively that has worked before. In 2001, almost the entire editorial board of the Springer journal Machine Learning resigned [1] in order to lend their support to a new open-access journal, Journal of Machine Learning Research, which quickly supplanted the former as the top journal in the field. It helped that this included a lot of senior people in the field (Stuart Russell, Geoffrey Hinton, Leslie Kaelbling, etc.) who would be the ones judging ML hiring and tenure cases at many universities. [1] Resignation letter: http://www.jmlr.org/statement.html http://www.jmlr.org/statement.html
- Vinnl 9y ago
  That only works if the editorial board "flips", though - unfortunately, universities mostly aren't able to start journals with reputable editorial boards. I'm currently leaning towards finding a way to convince large numbers of boards to flip as the most viable path to proper open access.

Aaargh20318 9y ago

> Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. This is very weird to me. It seems to me like a variant of the 'argument from authority' fallacy. Publications should be judged on their content, not on who owns the printing press.

pfortuny 9y ago
Oh, yes. The problem is not rasearch, the problem is the Administration and its permanent quest for "objective" (whatever that means) rating criteria...
ptero 9y ago
If I'm reading a paper in Nature I can assume that the editors and reviewers did first checks on content (BS can still get through, but it is rare). Self-published, unknown journal gives me none of that. Thus when making hiring decisions it does help to overweight established journals. And most tenure tracks in academia make mistakes costly. I do not like the current system, but established, well curated journals do provide some benefits in academia hiring. They do damage, too, but to a different group. My 2c.
- lucasverra 9y ago
  Would it a good use case for a blockchain validation system ? Decentralized (scientific) content reviewer ? Pay scarce resource (attention odf the well educated/trained scientist) with some new coin "SCI-coin" ?
- bobcostas55 9y ago
  >BS can still get through, but it is rare The data[0] suggests the exact opposite: the more prestigious the journal, the more bad science it attracts. [0] http://bjoern.brembs.net/2016/01/even-without-retractions-top-journals-publish-the-least-reliable-science/ http://bjoern.brembs.net/2016/01/even-without-retractions-to...
  - Ar-Curunir 9y ago

cabalamat 9y ago

> Many hiring decisions in academia still rely strongly on the reputation of journals where applicants published. Maybe there could be a policy of only counting articles in open-access journals. > It's also the primary reason why researchers don't boycott these journals: most of them are simply not in a position where they can afford to without seriously damaging their career prospects. Then that's where the log-jam is.

Vinnl 9y ago
> Maybe there could be a policy of only counting articles in open-access journals. Unfortunately it's not a policy, it's the easy option for those evaluating scientist, which is why they do it. Sometimes there's a policy to ignore the reputation (or actually, the flawed metric that's supposed to represent it called "Impact Factor"), but that's not widespread. Academics unfortunately have more to concern themselves with than spreading Open Access.

IshKebab 9y ago

> the reputation of journals where applicants published Yes which is why ordinary individuals can't just start a journal. But governments and universities come with already-established reputation. If Cambridge University started their own open access biology journal (or whatever) and encouraged their scientists to contribute I don't think it would be hard to imagine it succeeding. Have any universities actually tried?

lorenzhs 9y ago
To have a successful journal, you don't need a particular university's researchers to publish in it, even if it's a very reputable university. You need everybody working in the field wanting to publish there. All the supposedly really simple solutions suggested in this thread don't work. It's not like academics haven't thought about it.
- IshKebab 9y ago
  Sure, that's why you get a particular university's researchers to publish in it. That's exactly how you start making it so that everyone in the field wants to publish there. They may have thought about it but has anyone actually tried it? I'm genuinely asking. Edit: It seems they have and it worked. See mjn's answer below.
Vinnl 9y ago
Their scientists have careers that span beyond those universities, so "encouraging them to contribute" is easier said than done. (Also, their scientists have very diverse disciplines and thus many, many different journals to contribute to - so each individual journal still wouldn't have enough good scientists contributing to carry them.)

jancsika 9y ago

So why not divide and conquer? 1. University-scope undergrad journals 2. University-scope grad journals 3. University-scope science journals 4. University-scope humanities journals 5. Regional-scope science journals 6. Regional-scope humanities journals 7. Nation-scope obscure science journals 8. Nation-scope obscure humanities journals 9. Nation-scope science journals. 10. Nation-scope humanities journals. 11. Nature. You start with the people who haven't been educated to rationalize the racket, in journals that have the least amount of reputability in the first place. (Ignoring fake journals, journal mills, etc.) They are also more likely to understand the underlying technology and probably do a lot of the implementation work, too. Same for grads. Then you get the journals that only function within the scope of a single university-- i.e., they are used to publish and gain notoriety within the university but are of only limited value outside of it. Probably best to start with a high-fallutin' uni whose stature could be used to convinced them the publicity of being the first mover is reason enough for the risk. Then regional journals, where again the notoriety is relatively low and personal relationships are leveraged more for notoriety/credibility. (For example, an anthropological journal that covers a small region.) Because these academic communities probably already can recite the names of all the current scholars in the field, moving the research content isn't as disruptive. In each case the science journals should go first. Humanities have a natural envy of science's verifiability/falsifiability, so they'll quickly follow whatever the scientists do. Edit: clarification