5 ms·

But that's a self-fulfilling prophecy. If more people were publishing their null results in such a journal, one of those journals would emerge as the most prest

by mcbits 10y ago

But that's a self-fulfilling prophecy. If more people were publishing their null results in such a journal, one of those journals would emerge as the most prestigious null-result journals. It seems like an acceptable solution, especially if the alternative is going to be not publishing at all and increasing the positive result bias.

argonaut 10y ago
Being the most prestigious null-result journal doesn't rise to being a prestigious journal, though. Regardless, this is the reality today. I don't think trying to boost a null-result journal to the level of Nature or Science is a better way forwards than pressuring Nature or Science (and similar caliber journals) to publish more null result works.
- mcbits 10y ago
  As far as career advancement goes, I don't work in academia, but I would be more confident walking into an interview with a list of difficult and impactful experiments finding no effects than having even one highly cited paper in a prestigious journal that was later debunked by a couple of grad students in a null-result journal. The latter prospect should also frighten journal publishers into taking null results more seriously (especially if they published the earlier work), but only if the threat is real.
  - argonaut 10y ago
    Neither of the candidates you describe will even get a faculty interview. Reply to below: I think it's valid. The first researcher has not demonstrated they will be leader/pioneer/inventor/theorizer/discoverer of new research findings, if that's all they've done - this is what reputable research universities are looking for. The second researcher has, well, no findings at all.
    - mcbits 10y ago
      Another problem that needs solving. Edit: To clarify, I was also assuming some typical amount of other research credentials, as well. From Al-Khwarizmi's reply it sounds like the second candidate is likely to be the only one interviewed, which rules out the possibility of hiring the first candidate, the superior experimentalist.
      - deleted 10y ago
        [deleted]
      - argonaut 10y ago
        If they have a good research portfolio otherwise, then the first candidate's chances are unaffected, and the second candidate's chances at an interview are probably reduced. The first candidate's replication studies won't really count for anything (maybe a slight boost). The second candidate's bogus studies will count negatively. The process he describes is not reflective of US research university faculty hiring practices - the faculty will read your papers, they will solicit expert opinions from other leading researchers in the same research areas as you.
  - Al-Khwarizmi 10y ago
    Most assessment processes in academia are purely administrative processes where they count the number of cookies that you have earned. They don't take the actual quality of the work into account. In my country, it goes as far as having "objective" point scales like: publication in ISI JCR-indexed journals, 1 point for Q4, 2 points for Q3, 3 points for Q2, 5 points for Q1. Publication in non-indexed journal, 0.1 points. No one ever looks into whether the paper is good or crap, or whether the author has managed to submit almost the same paper to five journals. In fact, I know cases of honest people that tried to look into that kind of thing, but they didn't let them because it was against the published "objective" scale. I don't think the system is equally rotten in every country/institution, my country is probably among the most extreme places in this respect, but anyway this kind of assessment is the most common AFAICT, as the impact factor cult is often denounced by international scientific societies.
    - argonaut 10y ago
      This does not reflect hiring practices at reputable US research universities.
  - Aeolun 10y ago
    Chances of those grad students being correct are slim. Since you were published in this hugely impactful prestigious journal that thousands of people have read, and are debunked by those grads in a journal that nobody reads, no-one is even aware that it happened.

by mcbits 10y ago

argonaut 10y ago

Being the most prestigious null-result journal doesn't rise to being a prestigious journal, though. Regardless, this is the reality today. I don't think trying to boost a null-result journal to the level of Nature or Science is a better way forwards than pressuring Nature or Science (and similar caliber journals) to publish more null result works.

mcbits 10y ago
As far as career advancement goes, I don't work in academia, but I would be more confident walking into an interview with a list of difficult and impactful experiments finding no effects than having even one highly cited paper in a prestigious journal that was later debunked by a couple of grad students in a null-result journal. The latter prospect should also frighten journal publishers into taking null results more seriously (especially if they published the earlier work), but only if the threat is real.
- argonaut 10y ago
  Neither of the candidates you describe will even get a faculty interview. Reply to below: I think it's valid. The first researcher has not demonstrated they will be leader/pioneer/inventor/theorizer/discoverer of new research findings, if that's all they've done - this is what reputable research universities are looking for. The second researcher has, well, no findings at all.
  - mcbits 10y ago
    Another problem that needs solving. Edit: To clarify, I was also assuming some typical amount of other research credentials, as well. From Al-Khwarizmi's reply it sounds like the second candidate is likely to be the only one interviewed, which rules out the possibility of hiring the first candidate, the superior experimentalist.
    - deleted 10y ago
      [deleted]
    - argonaut 10y ago
      If they have a good research portfolio otherwise, then the first candidate's chances are unaffected, and the second candidate's chances at an interview are probably reduced. The first candidate's replication studies won't really count for anything (maybe a slight boost). The second candidate's bogus studies will count negatively. The process he describes is not reflective of US research university faculty hiring practices - the faculty will read your papers, they will solicit expert opinions from other leading researchers in the same research areas as you.
- Al-Khwarizmi 10y ago
  Most assessment processes in academia are purely administrative processes where they count the number of cookies that you have earned. They don't take the actual quality of the work into account. In my country, it goes as far as having "objective" point scales like: publication in ISI JCR-indexed journals, 1 point for Q4, 2 points for Q3, 3 points for Q2, 5 points for Q1. Publication in non-indexed journal, 0.1 points. No one ever looks into whether the paper is good or crap, or whether the author has managed to submit almost the same paper to five journals. In fact, I know cases of honest people that tried to look into that kind of thing, but they didn't let them because it was against the published "objective" scale. I don't think the system is equally rotten in every country/institution, my country is probably among the most extreme places in this respect, but anyway this kind of assessment is the most common AFAICT, as the impact factor cult is often denounced by international scientific societies.
  - argonaut 10y ago
    This does not reflect hiring practices at reputable US research universities.
- Aeolun 10y ago
  Chances of those grad students being correct are slim. Since you were published in this hugely impactful prestigious journal that thousands of people have read, and are debunked by those grads in a journal that nobody reads, no-one is even aware that it happened.