6 ms·

A journal dedicated to negative results is IMO not the solution here. Academics must publish in reputable, prestigious journals in order to advance their career

A journal dedicated to negative results is IMO not the solution here. Academics must publish in reputable, prestigious journals in order to advance their career (a publication in a non-prestigious journal basically counts for nothing). A journal dedicated to negative results is never going to be a prestigious, impactful, or widely-disseminated venue.

flukus 10y ago
Why is there no prestige associated with getting rid of rubbish? As a programmer the best feeling in the world is deleting code, reducing the cognitive load of the overall system without sacrificing it effectiveness. Does the academic world have the concept of technical debt?
- argonaut 10y ago
  One flaw in your analogy is that if all you do is actually just delete code (you don't replace the code), that's not usually valuable. It is easier to publish null results than it is to publish legitimate positive results (of course easiest of all is to publish bogus positive results). It doesn't require creativity or brilliance to publish null results, just competence (competence enough to replicate). Of course, it is still valuable, so a thoroughly analyzed and argued negative result should be publishable in prestigious venues - but this is more a reflection of the value to the community than the actual novelty/brilliance of the work. Put another way: a university is not going to hire you if all you've published are null results. They want leaders who are going to pioneer/discover/invent/theorize new fields.
  - bandrami 10y ago
    The Catholic church for centuries now has employed an advocatus Diaboli, someone whose entire job it is to examine claims of miracles and make the argument that in fact they are natural phenomena. They are priests who have been trained in science (there's a perhaps surprising number of those). I think this is a good thing. There should be somebody whose job it is to poke holes in the castles we build.
    - JorgeGT 10y ago
      > there's a perhaps surprising number of those The Vatican still runs an astronomical observatory that researches trans-neptunian objects, exoplanets and so on: http://www.vaticanobservatory.va/ http://www.vaticanobservatory.va/
      - bandrami 10y ago
        Similarly, an astronomer friend of mine trains Buddhist monks in physical astronomy. These are almost certainly good things.
    - argonaut 10y ago
      Yes. Those are peer reviewers. And people still publish negative result papers.
mcbits 10y ago
But that's a self-fulfilling prophecy. If more people were publishing their null results in such a journal, one of those journals would emerge as the most prestigious null-result journals. It seems like an acceptable solution, especially if the alternative is going to be not publishing at all and increasing the positive result bias.
- argonaut 10y ago
  Being the most prestigious null-result journal doesn't rise to being a prestigious journal, though. Regardless, this is the reality today. I don't think trying to boost a null-result journal to the level of Nature or Science is a better way forwards than pressuring Nature or Science (and similar caliber journals) to publish more null result works.
  - mcbits 10y ago
    As far as career advancement goes, I don't work in academia, but I would be more confident walking into an interview with a list of difficult and impactful experiments finding no effects than having even one highly cited paper in a prestigious journal that was later debunked by a couple of grad students in a null-result journal. The latter prospect should also frighten journal publishers into taking null results more seriously (especially if they published the earlier work), but only if the threat is real.
    - argonaut 10y ago
      Neither of the candidates you describe will even get a faculty interview. Reply to below: I think it's valid. The first researcher has not demonstrated they will be leader/pioneer/inventor/theorizer/discoverer of new research findings, if that's all they've done - this is what reputable research universities are looking for. The second researcher has, well, no findings at all.
      - mcbits 10y ago
        Another problem that needs solving. Edit: To clarify, I was also assuming some typical amount of other research credentials, as well. From Al-Khwarizmi's reply it sounds like the second candidate is likely to be the only one interviewed, which rules out the possibility of hiring the first candidate, the superior experimentalist.

flukus 10y ago

Why is there no prestige associated with getting rid of rubbish? As a programmer the best feeling in the world is deleting code, reducing the cognitive load of the overall system without sacrificing it effectiveness. Does the academic world have the concept of technical debt?

argonaut 10y ago
One flaw in your analogy is that if all you do is actually just delete code (you don't replace the code), that's not usually valuable. It is easier to publish null results than it is to publish legitimate positive results (of course easiest of all is to publish bogus positive results). It doesn't require creativity or brilliance to publish null results, just competence (competence enough to replicate). Of course, it is still valuable, so a thoroughly analyzed and argued negative result should be publishable in prestigious venues - but this is more a reflection of the value to the community than the actual novelty/brilliance of the work. Put another way: a university is not going to hire you if all you've published are null results. They want leaders who are going to pioneer/discover/invent/theorize new fields.
- bandrami 10y ago
  The Catholic church for centuries now has employed an advocatus Diaboli, someone whose entire job it is to examine claims of miracles and make the argument that in fact they are natural phenomena. They are priests who have been trained in science (there's a perhaps surprising number of those). I think this is a good thing. There should be somebody whose job it is to poke holes in the castles we build.
  - JorgeGT 10y ago
    > there's a perhaps surprising number of those The Vatican still runs an astronomical observatory that researches trans-neptunian objects, exoplanets and so on: http://www.vaticanobservatory.va/ http://www.vaticanobservatory.va/
    - bandrami 10y ago
      Similarly, an astronomer friend of mine trains Buddhist monks in physical astronomy. These are almost certainly good things.
  - argonaut 10y ago
    Yes. Those are peer reviewers. And people still publish negative result papers.

mcbits 10y ago

But that's a self-fulfilling prophecy. If more people were publishing their null results in such a journal, one of those journals would emerge as the most prestigious null-result journals. It seems like an acceptable solution, especially if the alternative is going to be not publishing at all and increasing the positive result bias.

argonaut 10y ago
Being the most prestigious null-result journal doesn't rise to being a prestigious journal, though. Regardless, this is the reality today. I don't think trying to boost a null-result journal to the level of Nature or Science is a better way forwards than pressuring Nature or Science (and similar caliber journals) to publish more null result works.
- mcbits 10y ago
  As far as career advancement goes, I don't work in academia, but I would be more confident walking into an interview with a list of difficult and impactful experiments finding no effects than having even one highly cited paper in a prestigious journal that was later debunked by a couple of grad students in a null-result journal. The latter prospect should also frighten journal publishers into taking null results more seriously (especially if they published the earlier work), but only if the threat is real.
  - argonaut 10y ago
    Neither of the candidates you describe will even get a faculty interview. Reply to below: I think it's valid. The first researcher has not demonstrated they will be leader/pioneer/inventor/theorizer/discoverer of new research findings, if that's all they've done - this is what reputable research universities are looking for. The second researcher has, well, no findings at all.
    - mcbits 10y ago
      Another problem that needs solving. Edit: To clarify, I was also assuming some typical amount of other research credentials, as well. From Al-Khwarizmi's reply it sounds like the second candidate is likely to be the only one interviewed, which rules out the possibility of hiring the first candidate, the superior experimentalist.