3 ms·

Need to start incentivising negative results, reproducing results and even finding no results at all. It should be possible to publish to some kind of database

Need to start incentivising negative results, reproducing results and even finding no results at all. It should be possible to publish to some kind of database a data analysis with methods and data even if you don't find anything, because then maybe other scientists could look at it and say "ok, this seem to not yield any result, but I could try to change it" adn he either finds it ant publishes, or doesn't find and publish anyway, so we start to get a collection of results and be much more skeptical when one of 8 studies, for example, finds effect of p barely over 0.005.

bambax 11y ago
Yes. If journals are biased against negative results, then governments should move in. Why do we need "journals" with the implication of a limited page count per issue anyway; in this digital age a database or wikipedia-style format should be able to host any number of results. And it should not mater that some governments won't fund suchan initiative: all is needed is one, or a single benefactor, or maybe a tv-show à la Mythbusters. ResearchBusters would be a hell of a show!!
- thinkling 11y ago
  I recommend reading responses to other branches of the grandparent that address this. I agree that if people do decent studies and get null results, it would be nice if the data became publicly available so further studies can be seen in the context of previous negative results. But given the need for novelty in results, I wonder how much work actually duplicates work previously undertaken with a null result. I suspect it's much less than many people in this discussion are assuming. And secondly, there's the issue of forcing people to do decent write-ups. Study results are useless without a very detailed description of both the experimental setup and the data collection/analysis procedures that were followed. Those write-ups take time and effort, and for an unpublishable result many researchers won't find the motivation. One could argue that funding agencies and institutional IRBs must force researchers to write much of this ahead of running the experiment. That might help a bit, but will still leave substantial holes.

bambax 11y ago

Yes. If journals are biased against negative results, then governments should move in. Why do we need "journals" with the implication of a limited page count per issue anyway; in this digital age a database or wikipedia-style format should be able to host any number of results. And it should not mater that some governments won't fund suchan initiative: all is needed is one, or a single benefactor, or maybe a tv-show à la Mythbusters. ResearchBusters would be a hell of a show!!

thinkling 11y ago
I recommend reading responses to other branches of the grandparent that address this. I agree that if people do decent studies and get null results, it would be nice if the data became publicly available so further studies can be seen in the context of previous negative results. But given the need for novelty in results, I wonder how much work actually duplicates work previously undertaken with a null result. I suspect it's much less than many people in this discussion are assuming. And secondly, there's the issue of forcing people to do decent write-ups. Study results are useless without a very detailed description of both the experimental setup and the data collection/analysis procedures that were followed. Those write-ups take time and effort, and for an unpublishable result many researchers won't find the motivation. One could argue that funding agencies and institutional IRBs must force researchers to write much of this ahead of running the experiment. That might help a bit, but will still leave substantial holes.