12 ms·
Negative findings are just as good to publish so people don’t repeat mistake twice
by m3kw9 8y ago
Negative findings are just as good to publish so people don’t repeat mistake twice
- roywiggins 8y agoNegative findings don't prove that the idea was a mistake- it's just failure to find sufficient evidence. You might have just been unlucky.
- nneonneo 8y agoMeta-analyses based on lots of studies, though, benefit from having unbiased publication of study results.
- jfz 8y agoThis is critical - even if no individual study has positive results, the data from each study can be aggregated and that meta-analysis can find results much more easily. I remember reading (but can't find the link to) an article about a medicine that took many trials to have one trial with statistical significance, but a meta-analysis of the first five or so trials would have revealed the efficacy of the drug under test much sooner and at a much lower cost.
- tmalsburg2 8y agoMeta-analyses suffer from publication bias, though. Studies with one outcome are more likely to be published than those with some other outcome, and therefore the input for the meta-anslysis is biased.
- gus_massa 8y agoThe problem is that it is too easy to have wrong ideas. If there only metric is how many studies you (or your group) have published, some will try to game it.
- rtpg 8y agoIn domains like chemistry, you hae this research process of "try every combo of material x process and see what you get". Hopefully detailed null results will avoid too many people going down the same path. But even more hopefully, it will let other researchers read the process and perhaps find another way to actually get to success. "We tried A x B in this way, that way, some other way, none of them worked" is pretty valuable info. Also pretty good when it comes to confirmation of theoretical work
- adtac 8y ago>We tried A x B in this way, that way, some other way, none of them worked How searchable is this data? Like, do I need to be an expert who is up-to-date on most proceedings in the subfield to know this, or is this information easy to pull up with a few searches?
- duxup 8y ago" it is too easy to have wrong ideas." I would be so published all the time...
- oldgradstudent 8y agoNot all negative results are created equal. It is very easy to run a terrible study. Even with a good idea, it is very easy to run a terrible study.
- solomatov 8y agoA possible problem I see is that it's hard to distinguish truly bad idea from a low quality implementation of a good one.
- tmalsburg2 8y agoA negative result is not necessarily the result of a mistake. Null results can be as important for the progress of science as positive findings.
- WhompingWindows 8y ago"Negative" findings or "null" findings don't indicate a mistake. Let's say I'm studying whether or not a certain vitamin, lets say Vitamin HN, helps prevent the common cold. The null hypothesis would be that it does not help prevent the common cold, as most things do not and that's where we must assume our Vitamin HN will fall. If we find that it doesn't actually prevent cold, that is not a mistake, that is simply the calculation of a lack of treatment effect difference in two groups of a sample of a population. There is no "mistake" inherent to null vs. confirmed hypotheses.