4 ms·
Well, for one, because the standard for publication in many fields is “there’s a less-than-5% chance that we observed these effects because of coincidence”. Co
by zamfi 8y ago
Well, for one, because the standard for publication in many fields is “there’s a less-than-5% chance that we observed these effects because of coincidence”.
Combine that with people not publishing negative-results studies, and all it takes is someone doing 20 studies (or worse, 20 analyses on the same data set) in order to find that 1-in-20 chance occurring...by chance.
Of course, this has little to do with research itself and much more to do with the standards researchers hold each other to.
- viraptor 8y agoWhile you're right about the 5%-or-less chance, it's still a 5% chance we're aiming for. If we end up with 40% over a large number of papers, something went wrong. Not publishing the negative results shouldn't affect this number. They're not included in the 5% chance in the first place.
- evandijk70 8y agoThis is a common misunderstanding of p-values. There is a 5% chance of observing the effect, if the effect is not there. I think the difference is best illustrated by an example: If you have 20 researchers investigating a hypothesis that turns out to be false 1 of them will report evidence for a false hypothesis, which will likely not replicate. Thus, if for every true hypotheses investigated, 20 false hypotheses are investigated, and the 19 researchers with false hypotheses do not report their result, that means that 1/2 of the reported positive results will not replicate.
- coldtea 8y agoThat's a cyclical argument though to what the parent asked though. "Research is OK to have ac40% non-replication rate" because "the standard for publication in many fields is low"