3 ms·
I disagree. > a bias toward publishing positive results ... fishing for significant-looking results from a huge number of hypotheses If we accept a lower p-va
by mabbo 9y ago
I disagree.
> a bias toward publishing positive results ... fishing for significant-looking results from a huge number of hypotheses
If we accept a lower p-value as significant, there will be fewer hypotheses that have positive results, and therefore less to publish.
Should we really do nothing at all until we can come up with a perfect solution to all aspects of the problem? This won't fix the real problem, but it will reduce the number of pop-sci articles spouting bullshit results that don't reproduce next week.
- shmageggy 9y ago> it will reduce the number of pop-sci articles spouting bullshit results that don't reproduce next week. I agree that doing something is better than doing nothing, but I disagree that this will reduce bad results in a meaningful way. Setting a more stringent threshold for significance will elimnate studies that reach the wrong conclusion due to a statistical fluke, i.e. there's no effect but due to random chance, the noise in the data makes it look like there is. But IMO these are relatively rare. It seems to me that the vast majority of bad studies reach significance by virtue of methodological issues that enable one to reach any arbitrary level of significance just by trying hard enough. If Daryl Bem continued doing what he was doing, he would have eventually reached p<0.005. edit: I just glanced over Bem's paper and he reports several p-values under 0.005 and several other very close to it.