4 ms·
Yeah, that seems completely absurd. If the experiments are rigorous enough, and the p value is low enough, you should be able to trust the results, no matter ho
by Karellen 4y ago
Yeah, that seems completely absurd. If the experiments are rigorous enough, and the p value is low enough, you should be able to trust the results, no matter how small the effect is.
If, for example, research shows that social media use increases one's risk of depression by, say, between 32% and 34%, with p < 0.000001, they're going to discount a result that only has a 1-in-a-million chance of being a fluke simply because it can only show a 1/3 rise in risk, rather than a 3x rise?
That's not skepticism. That's dogma.
- lukeasrodgers 4y agoYes but in very large datasets, isn't it the case that you end up finding correlations for many things at very low p values? That is (one of the reasons) why effect size matters. Still, the "odds ratios are below 3" requirement seems excessive.
- Karellen 4y agoIf you're just looking for any correlations, then p-values aren't really applicable due to the sharpshooter fallacy. https://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy https://en.wikipedia.org/wiki/Texas_sharpshooter_fallacy You need a specific hypothesis to test before you start examining (or better, collecting) your data for a p-value to make any sense.
- mrbungie 4y agoOf course you should trust the results and publish them anyway (or so should happen in an ideal world ;) ), but they may be disregarded as uninteresting or unworthy in their domain when reviewed. P-value and effect sizes by themselves are not meaningful enough, you should always consider the full context of the experiment, subject of study and domain. Edit/PS: 3x or more is excessive though.