3 ms·
I don't think people understand how bad the third one is. If 100 labs try an idea and only one publishes, then clearly we can get a spurious p-value of .01. But
by syrrim 7y ago
I don't think people understand how bad the third one is. If 100 labs try an idea and only one publishes, then clearly we can get a spurious p-value of .01. But what if 10 labs try it, and 7 publish their non-significant results. Then, a meta-analysis takes the results of those 7 studies and combines them together. Here the meta-analysis can get a spurious p-value of .01 with an order of magnitude less original studies and with most of them publishing their results. The culture of selective publishing and of parading any significant result no matter how small the effect size is incompatible with correct science.
- FabHK 7y agoHow would a meta-analysis wrongly arrive at a spurious p=0.01 when 7 of 10 labs publish their (non-significant) results?