3 ms·
Just multiplying the P-values from different studies is invalid. You can intuit this by imagining 10000 studies, each of which had a P-value of 0.99. Seeing 10
by carbocation 14y ago
Just multiplying the P-values from different studies is invalid.
You can intuit this by imagining 10000 studies, each of which had a P-value of 0.99. Seeing 10000 negative studies and no positive studies, you'd be convinced that there was no real effect, intuitively. But multiplying 0.99 10000 times yields a P-value of 2.2 x 10^(-44), a result of impressive significance.
To combine studies, many people use inverse variance weighted meta-analysis, e.g., https://en.wikipedia.org/wiki/Inverse-variance_weighting https://en.wikipedia.org/wiki/Inverse-variance_weighting
- simonster 14y agoYou're right. There is a mathematical explanation for this in addition to the intuitive one you provide: If the null hypothesis is true, the p-value of each test is uniformly distributed between 0 and 1, but the product of p-values will not be uniformly distributed between 0 and 1, so it is not a p-value. To combine p-values, what we really want is the CDF of the product of p1 and p2 in the distribution formed by the product of uniform random variables. Mathematica can compute this; the general formula is (p1p2)(1-log(p1p2)), and for the two p-values in question this yields p = 0.0019 (which is close to what I said initially, but only because I made a calculation error; .02*.01 = 0.0002). Thanks for pointing out the mistake in my reasoning! I learned something.