3 ms·

EXTREMELY small study. Just push for a larger study on this http://www.academia.edu/2057894/The_common_pain_of_surrealism_and_death_Acetaminophen_reduces_compe

by gte910h 14y ago

EXTREMELY small study. Just push for a larger study on this http://www.academia.edu/2057894/The_common_pain_of_surrealism_and_death_Acetaminophen_reduces_compensatory_affirmation_following_meaning_threats http://www.academia.edu/2057894/The_common_pain_of_surrealis... 121 people Search for "We recruited"

simonster 14y ago
In Study 2 there we 236 subjects, which AFAIK is a pretty big sample size for these kinds of experiments. There error bars on the graph are still somewhat large, so there was a lot of variability, but assuming there is no bias in what data the experimenters chose to include, combining both studies, p < .02*.01 = .002. Furthermore, as the article notes, there was a prior study that also showed cognitive effects of Tylenol. While these effects could be spurious due to publication or selection bias or deliberate manipulation, it's not all that plausible that they happened by chance.
- carbocation 14y ago
  Just multiplying the P-values from different studies is invalid. You can intuit this by imagining 10000 studies, each of which had a P-value of 0.99. Seeing 10000 negative studies and no positive studies, you'd be convinced that there was no real effect, intuitively. But multiplying 0.99 10000 times yields a P-value of 2.2 x 10^(-44), a result of impressive significance. To combine studies, many people use inverse variance weighted meta-analysis, e.g., https://en.wikipedia.org/wiki/Inverse-variance_weighting https://en.wikipedia.org/wiki/Inverse-variance_weighting
  - simonster 14y ago
    You're right. There is a mathematical explanation for this in addition to the intuitive one you provide: If the null hypothesis is true, the p-value of each test is uniformly distributed between 0 and 1, but the product of p-values will not be uniformly distributed between 0 and 1, so it is not a p-value. To combine p-values, what we really want is the CDF of the product of p1 and p2 in the distribution formed by the product of uniform random variables. Mathematica can compute this; the general formula is (p1p2)(1-log(p1p2)), and for the two p-values in question this yields p = 0.0019 (which is close to what I said initially, but only because I made a calculation error; .02*.01 = 0.0002). Thanks for pointing out the mistake in my reasoning! I learned something.

by gte910h 14y ago

simonster 14y ago

In Study 2 there we 236 subjects, which AFAIK is a pretty big sample size for these kinds of experiments. There error bars on the graph are still somewhat large, so there was a lot of variability, but assuming there is no bias in what data the experimenters chose to include, combining both studies, p < .02*.01 = .002. Furthermore, as the article notes, there was a prior study that also showed cognitive effects of Tylenol. While these effects could be spurious due to publication or selection bias or deliberate manipulation, it's not all that plausible that they happened by chance.

carbocation 14y ago
Just multiplying the P-values from different studies is invalid. You can intuit this by imagining 10000 studies, each of which had a P-value of 0.99. Seeing 10000 negative studies and no positive studies, you'd be convinced that there was no real effect, intuitively. But multiplying 0.99 10000 times yields a P-value of 2.2 x 10^(-44), a result of impressive significance. To combine studies, many people use inverse variance weighted meta-analysis, e.g., https://en.wikipedia.org/wiki/Inverse-variance_weighting https://en.wikipedia.org/wiki/Inverse-variance_weighting
- simonster 14y ago
  You're right. There is a mathematical explanation for this in addition to the intuitive one you provide: If the null hypothesis is true, the p-value of each test is uniformly distributed between 0 and 1, but the product of p-values will not be uniformly distributed between 0 and 1, so it is not a p-value. To combine p-values, what we really want is the CDF of the product of p1 and p2 in the distribution formed by the product of uniform random variables. Mathematica can compute this; the general formula is (p1p2)(1-log(p1p2)), and for the two p-values in question this yields p = 0.0019 (which is close to what I said initially, but only because I made a calculation error; .02*.01 = 0.0002). Thanks for pointing out the mistake in my reasoning! I learned something.