3 ms·
> At the standard p-value cutoff of .05, even when there's absolutely no effect going on the probability of getting a spurious positive result when you do n com
by Alex3917 11y ago
> At the standard p-value cutoff of .05, even when there's absolutely no effect going on the probability of getting a spurious positive result when you do n comparisons is equal to 1 - (.95^n).
That's only the probability of getting a positive result by chance; the total probability of getting a positive result even if there is no effect is much higher.
- rcthompson 11y ago"Probability of getting a positive result by chance" and "probability of getting a positive result when there is no effect" are the same thing. This is what the p-value measures. (The full phrase is generally a combination of the two: "probability of getting a positive result by chance when there is no effect"; but that's so long that people shorten it.)
- Alex3917 11y agoThere are lots of reasons you could get a false positive result that have nothing to do with chance: e.g. non-representative sample, miscalibrated equipment, biased methodology, researcher fraud, flawed statistical analysis, etc. C.f. http://blog.minitab.com/blog/adventures-in-statistics/how-to-correctly-interpret-p-values http://blog.minitab.com/blog/adventures-in-statistics/how-to...
- rcthompson 11y agoIndeed, but the comment that you quoted was referring to the case where the methodology and p-values are sound, and the only issue is testing multiple hypotheses without correction.