3 ms·
Imagine that your sample of kids in Kent are actually 5mm smaller on average than your samples from the 9 other counties. The question to then ask is: how likel
by stdbrouw 12y ago
Imagine that your sample of kids in Kent are actually 5mm smaller on average than your samples from the 9 other counties. The question to then ask is: how likely is it that this would happen purely due to chance? After all, even a fair die will occasionally throw tens or even hundreds of sixes in row. Likewise, your Kent sample might just be a bit odd.
A p-value answers this question: imagine that there's no gravitational effect, how often would your sample of children in Kent end up 5mm smaller anyway due to chance? In most cases, the lower this number, the better, because it means something other than chance might be at play. If your p-value were 0.95, you'd conclude that even though the sample of Kent kids is smaller, this would have happened purely due to chance 95 out of a 100 times, so there's not likely to be a real difference.
Of course, with samples as large as those you mention, your p-value is pretty much guaranteed to be low, and hence you're guaranteed to have a statistically significant effect. It's probably easier to understand in the context of much smaller sample sizes: if you play three tennis matches against a friend, and he wins 2 out of 3, what's the chances he's actually better than you, and what are the chances that he just got lucky this time? Turns out, it's 50/50. That's what a p-value is trying to get at.
Which is one of the big reasons why p-values get so much (deserved) hate: they're influenced heavily by both effect size and sample size, so you can't tell one from the other.