4 ms·
Yeah, my point isn't that the statistical significance is erroneous or that the difference shouldn't have been significant. In my totally made up example, ther
by glaugh 13y ago
Yeah, my point isn't that the statistical significance is erroneous or that the difference shouldn't have been significant.
In my totally made up example, there's really a .1mm difference. And maybe to someone that's meaningful, so it's worth reporting. But it's not a big enough effect to say "When I see someone on the street I can look at their height and know which of these two cities they're from."
And my peeve is that people will see a statistically significant difference and think that that alone makes it an important finding, versus a very real finding that might not actually matter that much.
Or, related, it's bad to see something with a p-value of .15 and a large effect size and decide that since it's not significant at .05 it's not interesting. Since that combination typically means that you have a too-small sample size, the best interpretation is probably more like "This might be interesting. Looks like we don't have enough data to tell if there's a relationship here, but it probably deserves another look."
(Edited to add that last thought, and clarify the previous one)