3 ms·
>"Statistical significance has NOTHING to do with magnitude of difference!" Here is the equation for a t-statistic (used for the common t-test): d=mean(a)-mea
by nonbel 11y ago
>"Statistical significance has NOTHING to do with magnitude of difference!"
Here is the equation for a t-statistic (used for the common t-test):
d=mean(a)-mean(b)
s=sqrt(var(a) + var(b))/sqrt(n)
t=d/s
I see the magnitude of difference (d) right in the numerator. The t-value is then compared to the t-distribution to get the tail probability.
- xtacy 11y agoYou are also normalising the difference by the variance, so the t-statistic has no units.
- nonbel 11y agoThe claim the significance has nothing to do with the magnitude of a difference is just wrong though. It clearly does, this information is just merged with other information about the variance and sample size to get the p-value, which is compared to a threshold to get significance.
- yoplait_ 11y agoit's not "just wrong". On the contrary, it is asymptotically true. In a world of abundant data, that matters.
- nonbel 11y agoPlease expand on this.
- xtacy 11y agoI think what the article meant to say was that, for the same number of samples, the p-value when you have delta=50 (the difference between groups), stdev=10 is the same as the p-value of delta=0.5, stdev=0.1. Depending on the study, finding a delta whose p-value is significant does not necessarily mean that the size of the effect (i.e., delta) might be significant enough to be useful.