3 ms·
This is extremely important and I'm not surprised the media is either ignorant, doesn't understand, or 'stretches the truth' to adhere to their worldview. Stati
by eanzenberg 11y ago
This is extremely important and I'm not surprised the media is either ignorant, doesn't understand, or 'stretches the truth' to adhere to their worldview. Statistical significance has NOTHING to do with magnitude of difference!
- nonbel 11y ago>"Statistical significance has NOTHING to do with magnitude of difference!" Here is the equation for a t-statistic (used for the common t-test): d=mean(a)-mean(b) s=sqrt(var(a) + var(b))/sqrt(n) t=d/s I see the magnitude of difference (d) right in the numerator. The t-value is then compared to the t-distribution to get the tail probability.
- xtacy 11y agoYou are also normalising the difference by the variance, so the t-statistic has no units.
- nonbel 11y agoThe claim the significance has nothing to do with the magnitude of a difference is just wrong though. It clearly does, this information is just merged with other information about the variance and sample size to get the p-value, which is compared to a threshold to get significance.
- yoplait_ 11y agoit's not "just wrong". On the contrary, it is asymptotically true. In a world of abundant data, that matters.
- nonbel 11y agoPlease expand on this.
- xtacy 11y agoI think what the article meant to say was that, for the same number of samples, the p-value when you have delta=50 (the difference between groups), stdev=10 is the same as the p-value of delta=0.5, stdev=0.1. Depending on the study, finding a delta whose p-value is significant does not necessarily mean that the size of the effect (i.e., delta) might be significant enough to be useful.