3 ms·

P-values are much more problematic at the alpha=.05 threshold than the alpha=.005 or alpha=.01 thresholds. In my work as a data scientist, 99% of the time this

P-values are much more problematic at the alpha=.05 threshold than the alpha=.005 or alpha=.01 thresholds. In my work as a data scientist, 99% of the time this is not even relevant. We have huge sample sizes and most of the p-values we see are <.0001, meaning there is SO much data that even a very very small effect size can be found "significant." The question then: does a mean of 20.02 being significantly different from a mean of 20.03 over 10 million patients, is that actually important to us?

mswen 7y ago
Yes, thank you. Practical significance. You cannot get away with just running statistical test or deep learning models. You often need to ask whether a change in product, service or process to take advantage of some trend or difference between customers segment will actually be worth doing. That involves getting your hands dirty in the business, trying to identify all the costs and side-effects of a change, and then how much of a size effect you will actually capture once the changes are made. My views in more detail [1] [1] http://computationalimagination.com/article_practical_significance.php http://computationalimagination.com/article_practical_signif...