4 ms·
It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That's why we tell students to consider both
by limbicsystem 7y ago
It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset) and variance explained. Loftus and Loftus make the point nicely: p tells you if you have enough samples and any effect to consider, variance explained tells you if it's worth pursuing. Both are useful guides to a thoughtful analysis. In addition, I'd make a case for thinking about the scientific significance and importance of the hypothesis and the Bayesian prior. And to put a positive spin on this, given how easy it is to get small p values, big ones are pretty much a red flag to stop the analysis and go and do something more productive instead.
- nonbel 7y ago> "It is true that, as Fisher points out, with enough samples you are almost guaranteed to reject the null hypothesis. " Where does Fisher point this out? > "That's why we tell students to consider both p values (which you could think of as a form of quality control on the dataset)" How is this "quality control"? It just tells you whether your sample size was large enough to pass an arbitrary threshold...