3 ms·
It's true that effect sizes are often more important, but it's also true that they're also often incorrect. See e.g. Ioannidis, J. P. A. (2008). Why Most Disco
by capnrefsmmat 13y ago
It's true that effect sizes are often more important, but it's also true that they're also often incorrect. See e.g.
Ioannidis, J. P. A. (2008). Why Most Discovered True Associations Are Inflated. Epidemiology, 19(5), 640–648. doi:10.1097/EDE.0b013e31818131e7
Most studies are underpowered and are incapable of detecting the true effect. Only if they get lucky and observe an abnormally large effect will they obtain a statistically significant result, so the published results tend to be significant overestiates.
For another good example, see
Gelman, A., & Weakliem, D. (2009). Of beauty, sex, and power: statistical challenges in estimating small effects. American Scientist, 97, 310–316.
http://www.stat.columbia.edu/~gelman/research/unpublished/power4r.pdf http://www.stat.columbia.edu/~gelman/research/unpublished/po...
- Fomite 13y agoI think part of the point there is not to pass effect estimates through a significance test filter first. Most studies are underpowered to detect a true effect at alpha = 0.05. That doesn't actually suggest that most studies are wrong as much as if a study is underpowered and doesn't find a significant finding, we assert its dull and uninteresting. Ironically, the Ioannidis paper is in Epidemiology, which is a journal that is fairly anti-significance testing, but where I still get reviewers suggesting that an effect measure with a confidence interval that brushes against the null must mean nothing at all.