3 ms·
Not a huge fan of this idea. For example, people who analyze twitter data can get very small p-values because they analyze millions of tweets even though the ef
by jimmar 9y ago
Not a huge fan of this idea. For example, people who analyze twitter data can get very small p-values because they analyze millions of tweets even though the effects they find are very small. See https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1336700 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1336700
- amelius 9y agoI don't see the problem as long as you clearly separate significance and effect-size.
- seanwilson 9y agoThis bothers me a lot of media reporting. The headline will be something like "X is good/bad for you" with a tiny effect size but the way it's reported makes you assume it's a large effect size. Usually the effect size won't be discussed in any depth or at all, they just want to sum it up in black and white. If the effect size is tiny it's probably just noise that would disappear in a higher quality future study.
- moultano 9y agoI'd rather hear about small things that are true than large things that are false.
- cropsieboss 9y agoThe things is that these small things might just be noise from some confounding factors. http://jaoa.org/article.aspx?articleid=2517494 http://jaoa.org/article.aspx?articleid=2517494 For example, here sample size is huge, USA population gets significant increased risk, while EU population does not. Mixing the two together would result in a smaller but still significant increased risk. Given the size, it's quite clear that USA population has many other confounding factors that cannot be eliminated by mathematics alone (there is no control).
- nonbel 9y agoStatistical significance doesn't mean an effect is "true", or "real". So getting rid if it shouldn't cause you any issues.
- stewbrew 9y agoThe problem here is that you most likely hear about false positives.
- mhermher 9y agoThis is why I think better multiple-test-correction methods are more important and would lead better to a desirable outcome than just lowering alpha.