3 ms·
> If the p-Value is less than significance level (ideally 0.05), Erm, no. P=0.05 is borderline meaningless, there could as much as 30% chance you are wrong ab
by ekianjo 11y ago
> If the p-Value is less than significance level (ideally 0.05),
Erm, no. P=0.05 is borderline meaningless, there could as much as 30% chance you are wrong about the actual difference being there depending on the true probability of the initial hypothesis.
P-values should be used with strong caution.
- snydly 11y ago> P-values should be used with strong caution. FiveThirtyEight (and Scientific American, and others) did some pretty interesting articles about this recently if you haven't seen it: http://fivethirtyeight.com/features/science-isnt-broken/ http://fivethirtyeight.com/features/science-isnt-broken/ Just from personal experience, the use of p-values is really broken in biology/chemistry. The things I've heard principal investigators say...
- GFK_of_xmaspast 11y ago> could as much as 30% chance you are wrong about the actual difference being there depending on the true probability of the initial hypothesis. I'm having trouble parsing this, are you talking about the power of the test?
- capnrefsmmat 11y agoNo, the base rate. If you're doing a significance test to detect a rare disease that only occurs in a tiny fraction of people, for example, the majority of statistically significant results you obtain will be false, despite having p < 0.05. This is a common problem in many fields, and you can use false discovery rate control methods to account for it: http://www.statisticsdonewrong.com/p-value.html http://www.statisticsdonewrong.com/p-value.html
- cetacea 11y agoEven better, p-values should not be used at all. If I have data in hand, I want to use it to find out the probability that my hypothesis is true. But p-value analysis requires me to instead ask a different question that I don't really care about, involving whether my data are consistent with the null hypothesis. Everything is just so much more sensible if you allow yourself to assign probabilities to hypotheses, rather than assuming a hypothesis from the outset and computing opaque statistics relating to your data.
- healer 11y agoThere is in fact a probability attached to p-values. A p-value of 0.05 for instance means your conclusions will be wrong 5 out of 100 times. You can reduce the p-value to e.g. 0.001 or any other value you want.
- cwyers 11y agoNo, it means that the probability of seeing an effect of that magnitude on a dataset of that size when the null hypothesis is true will happen due to random chance 5 out of 100 times. It says NOTHING about your hypothesis, it is entirely a statement about the null hypothesis.