7 ms·
Lest anyone skimming the headline misses it, the point here isn't that these scientists couldn't correctly explain a p-value. The scientists here all could. It'
by peatmoss 11y ago
Lest anyone skimming the headline misses it, the point here isn't that these scientists couldn't correctly explain a p-value. The scientists here all could. It's that they couldn't explain it in a way that made intuitive sense to anyone. So, rather than being a dig at scientists, I think the points here are:
1. p-values are intrinsically none-too-intuitive.
2. What p-values tell us isn't necessarily that interesting.
- Alex3917 11y ago> What p-values tell us isn't necessarily that interesting. The problem is that the concept sounds more important than it is if you don't understand it. This is a bad failure state. The issue is that the probability of some phenomena happening by chance is mostly orthogonal to the probability of that phenomena being true or whatever, but this seems outside the ability of most people to grasp.
- peatmoss 11y agoI can also say that, when I was starting stats, the term p-value seemed to hint tantalizingly at "probability." I cannot be the only person seduced into erroneously thinking that a p-value had something to do with the probability that something could be untrue.
- lmm 11y agoWell a p-value is a probability, so I don't think that's exactly misleading.
- jsprogrammer 11y ago> 1. p-values are intrinsically none-too-intuitive P-values are quite intuitive. They are the ratio of observations made which agree with the null hypothesis to the total number of observations made. Edit: If you are going to downmod, at least point out the error you perceive.
- nickpsecurity 11y agoThat was as intuitive as the seemless connection between the Magic Bullet Theory and Ballistics.
- jsprogrammer 11y agoIt comes directly from the calculation of p-values through probability theory.
- omginternets 11y agoI'm reminded of a joke about how Haskell empowers you to apply the infinite power of abstract mathematics using the intuitive simplicity of abstract mathematics. I think the resistance you're encountering stems from the fact that you're appealing to formality rather than intuition.
- senekerim 11y agoAlso from the fact that he doesn't know what he's talking about.
- jsprogrammer 11y agoYou talking about me?
- jsprogrammer 11y agoCounting positive observations vs. total observations is a formality rather than intuition? The formality is the axioms of probability. The intuition is that you make a theory and measure how many predictions it gets right.
- omginternets 11y agoIt's all relative, as they say.
- cousin_it 11y agoEspecially when publication bias comes into play: https://xkcd.com/882/ https://xkcd.com/882/
- KingMob 11y agoSadly true. On the one hand, the scientists ignored the multiple comparisons problem, and should have applied something like the Bonferroni or False Discovery Rate corrections. OTOH, they got publications and publicity for ignoring it.
- vlehto 11y ago>2. What p-values tell us isn't necessarily that interesting. High p-value is the first thing to look at a study. If it's high, then you can simply assume the findings are unreliable. But if it's low, it doesn't yet mean anything. The p-values allow you to discard worst mumbojumbo quickly. That might not be interesting, but it's valuable. The thing to remember is that "we don't know" =/ "untrue".
- blahblah3 11y agoI would argue that the p-value way of reasoning (probability of something happening given a hypothesis) corresponds to an intuitive way many people reason about probability. For example, suppose the police are investigating the robbery of a home. They find a stranger's (call him person S) fingerprints all over the house. If S claimed he was innocent, a questioner might ask: "If you're really innocent, then why were your fingerprints all over the house?" This is another way of stating that P(Data | innocent) is small (similar to the p-value concept). Since we often don't have a strong prior reason for believing a given individual is innocent or guilty, the p-value heuristic can be reasonable. The formal connection to P(innocent | Data) is given by Bayes' theorem
- btilly 11y agoThere is a third point. Which is that scientists consistently agree with clearly incorrect statements about p-values. That means that knowing the correct definition is no guarantee that you're actually using it correctly. (Odds are you aren't.)
- vectorjohn 11y agoI've come to think that if someone can't explain something well, they probably don't understand it as well as they think. I've heard people that fancy themselves experts on this or that software concept, and when trying to explain it use all the technical jargon they possibly can. It comes off as just regurgitating things they've heard before. In other words, I suspect the scientists that couldn't explain p-values in an intuitive way, didn't themselves fully grasp the subtleties of p-values.