4 ms·
It doesn't make sense to say in a binary sense that an observation agrees or disagrees with the null hypothesis, there are only probabilities. The p-value is th
by evanpw 11y ago
It doesn't make sense to say in a binary sense that an observation agrees or disagrees with the null hypothesis, there are only probabilities. The p-value is the probability of seeing a set of values as extreme as your actual observations, assuming that the null hypothesis is true.
- jsprogrammer 11y ago>It doesn't make sense to say... That depends on your hypothesis. >The p-value is the probability of seeing a set of values That is a circular definition. What is, "probability"? P-values are not prescriptive statements about future observations, but are descriptions of actual observations.
- evanpw 11y agoAs TeMPOraL points out, you seem to be defining probability in general. The p-value is a very specific concept in statistics, not an abbreviation for "probability value" (https://en.wikipedia.org/wiki/P-value https://en.wikipedia.org/wiki/P-value).
- jsprogrammer 11y agoYour link defines p-value as a probability. >More specifically, the p-value is defined as the probability of... Edit: A good exercise would be to start with the axioms[0] of probability theory and then derive the p-value for a simple experiment, using only those axioms (and your measured values). [0] https://en.wikipedia.org/wiki/Probability_axioms https://en.wikipedia.org/wiki/Probability_axioms
- deleted 11y ago[deleted]
- evanpw 11y agoExample: The null hypothesis is that X has a standard normal distribution, and you have one observation, with value 0.7. Since you only have one observation, your definition of p-value can only give the values 0 or 1, depending on whether you consider 0.7 to agree or disagree with the null hypothesis. The actual answer is 0.2419637 (or double that, depending on whether you do a one-tailed or two-tailed test), the probability of drawing a value >= 0.7 from a standard normal distribution (you can get this value by computing an integral on the density function of the standard normal).
- jsprogrammer 11y agoYou are only taking a single observation in that example, making analysis through probability theory effectively useless. Note that, probabilities of 0 or 1 are completely allowed under the theory and the computed value in your example is still consistent with my statement and probability theory. An issue is that probability distributions (such as standard normal) don't exist in the axioms and so must be constructed from the axioms. So, a p-value test using a standard normal distribution is taking a shortcut that hides some of the calculations, making it appear that p-value is not actually counting a ratio, however, if you look at the full calculation from the axioms, p-value represents the ratio of agreeing observations to total observations. Basically, a single observation at the level of testing a standard normal distribution may actually represent multiple observations at the level of the math required to construct the full calculation (this entirely depends on the hypothesis being tested).
- evanpw 11y agoLet Ω = R (real line), F = the set of Lebesque-measurable subsets of R, and P(E) = the integral over E of the density function of the standard normal. This is a probability space in the Kolmogorov sense, and my previous example carries through perfectly well in this framework. You observe x = 0.7, and compute the probability of the event X >= 0.7 as an integral. There are no ratios, and you don't need to label the observation as agreeing or disagreeing with the null hypothesis.
- jsprogrammer 11y agoSee my response here: https://news.ycombinator.com/item?id=10630389 https://news.ycombinator.com/item?id=10630389 You are doing it wrong. A p-value based on a single observation is meaningless. P-values are much more useful when they are continuously computed on a running experiment with sequential observations. See the work at the LHC for an example.
- TeMPOraL 11y agoFWIW, 'jsprogrammer is smuggling here the basic definition of probability - that P(event) = (number of observations in which an event happens) / (total number of observations).
- germanier 11y agoThat only is the case if you work on a discrete uniform distribution, which you usually don't.
- jsprogrammer 11y agoThe axioms are only defined on discrete uniform distributions. All other distributions must be constructed from the discrete uniform.
- evanpw 11y agoThis is not true. If you have a discrete uniform distribution, you can only have finitely many elements in your probability space, so it wouldn't make sense to have an axiom which refers to countably-many disjoint sets. The axioms are most useful for continuous distributions, and are closely related to measure theory.
- jsprogrammer 11y agoThe discrete, uniform nature is obvious in the rule of computation: Σ P(E) Each "unit" of the computation is weighted equally (though, the value of the units may differ).
- evanpw 11y agoI'll give you the benefit of the doubt and assume that you aren't deliberately trolling me, but you do seem to be deeply confused. I'll just give you a link to some lecture notes that go all the way from the axioms of probability (lecture 1) to p-values (lecture 12) and leave it at that: http://www.win.tue.nl/~rmcastro/2DI90/index.php?page=lectures http://www.win.tue.nl/~rmcastro/2DI90/index.php?page=lecture....