4 ms·

Your link defines p-value as a probability. >More specifically, the p-value is defined as the probability of... Edit: A good exercise would be to start with t

Your link defines p-value as a probability. >More specifically, the p-value is defined as the probability of... Edit: A good exercise would be to start with the axioms[0] of probability theory and then derive the p-value for a simple experiment, using only those axioms (and your measured values). [0] https://en.wikipedia.org/wiki/Probability_axioms https://en.wikipedia.org/wiki/Probability_axioms

deleted 11y ago
[deleted]
evanpw 11y ago
Example: The null hypothesis is that X has a standard normal distribution, and you have one observation, with value 0.7. Since you only have one observation, your definition of p-value can only give the values 0 or 1, depending on whether you consider 0.7 to agree or disagree with the null hypothesis. The actual answer is 0.2419637 (or double that, depending on whether you do a one-tailed or two-tailed test), the probability of drawing a value >= 0.7 from a standard normal distribution (you can get this value by computing an integral on the density function of the standard normal).
- jsprogrammer 11y ago
  You are only taking a single observation in that example, making analysis through probability theory effectively useless. Note that, probabilities of 0 or 1 are completely allowed under the theory and the computed value in your example is still consistent with my statement and probability theory. An issue is that probability distributions (such as standard normal) don't exist in the axioms and so must be constructed from the axioms. So, a p-value test using a standard normal distribution is taking a shortcut that hides some of the calculations, making it appear that p-value is not actually counting a ratio, however, if you look at the full calculation from the axioms, p-value represents the ratio of agreeing observations to total observations. Basically, a single observation at the level of testing a standard normal distribution may actually represent multiple observations at the level of the math required to construct the full calculation (this entirely depends on the hypothesis being tested).
  - evanpw 11y ago
    Let Ω = R (real line), F = the set of Lebesque-measurable subsets of R, and P(E) = the integral over E of the density function of the standard normal. This is a probability space in the Kolmogorov sense, and my previous example carries through perfectly well in this framework. You observe x = 0.7, and compute the probability of the event X >= 0.7 as an integral. There are no ratios, and you don't need to label the observation as agreeing or disagreeing with the null hypothesis.
    - jsprogrammer 11y ago
      See my response here: https://news.ycombinator.com/item?id=10630389 https://news.ycombinator.com/item?id=10630389 You are doing it wrong. A p-value based on a single observation is meaningless. P-values are much more useful when they are continuously computed on a running experiment with sequential observations. See the work at the LHC for an example.

deleted 11y ago

[deleted]

evanpw 11y ago

Example: The null hypothesis is that X has a standard normal distribution, and you have one observation, with value 0.7. Since you only have one observation, your definition of p-value can only give the values 0 or 1, depending on whether you consider 0.7 to agree or disagree with the null hypothesis. The actual answer is 0.2419637 (or double that, depending on whether you do a one-tailed or two-tailed test), the probability of drawing a value >= 0.7 from a standard normal distribution (you can get this value by computing an integral on the density function of the standard normal).

jsprogrammer 11y ago
You are only taking a single observation in that example, making analysis through probability theory effectively useless. Note that, probabilities of 0 or 1 are completely allowed under the theory and the computed value in your example is still consistent with my statement and probability theory. An issue is that probability distributions (such as standard normal) don't exist in the axioms and so must be constructed from the axioms. So, a p-value test using a standard normal distribution is taking a shortcut that hides some of the calculations, making it appear that p-value is not actually counting a ratio, however, if you look at the full calculation from the axioms, p-value represents the ratio of agreeing observations to total observations. Basically, a single observation at the level of testing a standard normal distribution may actually represent multiple observations at the level of the math required to construct the full calculation (this entirely depends on the hypothesis being tested).
- evanpw 11y ago
  Let Ω = R (real line), F = the set of Lebesque-measurable subsets of R, and P(E) = the integral over E of the density function of the standard normal. This is a probability space in the Kolmogorov sense, and my previous example carries through perfectly well in this framework. You observe x = 0.7, and compute the probability of the event X >= 0.7 as an integral. There are no ratios, and you don't need to label the observation as agreeing or disagreeing with the null hypothesis.
  - jsprogrammer 11y ago
    See my response here: https://news.ycombinator.com/item?id=10630389 https://news.ycombinator.com/item?id=10630389 You are doing it wrong. A p-value based on a single observation is meaningless. P-values are much more useful when they are continuously computed on a running experiment with sequential observations. See the work at the LHC for an example.