4 ms·

No. Probabilities can be random variables themselves. Say p is the probability an image is of a cat. p could be low, e.g. 0.05. However, perhaps p has the 0.

No. Probabilities can be random variables themselves. Say p is the probability an image is of a cat. p could be low, e.g. 0.05. However, perhaps p has the 0.9 credible interval (0.01, 0.1). The width of that interval expresses how confident you are of the point estimate. The correct translation of "I don't know" is a wide credible interval.

dwohnitmok 8y ago
I disagree with this characterization, especially in a Bayesian context (which credible interval comes from). Parameters into a model can be modeled as random variables. Probabilities themselves cannot be, unless they are themselves parameters (or happen to coincide with a parameter) into another part of a statistical model. A statistical estimation of an event (in the sense of probability theory) must bottom out at a single number. Let's take a coin toss as an example. It does not make sense to say I think that the coin has a 0.05 chance of flipping heads with a 0.9 credible interval of (0.01, 0.1). It does not make sense even to have a probability distribution on coming up heads (strictly speaking). Coming up heads is an event and events do not distributions associated with them, but rather single probabilities. On the other hand, we can model the outcomes of a coin with a Bernoulli distribution with parameter p where the probability of heads is p and that of tails is 1 - p. Notice here that p is a parameter that just happens to coincide with our probability. It then makes sense to talk of p as a random variable and having a distribution, credible interval, etc. Note, however, that our estimation of the event of the coin coming up heads is still a single number at the end of the day. Our intermediate calculations might just involve integrating over the space of intermediate values of p, which means that p no longer coincides with the probability of heads (indeed it doesn't really make sense because p is no longer a single number). But how do we express the intuition of "confidence in my probability estimate?" After all it seems like there's a difference between me saying this coin has a 50% chance of heads when I know it's fair and me saying a 50% chance of heads when you have me an unfamiliar coin I've never inspected (even though reasonable statistical models for either case might both say it's 50%). In Bayesian statistics this is encoded entirely by the function that updates your posterior probability. If you are very "confident" your posterior probability will remain similar to your prior probability no matter what evidence you see (I know for sure this coin is fair so even if it comes up tails 1000 times I still think the next time will be 50% chance of heads). If you are not very confident your posterior probability might change more dramatically (hmm... I saw a string of tails, I'm pretty sure the next coin flip probably won't be heads again and I'll modify my probability accordingly).
- conjectures 8y ago
  Unfortunately you're confused about Bayesian statistics. A classic Bayesian analysis of coin flips would look something like: x_i ~ Bernoulli(p) p ~ Beta(1,1) For which the posterior IIRC: p | x ~ Beta(n_tails+1, n_heads + 1) Which, if you look up the variance of the Beta distribution, you will see a credible interval p shrink around the truth of p=0.5 if the coin is indeed fair. This encapsulates the fact that if we have 5 heads and 5 tails we think the coin is fair. But if we have 500 heads and 500 tails we still think the coin is fair, but we're much more confident that it is so.
  - dwohnitmok 8y ago
    I think we're talking past each other. I'm saying this: A probability can only be assigned to an event. A distribution can only be assigned to the collection of all events. A probability cannot be an event except as a parameter into another part of a model. Hence statements of the form "This is a picture of a cat with probability X with this credible interval" are category errors. Notice how what you're talking about in this example is the parameter p not the probability that the coin turns up heads. Indeed to find the latter you must explicitly integrate over your p. We agree on the analysis of confidence (although I'm not sure I'd say "I think the coin is fair with low confidence" after e.g. a single heads and a single tails; I'd probably say "I don't know, but that's the mode of my distribution"). My hand-wavy statement about updates alludes to the general case, where you can't necessarily describe your model easily with parameters, but instead have an arbitrary joint distribution. With parameters to your model you can talk about distributions and summary statistics about your parameters (e.g. p for Bernoulli), which is a special case of a general update function, but even if you don't have parameters for which to talk about, your update function still has all the information about "confidence" you need.
    - conjectures 8y ago
      - We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). - We can define a=icdf_p(0.05), b=icdf_p(0.95) under the posterior. - (a,b) is an event in the sigma algebra (of the posterior probability space) for p whose mass is 0.9. - (a,b) is also a central credible interval. I don't think you'll be able to point to one of those statements above and show that it is wrong. I think the stumbling point here is the notion of random measures. Random measures have sample spaces, sigma algebras of events, and their own distributions. Events have probabilities, as you say. This is the sense in which we can speak of probabilities (i.e. probability distributions) having probabilities (i.e. probability distributions). Because Binomials are one dimensional it's particularly easy to think about intervals.

dwohnitmok 8y ago

I disagree with this characterization, especially in a Bayesian context (which credible interval comes from). Parameters into a model can be modeled as random variables. Probabilities themselves cannot be, unless they are themselves parameters (or happen to coincide with a parameter) into another part of a statistical model. A statistical estimation of an event (in the sense of probability theory) must bottom out at a single number. Let's take a coin toss as an example. It does not make sense to say I think that the coin has a 0.05 chance of flipping heads with a 0.9 credible interval of (0.01, 0.1). It does not make sense even to have a probability distribution on coming up heads (strictly speaking). Coming up heads is an event and events do not distributions associated with them, but rather single probabilities. On the other hand, we can model the outcomes of a coin with a Bernoulli distribution with parameter p where the probability of heads is p and that of tails is 1 - p. Notice here that p is a parameter that just happens to coincide with our probability. It then makes sense to talk of p as a random variable and having a distribution, credible interval, etc. Note, however, that our estimation of the event of the coin coming up heads is still a single number at the end of the day. Our intermediate calculations might just involve integrating over the space of intermediate values of p, which means that p no longer coincides with the probability of heads (indeed it doesn't really make sense because p is no longer a single number). But how do we express the intuition of "confidence in my probability estimate?" After all it seems like there's a difference between me saying this coin has a 50% chance of heads when I know it's fair and me saying a 50% chance of heads when you have me an unfamiliar coin I've never inspected (even though reasonable statistical models for either case might both say it's 50%). In Bayesian statistics this is encoded entirely by the function that updates your posterior probability. If you are very "confident" your posterior probability will remain similar to your prior probability no matter what evidence you see (I know for sure this coin is fair so even if it comes up tails 1000 times I still think the next time will be 50% chance of heads). If you are not very confident your posterior probability might change more dramatically (hmm... I saw a string of tails, I'm pretty sure the next coin flip probably won't be heads again and I'll modify my probability accordingly).

conjectures 8y ago
Unfortunately you're confused about Bayesian statistics. A classic Bayesian analysis of coin flips would look something like: x_i ~ Bernoulli(p) p ~ Beta(1,1) For which the posterior IIRC: p | x ~ Beta(n_tails+1, n_heads + 1) Which, if you look up the variance of the Beta distribution, you will see a credible interval p shrink around the truth of p=0.5 if the coin is indeed fair. This encapsulates the fact that if we have 5 heads and 5 tails we think the coin is fair. But if we have 500 heads and 500 tails we still think the coin is fair, but we're much more confident that it is so.
- dwohnitmok 8y ago
  I think we're talking past each other. I'm saying this: A probability can only be assigned to an event. A distribution can only be assigned to the collection of all events. A probability cannot be an event except as a parameter into another part of a model. Hence statements of the form "This is a picture of a cat with probability X with this credible interval" are category errors. Notice how what you're talking about in this example is the parameter p not the probability that the coin turns up heads. Indeed to find the latter you must explicitly integrate over your p. We agree on the analysis of confidence (although I'm not sure I'd say "I think the coin is fair with low confidence" after e.g. a single heads and a single tails; I'd probably say "I don't know, but that's the mode of my distribution"). My hand-wavy statement about updates alludes to the general case, where you can't necessarily describe your model easily with parameters, but instead have an arbitrary joint distribution. With parameters to your model you can talk about distributions and summary statistics about your parameters (e.g. p for Bernoulli), which is a special case of a general update function, but even if you don't have parameters for which to talk about, your update function still has all the information about "confidence" you need.
  - conjectures 8y ago
    - We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). - We can define a=icdf_p(0.05), b=icdf_p(0.95) under the posterior. - (a,b) is an event in the sigma algebra (of the posterior probability space) for p whose mass is 0.9. - (a,b) is also a central credible interval. I don't think you'll be able to point to one of those statements above and show that it is wrong. I think the stumbling point here is the notion of random measures. Random measures have sample spaces, sigma algebras of events, and their own distributions. Events have probabilities, as you say. This is the sense in which we can speak of probabilities (i.e. probability distributions) having probabilities (i.e. probability distributions). Because Binomials are one dimensional it's particularly easy to think about intervals.