4 ms·

Unfortunately you're confused about Bayesian statistics. A classic Bayesian analysis of coin flips would look something like: x_i ~ Bernoulli(p) p ~ Beta(1,1)

Unfortunately you're confused about Bayesian statistics. A classic Bayesian analysis of coin flips would look something like: x_i ~ Bernoulli(p) p ~ Beta(1,1) For which the posterior IIRC: p | x ~ Beta(n_tails+1, n_heads + 1) Which, if you look up the variance of the Beta distribution, you will see a credible interval p shrink around the truth of p=0.5 if the coin is indeed fair. This encapsulates the fact that if we have 5 heads and 5 tails we think the coin is fair. But if we have 500 heads and 500 tails we still think the coin is fair, but we're much more confident that it is so.

dwohnitmok 8y ago
I think we're talking past each other. I'm saying this: A probability can only be assigned to an event. A distribution can only be assigned to the collection of all events. A probability cannot be an event except as a parameter into another part of a model. Hence statements of the form "This is a picture of a cat with probability X with this credible interval" are category errors. Notice how what you're talking about in this example is the parameter p not the probability that the coin turns up heads. Indeed to find the latter you must explicitly integrate over your p. We agree on the analysis of confidence (although I'm not sure I'd say "I think the coin is fair with low confidence" after e.g. a single heads and a single tails; I'd probably say "I don't know, but that's the mode of my distribution"). My hand-wavy statement about updates alludes to the general case, where you can't necessarily describe your model easily with parameters, but instead have an arbitrary joint distribution. With parameters to your model you can talk about distributions and summary statistics about your parameters (e.g. p for Bernoulli), which is a special case of a general update function, but even if you don't have parameters for which to talk about, your update function still has all the information about "confidence" you need.
- conjectures 8y ago
  - We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). - We can define a=icdf_p(0.05), b=icdf_p(0.95) under the posterior. - (a,b) is an event in the sigma algebra (of the posterior probability space) for p whose mass is 0.9. - (a,b) is also a central credible interval. I don't think you'll be able to point to one of those statements above and show that it is wrong. I think the stumbling point here is the notion of random measures. Random measures have sample spaces, sigma algebras of events, and their own distributions. Events have probabilities, as you say. This is the sense in which we can speak of probabilities (i.e. probability distributions) having probabilities (i.e. probability distributions). Because Binomials are one dimensional it's particularly easy to think about intervals.
  - dwohnitmok 8y ago
    The second step p = Probability(X_i = heads) is technically wrong (although a common abuse of notation). It's strictly speaking p = Probability(X_i = heads | p). So while it is meaningful to say: "p has this probability distribution." It is not meaningful to say: "The probability the coin comes up heads has this probability distribution." This is because p and the probability the coin comes up heads are not the same thing (this is even more apparent if you have a non-Bernoulli scenario) when p is no longer fixed. It's not really just a case of marginal vs conditional probabilities either. Even in the case of a conditional probability, it's only defined for a fixed condition. Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability.
    - conjectures 8y ago
      > Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability. But that's exactly what it does mean. For the Dirichlet Process, the probability of hitting a set in the partition of a sample space is random. A DP is a couple of generalisations up from the beta distribution on p under discussion [1]. I.e. this specializes to the current situation as: * The sample space theta is the set {H, T}. * The only interesting partition to talk about is {A_1, A_2}={{H}, {T}}. * The Dirichlet distribution on (G(A_1), G(A_2)) is therefore 2 dimensional, which is a Beta. * G(A_1) is the probability measure of hitting heads. No specific parametric model involving univariate p needs to be involved. [1] c.f. eqn 2 of http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf

dwohnitmok 8y ago

I think we're talking past each other. I'm saying this: A probability can only be assigned to an event. A distribution can only be assigned to the collection of all events. A probability cannot be an event except as a parameter into another part of a model. Hence statements of the form "This is a picture of a cat with probability X with this credible interval" are category errors. Notice how what you're talking about in this example is the parameter p not the probability that the coin turns up heads. Indeed to find the latter you must explicitly integrate over your p. We agree on the analysis of confidence (although I'm not sure I'd say "I think the coin is fair with low confidence" after e.g. a single heads and a single tails; I'd probably say "I don't know, but that's the mode of my distribution"). My hand-wavy statement about updates alludes to the general case, where you can't necessarily describe your model easily with parameters, but instead have an arbitrary joint distribution. With parameters to your model you can talk about distributions and summary statistics about your parameters (e.g. p for Bernoulli), which is a special case of a general update function, but even if you don't have parameters for which to talk about, your update function still has all the information about "confidence" you need.

conjectures 8y ago
- We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). - We can define a=icdf_p(0.05), b=icdf_p(0.95) under the posterior. - (a,b) is an event in the sigma algebra (of the posterior probability space) for p whose mass is 0.9. - (a,b) is also a central credible interval. I don't think you'll be able to point to one of those statements above and show that it is wrong. I think the stumbling point here is the notion of random measures. Random measures have sample spaces, sigma algebras of events, and their own distributions. Events have probabilities, as you say. This is the sense in which we can speak of probabilities (i.e. probability distributions) having probabilities (i.e. probability distributions). Because Binomials are one dimensional it's particularly easy to think about intervals.
- dwohnitmok 8y ago
  The second step p = Probability(X_i = heads) is technically wrong (although a common abuse of notation). It's strictly speaking p = Probability(X_i = heads | p). So while it is meaningful to say: "p has this probability distribution." It is not meaningful to say: "The probability the coin comes up heads has this probability distribution." This is because p and the probability the coin comes up heads are not the same thing (this is even more apparent if you have a non-Bernoulli scenario) when p is no longer fixed. It's not really just a case of marginal vs conditional probabilities either. Even in the case of a conditional probability, it's only defined for a fixed condition. Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability.
  - conjectures 8y ago
    > Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability. But that's exactly what it does mean. For the Dirichlet Process, the probability of hitting a set in the partition of a sample space is random. A DP is a couple of generalisations up from the beta distribution on p under discussion [1]. I.e. this specializes to the current situation as: * The sample space theta is the set {H, T}. * The only interesting partition to talk about is {A_1, A_2}={{H}, {T}}. * The Dirichlet distribution on (G(A_1), G(A_2)) is therefore 2 dimensional, which is a Beta. * G(A_1) is the probability measure of hitting heads. No specific parametric model involving univariate p needs to be involved. [1] c.f. eqn 2 of http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf