4 ms·

- We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). -

- We flip some coins X_i~Bernoulli(p) - We can say that p = Probability(X_i = heads) - We have a posterior distribution for p (e.g. under Beta(1,1) prior). - We can define a=icdf_p(0.05), b=icdf_p(0.95) under the posterior. - (a,b) is an event in the sigma algebra (of the posterior probability space) for p whose mass is 0.9. - (a,b) is also a central credible interval. I don't think you'll be able to point to one of those statements above and show that it is wrong. I think the stumbling point here is the notion of random measures. Random measures have sample spaces, sigma algebras of events, and their own distributions. Events have probabilities, as you say. This is the sense in which we can speak of probabilities (i.e. probability distributions) having probabilities (i.e. probability distributions). Because Binomials are one dimensional it's particularly easy to think about intervals.

dwohnitmok 8y ago
The second step p = Probability(X_i = heads) is technically wrong (although a common abuse of notation). It's strictly speaking p = Probability(X_i = heads | p). So while it is meaningful to say: "p has this probability distribution." It is not meaningful to say: "The probability the coin comes up heads has this probability distribution." This is because p and the probability the coin comes up heads are not the same thing (this is even more apparent if you have a non-Bernoulli scenario) when p is no longer fixed. It's not really just a case of marginal vs conditional probabilities either. Even in the case of a conditional probability, it's only defined for a fixed condition. Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability.
- conjectures 8y ago
  > Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability. But that's exactly what it does mean. For the Dirichlet Process, the probability of hitting a set in the partition of a sample space is random. A DP is a couple of generalisations up from the beta distribution on p under discussion [1]. I.e. this specializes to the current situation as: * The sample space theta is the set {H, T}. * The only interesting partition to talk about is {A_1, A_2}={{H}, {T}}. * The Dirichlet distribution on (G(A_1), G(A_2)) is therefore 2 dimensional, which is a Beta. * G(A_1) is the probability measure of hitting heads. No specific parametric model involving univariate p needs to be involved. [1] c.f. eqn 2 of http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf

dwohnitmok 8y ago

The second step p = Probability(X_i = heads) is technically wrong (although a common abuse of notation). It's strictly speaking p = Probability(X_i = heads | p). So while it is meaningful to say: "p has this probability distribution." It is not meaningful to say: "The probability the coin comes up heads has this probability distribution." This is because p and the probability the coin comes up heads are not the same thing (this is even more apparent if you have a non-Bernoulli scenario) when p is no longer fixed. It's not really just a case of marginal vs conditional probabilities either. Even in the case of a conditional probability, it's only defined for a fixed condition. Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability.

conjectures 8y ago
> Even in the case of random measures, while you can reasonably talk about the probability of a probability distribution, it still doesn't really make sense to talk about the probability distribution of a probability. But that's exactly what it does mean. For the Dirichlet Process, the probability of hitting a set in the partition of a sample space is random. A DP is a couple of generalisations up from the beta distribution on p under discussion [1]. I.e. this specializes to the current situation as: * The sample space theta is the set {H, T}. * The only interesting partition to talk about is {A_1, A_2}={{H}, {T}}. * The Dirichlet distribution on (G(A_1), G(A_2)) is therefore 2 dimensional, which is a Beta. * G(A_1) is the probability measure of hitting heads. No specific parametric model involving univariate p needs to be involved. [1] c.f. eqn 2 of http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf http://www.gatsby.ucl.ac.uk/~ywteh/research/npbayes/dp.pdf