We're interested in the probability of a coin flip yielding heads before we flip any coins. Uniform is just a really common prior to choose in this situation fo

We're interested in the probability of a coin flip yielding heads before we flip any coins. Uniform is just a really common prior to choose in this situation for a few reasons: -It's a special case of the beta distribution, which is the conjugate prior for binomial problems. This means that the distribution of the probability of getting heads given the coin flips is in the same family as the prior itself (ie: beta priors with binomial likelihoods yield beta posteriors). -The uniform (for this problem at least) is an "objective prior", which expresses that we don't have much information about whether the flip is biased. The example you give (modeling p^2 instead of p) is a great example of when the uniform would be a bad choice. The reason the uniform doesn't work in this case is because for binomial data (coin flips), a uniform prior is not invariant to reparametrization. If choosing priors was so simple as always going with the uniform, there'd be little reason to go with Bayes! The choice of prior sometimes makes a radical difference in the posterior (especially with small samples), and there's many things to consider when you choose priors (computational convenience, uninformative versus informative priors, hierarchical modeling, etc). http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Conjugate_prior http://en.wikipedia.org/wiki/Conjugate_prior