4 ms·

A question for people who know more about this than I do: Why was the uniform distribution on [0, 1] chosen initially? Choosing a different distribution would

by spicyj 13y ago

A question for people who know more about this than I do: Why was the uniform distribution on [0, 1] chosen initially? Choosing a different distribution would give a different result. (And it doesn't make much sense to say, "Always choose the uniform distribution!" because the choice of variable affects the meaning of the distribution -- if instead we wonder about the value of p^2 and choose a uniform distribution for it on [0, 1], won't we get a completely different result?)

christopheraden 13y ago
We're interested in the probability of a coin flip yielding heads before we flip any coins. Uniform is just a really common prior to choose in this situation for a few reasons: -It's a special case of the beta distribution, which is the conjugate prior for binomial problems. This means that the distribution of the probability of getting heads given the coin flips is in the same family as the prior itself (ie: beta priors with binomial likelihoods yield beta posteriors). -The uniform (for this problem at least) is an "objective prior", which expresses that we don't have much information about whether the flip is biased. The example you give (modeling p^2 instead of p) is a great example of when the uniform would be a bad choice. The reason the uniform doesn't work in this case is because for binomial data (coin flips), a uniform prior is not invariant to reparametrization. If choosing priors was so simple as always going with the uniform, there'd be little reason to go with Bayes! The choice of prior sometimes makes a radical difference in the posterior (especially with small samples), and there's many things to consider when you choose priors (computational convenience, uninformative versus informative priors, hierarchical modeling, etc). http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Conjugate_prior http://en.wikipedia.org/wiki/Conjugate_prior
bjterry 13y ago
If you were holding a physical coin in your hand, you would have to be crazy to select a uniform distribution unless it was shaped like a sphere. This is sort of a really minor pet peeve of mine when people use coin-flipping as an example for these things. If it's even vaguely coin-like, even the most ridiculous distortion (maybe it's made of uranium on one side and aluminum on the other) probably couldn't bring the true probability past 70% or something.
- spicyj 13y ago
  So what would you use as a reasonable prior here?
  - dthunt 13y ago
    For argumentative purposes (if trying to reach a consensus with someone who claims to be 95% sure that the coin is 'fair' and accepts the '45-55% heads is fair' definition), you would concentrate most of your probability mass there, and find a mutually agreeable shape. The 'it's not shaped like a sphere' thing suggests that some people would take into account visual information about the object to inform their priors, and be prone to rejecting the fitness of the uniform. That may conflict with your sensibilities if you expect very strange coins, but if the priors aren't too crazily bad, if you have enough time for a few more coin flips, and the coin isn't particularly brittle, what's an order of magnitude between friends? Edit: And for formal mathy purposes, ask someone else :)
  - bermanoid 13y ago
    If it's a normal looking coin being flipped in the air and caught before presentation, the only reasonable prior is that almost all of the probability mass should be that it's completely fair. Maybe I'd leave 1% to spread amongst the rest just in case the adversary has some seriously devious tricks up their sleeve. Because physics - it's not possible to bias a rigid body so that it rotates with non-constant angular speed when flipped, as long as air resistance can be neglected, and that means that a fair flip gives 50/50 odds as to what side you catch it on. (Edit: clarity) If other stuff is going on, like you're letting it bounce or something, then it depends on the particulars. It's rather easy to load a die, for instance.
  - darkxanthos 13y ago
    One of the beautiful aspects of Bayesian methods is that they are subjective. Due to that I think you should really be asking "What sort of priors are reasonable here?" From that perspective the uniform prior I use is reasonable (though non-committal and fairly uninformative), a triangular distribution with its peak at 50% (http://en.wikipedia.org/wiki/Triangular_distribution http://en.wikipedia.org/wiki/Triangular_distribution), the prior I have after updating tails and then heads, a normal distribution, or using the beta distribution with an equal number of both parameters. Honestly you may decide there's a fair chance the coin is heads biased so you choose a prior that has most of the mass of the probability above 50%. As long as you have a reason, it's reasonable. You don't have to worry too much that you've chosen the optimal prior since more data makes equals of priors in the long run. The only terrible thing you can do, is pick a prior that absolutely excludes certain hypotheses with 0% likelihood. No amount of data can overcome that.
bluecalm 13y ago
We would! Choosing priors is difficult and different people with different experience and views will have different (but still reasonable) priors. You may take personal (just use all your experience) or objective (try to come up with some scheme for assigning priors to types of situations) view on this problem. Work was done in both directions. It's more of a philosophical problem than math problem, although in many cases you want something specific: "I have no idea what the outcome might be but I think it's either extremely on the right or left so give me priori which converges fast to one or the other). Or in a coin example: "it's almost surely more or less fair, so I don't want to change my views after several unlucky flips".
darkxanthos 13y ago
Great question! With just the few updates I've given you're probably right that it would affect things significantly. However, the more data you have the less the prior matters. This is known as swamping the priors. In this case a uniform prior isn't incorrect, but you could definitely say its suboptimal and that I could make my examples much more accurate by choosing a prior that most represents my initial beliefs (like the one head one tail histogram for example.

by spicyj 13y ago

christopheraden 13y ago

We're interested in the probability of a coin flip yielding heads before we flip any coins. Uniform is just a really common prior to choose in this situation for a few reasons: -It's a special case of the beta distribution, which is the conjugate prior for binomial problems. This means that the distribution of the probability of getting heads given the coin flips is in the same family as the prior itself (ie: beta priors with binomial likelihoods yield beta posteriors). -The uniform (for this problem at least) is an "objective prior", which expresses that we don't have much information about whether the flip is biased. The example you give (modeling p^2 instead of p) is a great example of when the uniform would be a bad choice. The reason the uniform doesn't work in this case is because for binomial data (coin flips), a uniform prior is not invariant to reparametrization. If choosing priors was so simple as always going with the uniform, there'd be little reason to go with Bayes! The choice of prior sometimes makes a radical difference in the posterior (especially with small samples), and there's many things to consider when you choose priors (computational convenience, uninformative versus informative priors, hierarchical modeling, etc). http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Jeffreys_prior http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Beta_distribution http://en.wikipedia.org/wiki/Conjugate_prior http://en.wikipedia.org/wiki/Conjugate_prior

bjterry 13y ago

If you were holding a physical coin in your hand, you would have to be crazy to select a uniform distribution unless it was shaped like a sphere. This is sort of a really minor pet peeve of mine when people use coin-flipping as an example for these things. If it's even vaguely coin-like, even the most ridiculous distortion (maybe it's made of uranium on one side and aluminum on the other) probably couldn't bring the true probability past 70% or something.

spicyj 13y ago
So what would you use as a reasonable prior here?
- dthunt 13y ago
  For argumentative purposes (if trying to reach a consensus with someone who claims to be 95% sure that the coin is 'fair' and accepts the '45-55% heads is fair' definition), you would concentrate most of your probability mass there, and find a mutually agreeable shape. The 'it's not shaped like a sphere' thing suggests that some people would take into account visual information about the object to inform their priors, and be prone to rejecting the fitness of the uniform. That may conflict with your sensibilities if you expect very strange coins, but if the priors aren't too crazily bad, if you have enough time for a few more coin flips, and the coin isn't particularly brittle, what's an order of magnitude between friends? Edit: And for formal mathy purposes, ask someone else :)
- bermanoid 13y ago
  If it's a normal looking coin being flipped in the air and caught before presentation, the only reasonable prior is that almost all of the probability mass should be that it's completely fair. Maybe I'd leave 1% to spread amongst the rest just in case the adversary has some seriously devious tricks up their sleeve. Because physics - it's not possible to bias a rigid body so that it rotates with non-constant angular speed when flipped, as long as air resistance can be neglected, and that means that a fair flip gives 50/50 odds as to what side you catch it on. (Edit: clarity) If other stuff is going on, like you're letting it bounce or something, then it depends on the particulars. It's rather easy to load a die, for instance.
- darkxanthos 13y ago
  One of the beautiful aspects of Bayesian methods is that they are subjective. Due to that I think you should really be asking "What sort of priors are reasonable here?" From that perspective the uniform prior I use is reasonable (though non-committal and fairly uninformative), a triangular distribution with its peak at 50% (http://en.wikipedia.org/wiki/Triangular_distribution http://en.wikipedia.org/wiki/Triangular_distribution), the prior I have after updating tails and then heads, a normal distribution, or using the beta distribution with an equal number of both parameters. Honestly you may decide there's a fair chance the coin is heads biased so you choose a prior that has most of the mass of the probability above 50%. As long as you have a reason, it's reasonable. You don't have to worry too much that you've chosen the optimal prior since more data makes equals of priors in the long run. The only terrible thing you can do, is pick a prior that absolutely excludes certain hypotheses with 0% likelihood. No amount of data can overcome that.

bluecalm 13y ago

We would! Choosing priors is difficult and different people with different experience and views will have different (but still reasonable) priors. You may take personal (just use all your experience) or objective (try to come up with some scheme for assigning priors to types of situations) view on this problem. Work was done in both directions. It's more of a philosophical problem than math problem, although in many cases you want something specific: "I have no idea what the outcome might be but I think it's either extremely on the right or left so give me priori which converges fast to one or the other). Or in a coin example: "it's almost surely more or less fair, so I don't want to change my views after several unlucky flips".

darkxanthos 13y ago

Great question! With just the few updates I've given you're probably right that it would affect things significantly. However, the more data you have the less the prior matters. This is known as swamping the priors. In this case a uniform prior isn't incorrect, but you could definitely say its suboptimal and that I could make my examples much more accurate by choosing a prior that most represents my initial beliefs (like the one head one tail histogram for example.