3 ms·

more like a verbose explanation of Bayes' Theorem... I'll give my succinct spin: Throw a coin in the air 10 times. Say 6 times the coin lands heads up, and 4 t

more like a verbose explanation of Bayes' Theorem... I'll give my succinct spin: Throw a coin in the air 10 times. Say 6 times the coin lands heads up, and 4 times tails up. A Bayesian would say: the most likely explanation is that the coin is biased in favor of heads with Prob(heads) = .6 and Prob(tails) = .4. Because given such a model, the chance of this outcome is 25% (10 choose 6 * .6^6 * .4^4). Less likely is the explanation that the coin is fair. A Frequentist would begin by assuming a model, likely that the coin is fair. She would then argue that the probability of getting 6 heads in 10 tosses is about 21% (10 choose 6 * 1/2^10), and she would have tools (number of STDs from the mean, etc) to test how much of an outlier this observation was, in order to validate her model. In summary, the Bayesian considers many models at once, and asks how likely are they given the observations. A Frequentist starts with one model, and asks how likely is the observation. The Frequentist uses the scientific method: develop a hypothesis, then test it. The Bayesian does something else, which I think is better, but ill-defined. She somehow tests all hypotheses at once, and puts different confidences in each.

Strilanc 14y ago
The example is not quite right. The Bayesian starts with a prior, some weighting of likelihoods. In the case of a coin a reasonable prior would place a lot of initial probability on the coin being fair or almost fair. You'd need to flip a biased coin more times in order to overcome the prior and make 'biased' the dominant hypothesis.
- dmvaldman 14y ago
  That is correct. I tried to allude to that in the end when I said how a Bayesian mysteriously gives a certain confidence to the space of models. In the case of a uniform prior I believe my example is correct. Usually when Bayesian probability is explained, it often comes across as "duh, that's what anyone would do" except not rigorously with equations. I tried to give an example contrasting a Frequentist and Bayesian approach.
jules 14y ago
This is not exactly correct. Both methods you mention here are frequentist. The first is called maximum likelihood and the second is hypothesis testing. What a Bayesian does is not ill-defined. On the contrary, what a Bayesian does is two steps: (1) determine a prior (2) follow the math mechanically. What frequentists do when they do inference is actually much more ill-defined, as it required hidden assumptions intermixed with a mathematical derivation instead of cleanly separating assumptions from maths. In principle the bayesian way is: given assumptions, simply follow the axioms of probability to their logical conclusions. Of course that may not always be computationally feasible, so approximations have to be made sometimes. Frequentists require divine inspiration to develop a new method for each kind of problem. The bayesian way to solve the problem would be to take a prior probability distribution on the probability of heads, and then condition that on the data that the coin ended up on heads 6 times and on tails 4 times. For example with a uniform prior, we get this posterior: http://www.wolframalpha.com/input/?i=1*p%5E6%281-p%29%5E4%2Fintegral%281*q%5E6%281-q%29%5E4%2Cq%3D0..1%29 http://www.wolframalpha.com/input/?i=1*p%5E6%281-p%29%5E4%2F... As your can see the peak is indeed at p=0.6, but there is probability mass around it as well. Replace the factors of 1 with your favorite prior to see the effect (I couldn't convince wolfram alpha to compute a beta distribution in place of the 1, so you'll have to try that yourself).

Strilanc 14y ago

The example is not quite right. The Bayesian starts with a prior, some weighting of likelihoods. In the case of a coin a reasonable prior would place a lot of initial probability on the coin being fair or almost fair. You'd need to flip a biased coin more times in order to overcome the prior and make 'biased' the dominant hypothesis.

dmvaldman 14y ago
That is correct. I tried to allude to that in the end when I said how a Bayesian mysteriously gives a certain confidence to the space of models. In the case of a uniform prior I believe my example is correct. Usually when Bayesian probability is explained, it often comes across as "duh, that's what anyone would do" except not rigorously with equations. I tried to give an example contrasting a Frequentist and Bayesian approach.

jules 14y ago

This is not exactly correct. Both methods you mention here are frequentist. The first is called maximum likelihood and the second is hypothesis testing. What a Bayesian does is not ill-defined. On the contrary, what a Bayesian does is two steps: (1) determine a prior (2) follow the math mechanically. What frequentists do when they do inference is actually much more ill-defined, as it required hidden assumptions intermixed with a mathematical derivation instead of cleanly separating assumptions from maths. In principle the bayesian way is: given assumptions, simply follow the axioms of probability to their logical conclusions. Of course that may not always be computationally feasible, so approximations have to be made sometimes. Frequentists require divine inspiration to develop a new method for each kind of problem. The bayesian way to solve the problem would be to take a prior probability distribution on the probability of heads, and then condition that on the data that the coin ended up on heads 6 times and on tails 4 times. For example with a uniform prior, we get this posterior: http://www.wolframalpha.com/input/?i=1*p%5E6%281-p%29%5E4%2Fintegral%281*q%5E6%281-q%29%5E4%2Cq%3D0..1%29 http://www.wolframalpha.com/input/?i=1*p%5E6%281-p%29%5E4%2F... As your can see the peak is indeed at p=0.6, but there is probability mass around it as well. Replace the factors of 1 with your favorite prior to see the effect (I couldn't convince wolfram alpha to compute a beta distribution in place of the 1, so you'll have to try that yourself).