9 ms·

Thomas Bayes and the crisis in science

105 points by agonz253 8y ago

hooloovoo_zoo 8y ago
It's amusing that the people who are most militantly Bayesian aren't Bayesian statisticians. It's almost as if there are advantages and drawbacks to the Bayesian perspective. By the way, frequentism vs. Bayesianism has very little to do with Thomas Bayes. All statisticians accept the validity of Bayes' Theorem. Moreover, the two approaches are not mutually exclusive. It's always interesting to investigate the frequentist properties of Bayesian estimators and the implicit priors of frequentist estimators.
- nkozyra 8y ago
  That seems like a vast generalization across the board.
- RobertRoberts 8y ago
  I don't mean to segue here, but I got caught in a logical argument that ended abruptly with no definitive answer. I mentioned the Monty Hall problem to a friend (an engineer) and discussed the statical analysis done on this issue. (ie, statistically it's better to switch doors after the first is opened, as it improves your odds of winning slightly) But he only answered with "nope, Bayes' Theorem says the odds are the same no matter what you do". The only other point he added was that if you do something only once, you have the same odds every single time. I found this frustrating simplistic, because we _know_ through testing that this isn't true. That the odds are better if you switch doors. Is this the fanaticism of Bayesians? He even called me a "frequentist" as if it were some kind of pejorative. I've researched this issue, and even mentioned to him his logical flaw (you choose twice, not once) and still "Bayes theorum says there's no difference and you're wrong because you are a frequentist" sigh.
  - LeanderK 8y ago
    I am not really sure what this has to do with bayesian vs. frequentists? the bayesian theorem is a mathematical fact. It's true if we just accept some basic axioms. Bayesian/Frequentist divide is also pretty independent from the problem, since it's purely math-based? We are not estimating something...In fact, an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities.
    - RobertRoberts 8y ago
      >...an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities. It seems simple, but the claim (not mine) was that the theorum states that since you are only choosing "once" you can't use an estimation, as that is based on collected data... which somehow doesn't exist or apply to you. The reason is that somehow there are no odds/percentages/probabilities if there is only one instance of an event, or at least they don't change. (ie, 2 door, always 1 in 2 odds, period) Normally I wouldn't detract from a thread like this, but participation seems low, so I can take the down votes in hopes of getting a response that helps me with this conundrum.
      - jkabrg 8y ago
        Bayesian statistics is a way of updating your beliefs about the value of some unknown number (or value) after doing an experiment. You have to be uncertain about some variable, and then you work backwards to figure out its value. In the case of the Monty Hall problem, we have a complete understanding of what's going on, so there are no beliefs to update. There are no hidden rules or magic numbers we haven't been told about and we need to work out. You literally can't use Bayesian statistics if you understand a system completely. Also, Bayesian statistics is a tool. It's not a law of nature. It can get the wrong answers. There is literally no guarantee that you will get the right answer when using it. Garbage In Garbage Out. It's just that there are cases where, empirically, it can be useful.
        RobertRoberts 8y ago
        Thanks for this explanation, it certainly expels some of my confusion. It seems then based on your reply that my friend is possibly wrong for invoking Bayesian theory regarding the Monty Hall problem for the exact reason he claimed it was the only answer? (note, my friend is a brilliant person and dedicated engineer who understands math far above my level. I only program with statistics and the PhD's tell me what to do.) In other words, his argument is incorrect in his assessment because we actually _do_ have previous knowledge? (He pointed out that Bayes theorum was used for analysing insurance premiums for new kinds of insurance because there was no previous data to work with, which fits with your statement.) This seems like too easy an answer for him to have missed, which makes me suspect of my understanding of your reply. (yes, my initial question was serious, I appreciate you replying)
  - jkabrg 8y ago
    Does this comment deserve a downvote? I think he's asking a genuine question.
  - selimthegrim 8y ago
    He’s wrong, you can show switching doors improves your performance with Bayes’ theorem.
    - RobertRoberts 8y ago
      Edit: I removed my entire comment, it seems that there is a lot more online now about Bayesian theory than when I had looked this up, or my google-fu was off when I was researching this before. Thanks for giving me new search terms to consider. I have more arguments and examples to read through now. :P
      - rvern 8y ago
        “Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, ‘Do you want to pick door No. 2?’ Is it to your advantage to switch your choice?” Let A be the event that the car is behind door 1, B that the car is behind door 2, and C that the car is behind door 3. Let E be the event that the host opens door 3. We assume that the car is initially equally likely to be behind each door and that the host opens a door with a goat at random, never opening the door we picked. Since A, B, and C are exhaustive and mutually exhaustive propositions, we can calculate the marginal probability of E by using the law of total probability: P(E) = P(E ∧ A) + P(E ∧ B) + P(E ∧ C). Bayesians like to define joint probability from conditional probability instead of the reverse; that is, define P(A ∧ B) as P(A | B) P(B) instead of P(A | B) as P(A ∧ B) / P(B). So P(E ∧ A) = P(E | A) P(A). P(E | A) is 1/2 because we picked door 1, the car is behind door 1, and the host chooses at random a door that has a goat, of which there are two: 2 and 3. P(A) is 1/3. Therefore P(E ∧ A) is 1/2 × 1/3 = 1/6. Similarly, P(E ∧ B) = P(E | B) P(B). P(E | B) is 1 because we picked door 1 so the host will not open door 1 and we assume the car is behind door 2 so the host will not open door 2, leaving only door 3 to be opened. P(B) is 1/3. Therefore P(E ∧ B) is 1 × 1/3 = 1/3. P(E ∧ C) = P(E | C) P(C). P(E | C) is 0 because the host will never open the door the car is behind. P(C) is 1/3. Therefore P(E ∧ C) is 0 × 1/3 = 0. So P(E) = 1/6 + 1/3 + 0 = 1/2. We know that the host opened door 3 (this is E), so the car cannot be behind door 3. How likely is it to be behind door 1? By Bayes’ theorem, P(A | E) = P(E | A) P(A) / P(E). We said earlier that P(E | A) is 1/2, P(A) is 1/3, and P(E) is 1/2. So P(A | E) = (1/2 × 1/3) / (1/2) = 1/3. Given E, the car must be behind door 1 or door 2 since the host opened door 3. Therefore the sum of P(A | E) and P(B | E) must be 1. P(A | E) is 1/3, so P(B | E) is 2/3. The car is more likely to be behind door 2 than door 1. We initially picked door 1, so, if we want the car, we should switch.
  - rvern 8y ago
    Bayes’ theorem says no such thing. Your friend is wrong, but please do not let this reflect badly on the Bayesian interpretation of probability in general—it has nothing to do with this.
    - pash 8y ago
      And in fact a Bayesian approach, by being explicit about conditioning information, helps to show the “trick” behind the Monty Hall problem: when Hall opens a door, he reveals information about what’s behind the other two doors that the contestant didn’t have when he made his initial choice. The probabilities change in response to this new information.
  - AlexCoventry 8y ago
    The easiest way to convince your friend of his error is to imagine the odds when there are n doors, and Monty opens n-2 of them, the ones which don't contain the prize and aren't your first pick.
    - sova 8y ago
      Brilliant. Yes, the wrong solutions are highlighted by the door opening event, and thus your odds of success increase when you eliminate bad outcomes.
    - DougBTX 8y ago
      How could Bayesian statistics be applied in this case? I’m wondering if the situation is just too “simple” to make it applicable. Before Monty opens a door, so the prior probability, is 1/3 for each door. After he opens a door, that door will have zero probability, as we know he opens a door without a car, then how to update the probabilities afterwards? Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen. Chosen is 1/3 and not chosen is 2/3 before and after Monty opens a door, so perhaps there is no “Bayesian information” revealed by opening the door anyway.
      - Bromskloss 8y ago
        > Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen. Right. That might be the easiest way for this problem. More straightforwardly, without that shortcut: - Call the doors a, b, and c. Assume, without loss of generality, that we choose door a initially. Let the random variable X ∈ {a, b, c} be the door with the car. - Our prior probability is uniformly distributed: Pr(X = a) = Pr(X = b) = Pr(X = c) = 1/3. - The data Y that we collect is our observation of which door gets opened by the host. The likelihood function Pr(Y = y | X = x) is the probability of the observation being y (i.e., that Y = y), given that the underlying state is x (i.e., that X = x). The only non-zero likelihoods are Pr(Y = b | X = a) = Pr(Y = c | X = a) = 1/2 and Pr(Y = c | X = b) = Pr(Y = b | X = c) = 1. - Bayes' theorem, Pr(X = x | Y = y) = Pr(Y = y | X = x)·Pr(X = x)/Pr(Y = y), gives the answer, the posterior probability, which should be seen as a function of x. The denominator Pr(Y = y) = ∑ Pr(Y = y | X = x), sum over x ∈ {a, b, c}, is a normalising factor that makes the posterior probability distribution sum to 1. It is also the probability we, at the start of the game, assign to Y = y. In our problem, Pr(Y = a) = 0 and Pr(Y = b) = Pr(Y = c) = 1/2. How about you put in the numbers and see if it comes out right or if I have made a mistake? :-) Edit: Sorry, I just realised that we could have made it simpler by assuming that the host opens, say, door b.
  - md224 8y ago
    Here's how I would explain the Monty Hall problem: First, get the person to realize that switching doors will always flip your initial outcome. If you picked a losing door, switching gives you the winning one. If you picked the winning door, switching gives you a losing one. There is no winning-to-winning or losing-to-losing. If you switch, you will always get the opposite of whatever you originally picked. Once this idea has been understood, ask them to consider the probability that your original choice was a losing door. Once they agree that it's 2/3 (2 losing doors out of 3 possible choices), you can reply "that's right, and if there's a 2/3 chance that your current choice is a losing door, then there's a 2/3 chance that you're in a situation where switching will give you the winning door. Therefore, the chance that switching will give you the winning door is 2/3!"
  - Bromskloss 8y ago
    > "frequentist" as if it were some kind of pejorative. Ah, so he was right about that, at least! :-)
  - eli_gottlieb 8y ago
    Judea Pearl uses the Monty Hall problem in his latest book, and when he draws out the causal network, the issue becomes obvious. The game-show host chooses which door to open based on two factors: he has to choose a door you didn't, and he has to choose a door with a goat. Hence, the door he chooses to open is probabilistically informative about your initial choice. Treating the door he opened as information on which to condition does change the posterior.
- jhbadger 8y ago
  I think the that the virulent all-or-nothing Bayesian vs. frequentist debates are really more from earlier decades. These days most people are open to using techniques from both camps and are more interested in solving problems than being philosophically "pure".
- trentmb 8y ago
  Not gonna lie- I don't think I understand the difference. Every explanation seems to bounce off and over my thick skull.
- Bromskloss 8y ago
  > Moreover, the two approaches are not mutually exclusive. I would say that they are based on two different view of what probability is, and that subscribing to one means that you cannot accept the other.
rossdavidh 8y ago
On the one hand, I have (in the semiconductor manufacturing industry) encountered statisticians who were greatly averse to making any prior assumptions about the likelihood of something. Also, any article with the phrase "nonsense on stilts" in it is not entirely unworthy. It also does a credible job of explaining Bayes' theorem. On the other hand, I think it somewhat exaggerates the extent to which Bayes' theorem and its associated work are rejected by statisticians, and also the extent to which a magical p > 0.05 limit was advocated. I believe it was non-statisticians who wanted easy tests that they could apply without understanding much about statistics, who were to blame in that case.
- eli_gottlieb 8y ago
  Statistics is formalized philosophy of science, so of course scientists outside the stats departments wanted "easy tests". Arguably, a lot of the new applications of computational Bayesian methods in the natural science are the same statistical witch-doctoring with a bit more rigor. (Ok, I do computational Bayes for a living with natural scientists, so I'm inherently biased to view our work as shakier than it really is.)
  - Karrot_Kream 8y ago
    Aside: What do you do for a living?
    - AlexCoventry 8y ago
      LinkedIn in his HN user page. > Variational Bayesian methods and deep learning for affective neuroscience, using probabilistic programming systems ProbTorch and Pyro. It does sound like he's in for a rough time, epistemologically speaking.
      - eli_gottlieb 8y ago
        You'd be surprised how easily the witch-doctory passes for real science ;-)! Kidding, kidding, our empirical side actually has to be at least as rigorous as the other statistical methods used in neuroscience, usually more so. That's what they keep us around for!
eli_gottlieb 8y ago
>We are living in new Bayesian age. Applications of Bayesian probability are taking over our lives. Doctors, lawyers, engineers and financiers use computerized Bayesian networks to aid their decision-making. Psychologists and neuroscientists explore the Bayesian workings of our brains. Statisticians increasingly rely on Bayesian logic. Even our email spam filters work on Bayesian principles. While that's true, strictly speaking, it's akin to writing that, "We are living in a new logical age" when Boolean algebra was first finding wide application in engineering and the natural sciences. "Bayesian methods" just mean using statistical modeling techniques that conform to Bayes' rule as their normative guide. In "machine learning" or "frequentist" terms, it just means that good approximate-Bayesian reasoning minimizes the KL divergence between the true posterior and the approximate model (whether variational or by sampling or by training a neural network, whatever), as opposed to minimizing the classification hinge-loss or the mean squared error (though some of those losses have formally equivalent Bayesian priors).
- plassma 8y ago
  Strictly speaking, health care and genocide are the same thing, you just minimize different metrics.
gfodor 8y ago
Is there a metholodgy for bayesian analysis that avoids the choice of a specific prior but instead provides conclusions in the form of boundary/regions in "prior-space" and their effect on belief? For example, it would be incredibly useful if the output of research allowed a reader to gauge support of the conclusion in a minimally subjective way by explaining what effect choices in prior have on results. I'm not a statistician so I'm assuming this is a well understood thing, but would be curious to know if and how it is practiced.
- hooloovoo_zoo 8y ago
  Generally speaking, a good Bayesian analysis includes what's known as a "sensitivity analysis" which seeks to measure how sensitive the results are to the particular choice of prior. Additionally, if strong prior assumptions are not available, an "uninformative" prior is used. In such cases, the results tend to be pretty close to those from frequentist methods, except the frequentist methods lack the Bayesian probabilistic interpretation.
- Bromskloss 8y ago
  The final result of a Bayesian analysis is the posterior probability, which is the product of the prior probability (which can incorporate previous knowledge or convictions) and the likelihood (which is determined by the data you just collected). You can leave out the prior, and just report the likelihood. Another comment mentions this: https://news.ycombinator.com/item?id=17438643 https://news.ycombinator.com/item?id=17438643
Eliezer 8y ago
For a more detailed take advocating the particular solution "report likelihoods, not posteriors or p-values", see "Likelihoods, p-values, and the replication crisis": https://arbital.com/p/likelihoods_not_pvalues/?l=4xx https://arbital.com/p/likelihoods_not_pvalues/?l=4xx
AlexCoventry 8y ago
You can abuse Bayesian methods just as easily you can hack a p-value. Maybe more easily, since fewer people would be aware of the issues. What's needed is a shift in researcher attitudes and incentives, to emphasize development of reliable knowledge instead of publication record. Just changing the rules of the game slightly will only lead to people adjusting their game slightly. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf http://www.stat.columbia.edu/~gelman/research/unpublished/p_...
- outlace 8y ago
  Yes but with traditional frequentist approaches most scientists don’t understand what’s going on under the hood of the statistical tools they’re using. There are dozens and dozens of named tests like “Fisher’s exact test” and “Wilcoxon signed-rank test” and most scientists just sort of follow received wisdom about which test to use in which situation. Bayesian methods force you to actually think about how your data and model parameters are distributed, and explicitly specify a model.
  - eli_gottlieb 8y ago
    >Bayesian methods force you to actually think about how your data and model parameters are distributed, and explicitly specify a model. Yep. I know some very senior scientists who get a lot of mileage out of finding places where "model-free" methods are implicitly assuming a bloody stupid model, and then attacking them.
PGenes 8y ago
Awesome article! I'll be digesting this for some time to come.

105 points by agonz253 8y ago

It's amusing that the people who are most militantly Bayesian aren't Bayesian statisticians. It's almost as if there are advantages and drawbacks to the Bayesian perspective. By the way, frequentism vs. Bayesianism has very little to do with Thomas Bayes. All statisticians accept the validity of Bayes' Theorem. Moreover, the two approaches are not mutually exclusive. It's always interesting to investigate the frequentist properties of Bayesian estimators and the implicit priors of frequentist estimators.

nkozyra 8y ago
That seems like a vast generalization across the board.
RobertRoberts 8y ago
I don't mean to segue here, but I got caught in a logical argument that ended abruptly with no definitive answer. I mentioned the Monty Hall problem to a friend (an engineer) and discussed the statical analysis done on this issue. (ie, statistically it's better to switch doors after the first is opened, as it improves your odds of winning slightly) But he only answered with "nope, Bayes' Theorem says the odds are the same no matter what you do". The only other point he added was that if you do something only once, you have the same odds every single time. I found this frustrating simplistic, because we _know_ through testing that this isn't true. That the odds are better if you switch doors. Is this the fanaticism of Bayesians? He even called me a "frequentist" as if it were some kind of pejorative. I've researched this issue, and even mentioned to him his logical flaw (you choose twice, not once) and still "Bayes theorum says there's no difference and you're wrong because you are a frequentist" sigh.
- LeanderK 8y ago
  I am not really sure what this has to do with bayesian vs. frequentists? the bayesian theorem is a mathematical fact. It's true if we just accept some basic axioms. Bayesian/Frequentist divide is also pretty independent from the problem, since it's purely math-based? We are not estimating something...In fact, an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities.
  - RobertRoberts 8y ago
    >...an easy solution to the monty-hall problem is to just apply the base theorem and calculate the probabilities. It seems simple, but the claim (not mine) was that the theorum states that since you are only choosing "once" you can't use an estimation, as that is based on collected data... which somehow doesn't exist or apply to you. The reason is that somehow there are no odds/percentages/probabilities if there is only one instance of an event, or at least they don't change. (ie, 2 door, always 1 in 2 odds, period) Normally I wouldn't detract from a thread like this, but participation seems low, so I can take the down votes in hopes of getting a response that helps me with this conundrum.
    - jkabrg 8y ago
      Bayesian statistics is a way of updating your beliefs about the value of some unknown number (or value) after doing an experiment. You have to be uncertain about some variable, and then you work backwards to figure out its value. In the case of the Monty Hall problem, we have a complete understanding of what's going on, so there are no beliefs to update. There are no hidden rules or magic numbers we haven't been told about and we need to work out. You literally can't use Bayesian statistics if you understand a system completely. Also, Bayesian statistics is a tool. It's not a law of nature. It can get the wrong answers. There is literally no guarantee that you will get the right answer when using it. Garbage In Garbage Out. It's just that there are cases where, empirically, it can be useful.
      - RobertRoberts 8y ago
        Thanks for this explanation, it certainly expels some of my confusion. It seems then based on your reply that my friend is possibly wrong for invoking Bayesian theory regarding the Monty Hall problem for the exact reason he claimed it was the only answer? (note, my friend is a brilliant person and dedicated engineer who understands math far above my level. I only program with statistics and the PhD's tell me what to do.) In other words, his argument is incorrect in his assessment because we actually _do_ have previous knowledge? (He pointed out that Bayes theorum was used for analysing insurance premiums for new kinds of insurance because there was no previous data to work with, which fits with your statement.) This seems like too easy an answer for him to have missed, which makes me suspect of my understanding of your reply. (yes, my initial question was serious, I appreciate you replying)
- jkabrg 8y ago
  Does this comment deserve a downvote? I think he's asking a genuine question.
- selimthegrim 8y ago
  He’s wrong, you can show switching doors improves your performance with Bayes’ theorem.
  - RobertRoberts 8y ago
    Edit: I removed my entire comment, it seems that there is a lot more online now about Bayesian theory than when I had looked this up, or my google-fu was off when I was researching this before. Thanks for giving me new search terms to consider. I have more arguments and examples to read through now. :P
    - rvern 8y ago
      “Suppose you’re on a game show, and you’re given the choice of three doors: Behind one door is a car; behind the others, goats. You pick a door, say No. 1, and the host, who knows what’s behind the doors, opens another door, say No. 3, which has a goat. He then says to you, ‘Do you want to pick door No. 2?’ Is it to your advantage to switch your choice?” Let A be the event that the car is behind door 1, B that the car is behind door 2, and C that the car is behind door 3. Let E be the event that the host opens door 3. We assume that the car is initially equally likely to be behind each door and that the host opens a door with a goat at random, never opening the door we picked. Since A, B, and C are exhaustive and mutually exhaustive propositions, we can calculate the marginal probability of E by using the law of total probability: P(E) = P(E ∧ A) + P(E ∧ B) + P(E ∧ C). Bayesians like to define joint probability from conditional probability instead of the reverse; that is, define P(A ∧ B) as P(A | B) P(B) instead of P(A | B) as P(A ∧ B) / P(B). So P(E ∧ A) = P(E | A) P(A). P(E | A) is 1/2 because we picked door 1, the car is behind door 1, and the host chooses at random a door that has a goat, of which there are two: 2 and 3. P(A) is 1/3. Therefore P(E ∧ A) is 1/2 × 1/3 = 1/6. Similarly, P(E ∧ B) = P(E | B) P(B). P(E | B) is 1 because we picked door 1 so the host will not open door 1 and we assume the car is behind door 2 so the host will not open door 2, leaving only door 3 to be opened. P(B) is 1/3. Therefore P(E ∧ B) is 1 × 1/3 = 1/3. P(E ∧ C) = P(E | C) P(C). P(E | C) is 0 because the host will never open the door the car is behind. P(C) is 1/3. Therefore P(E ∧ C) is 0 × 1/3 = 0. So P(E) = 1/6 + 1/3 + 0 = 1/2. We know that the host opened door 3 (this is E), so the car cannot be behind door 3. How likely is it to be behind door 1? By Bayes’ theorem, P(A | E) = P(E | A) P(A) / P(E). We said earlier that P(E | A) is 1/2, P(A) is 1/3, and P(E) is 1/2. So P(A | E) = (1/2 × 1/3) / (1/2) = 1/3. Given E, the car must be behind door 1 or door 2 since the host opened door 3. Therefore the sum of P(A | E) and P(B | E) must be 1. P(A | E) is 1/3, so P(B | E) is 2/3. The car is more likely to be behind door 2 than door 1. We initially picked door 1, so, if we want the car, we should switch.
- rvern 8y ago
  Bayes’ theorem says no such thing. Your friend is wrong, but please do not let this reflect badly on the Bayesian interpretation of probability in general—it has nothing to do with this.
  - pash 8y ago
    And in fact a Bayesian approach, by being explicit about conditioning information, helps to show the “trick” behind the Monty Hall problem: when Hall opens a door, he reveals information about what’s behind the other two doors that the contestant didn’t have when he made his initial choice. The probabilities change in response to this new information.
- AlexCoventry 8y ago
  The easiest way to convince your friend of his error is to imagine the odds when there are n doors, and Monty opens n-2 of them, the ones which don't contain the prize and aren't your first pick.
  - sova 8y ago
    Brilliant. Yes, the wrong solutions are highlighted by the door opening event, and thus your odds of success increase when you eliminate bad outcomes.
  - DougBTX 8y ago
    How could Bayesian statistics be applied in this case? I’m wondering if the situation is just too “simple” to make it applicable. Before Monty opens a door, so the prior probability, is 1/3 for each door. After he opens a door, that door will have zero probability, as we know he opens a door without a car, then how to update the probabilities afterwards? Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen. Chosen is 1/3 and not chosen is 2/3 before and after Monty opens a door, so perhaps there is no “Bayesian information” revealed by opening the door anyway.
    - Bromskloss 8y ago
      > Seems the simple way to look at it is to not partition by door, but by chosen vs not chosen. Right. That might be the easiest way for this problem. More straightforwardly, without that shortcut: - Call the doors a, b, and c. Assume, without loss of generality, that we choose door a initially. Let the random variable X ∈ {a, b, c} be the door with the car. - Our prior probability is uniformly distributed: Pr(X = a) = Pr(X = b) = Pr(X = c) = 1/3. - The data Y that we collect is our observation of which door gets opened by the host. The likelihood function Pr(Y = y | X = x) is the probability of the observation being y (i.e., that Y = y), given that the underlying state is x (i.e., that X = x). The only non-zero likelihoods are Pr(Y = b | X = a) = Pr(Y = c | X = a) = 1/2 and Pr(Y = c | X = b) = Pr(Y = b | X = c) = 1. - Bayes' theorem, Pr(X = x | Y = y) = Pr(Y = y | X = x)·Pr(X = x)/Pr(Y = y), gives the answer, the posterior probability, which should be seen as a function of x. The denominator Pr(Y = y) = ∑ Pr(Y = y | X = x), sum over x ∈ {a, b, c}, is a normalising factor that makes the posterior probability distribution sum to 1. It is also the probability we, at the start of the game, assign to Y = y. In our problem, Pr(Y = a) = 0 and Pr(Y = b) = Pr(Y = c) = 1/2. How about you put in the numbers and see if it comes out right or if I have made a mistake? :-) Edit: Sorry, I just realised that we could have made it simpler by assuming that the host opens, say, door b.
- md224 8y ago
  Here's how I would explain the Monty Hall problem: First, get the person to realize that switching doors will always flip your initial outcome. If you picked a losing door, switching gives you the winning one. If you picked the winning door, switching gives you a losing one. There is no winning-to-winning or losing-to-losing. If you switch, you will always get the opposite of whatever you originally picked. Once this idea has been understood, ask them to consider the probability that your original choice was a losing door. Once they agree that it's 2/3 (2 losing doors out of 3 possible choices), you can reply "that's right, and if there's a 2/3 chance that your current choice is a losing door, then there's a 2/3 chance that you're in a situation where switching will give you the winning door. Therefore, the chance that switching will give you the winning door is 2/3!"
- Bromskloss 8y ago
  > "frequentist" as if it were some kind of pejorative. Ah, so he was right about that, at least! :-)
- eli_gottlieb 8y ago
  Judea Pearl uses the Monty Hall problem in his latest book, and when he draws out the causal network, the issue becomes obvious. The game-show host chooses which door to open based on two factors: he has to choose a door you didn't, and he has to choose a door with a goat. Hence, the door he chooses to open is probabilistically informative about your initial choice. Treating the door he opened as information on which to condition does change the posterior.
jhbadger 8y ago
I think the that the virulent all-or-nothing Bayesian vs. frequentist debates are really more from earlier decades. These days most people are open to using techniques from both camps and are more interested in solving problems than being philosophically "pure".
trentmb 8y ago
Not gonna lie- I don't think I understand the difference. Every explanation seems to bounce off and over my thick skull.
Bromskloss 8y ago
> Moreover, the two approaches are not mutually exclusive. I would say that they are based on two different view of what probability is, and that subscribing to one means that you cannot accept the other.

rossdavidh 8y ago

On the one hand, I have (in the semiconductor manufacturing industry) encountered statisticians who were greatly averse to making any prior assumptions about the likelihood of something. Also, any article with the phrase "nonsense on stilts" in it is not entirely unworthy. It also does a credible job of explaining Bayes' theorem. On the other hand, I think it somewhat exaggerates the extent to which Bayes' theorem and its associated work are rejected by statisticians, and also the extent to which a magical p > 0.05 limit was advocated. I believe it was non-statisticians who wanted easy tests that they could apply without understanding much about statistics, who were to blame in that case.

eli_gottlieb 8y ago
Statistics is formalized philosophy of science, so of course scientists outside the stats departments wanted "easy tests". Arguably, a lot of the new applications of computational Bayesian methods in the natural science are the same statistical witch-doctoring with a bit more rigor. (Ok, I do computational Bayes for a living with natural scientists, so I'm inherently biased to view our work as shakier than it really is.)
- Karrot_Kream 8y ago
  Aside: What do you do for a living?
  - AlexCoventry 8y ago
    LinkedIn in his HN user page. > Variational Bayesian methods and deep learning for affective neuroscience, using probabilistic programming systems ProbTorch and Pyro. It does sound like he's in for a rough time, epistemologically speaking.
    - eli_gottlieb 8y ago
      You'd be surprised how easily the witch-doctory passes for real science ;-)! Kidding, kidding, our empirical side actually has to be at least as rigorous as the other statistical methods used in neuroscience, usually more so. That's what they keep us around for!

eli_gottlieb 8y ago

>We are living in new Bayesian age. Applications of Bayesian probability are taking over our lives. Doctors, lawyers, engineers and financiers use computerized Bayesian networks to aid their decision-making. Psychologists and neuroscientists explore the Bayesian workings of our brains. Statisticians increasingly rely on Bayesian logic. Even our email spam filters work on Bayesian principles. While that's true, strictly speaking, it's akin to writing that, "We are living in a new logical age" when Boolean algebra was first finding wide application in engineering and the natural sciences. "Bayesian methods" just mean using statistical modeling techniques that conform to Bayes' rule as their normative guide. In "machine learning" or "frequentist" terms, it just means that good approximate-Bayesian reasoning minimizes the KL divergence between the true posterior and the approximate model (whether variational or by sampling or by training a neural network, whatever), as opposed to minimizing the classification hinge-loss or the mean squared error (though some of those losses have formally equivalent Bayesian priors).

plassma 8y ago
Strictly speaking, health care and genocide are the same thing, you just minimize different metrics.

gfodor 8y ago

Is there a metholodgy for bayesian analysis that avoids the choice of a specific prior but instead provides conclusions in the form of boundary/regions in "prior-space" and their effect on belief? For example, it would be incredibly useful if the output of research allowed a reader to gauge support of the conclusion in a minimally subjective way by explaining what effect choices in prior have on results. I'm not a statistician so I'm assuming this is a well understood thing, but would be curious to know if and how it is practiced.

hooloovoo_zoo 8y ago
Generally speaking, a good Bayesian analysis includes what's known as a "sensitivity analysis" which seeks to measure how sensitive the results are to the particular choice of prior. Additionally, if strong prior assumptions are not available, an "uninformative" prior is used. In such cases, the results tend to be pretty close to those from frequentist methods, except the frequentist methods lack the Bayesian probabilistic interpretation.
Bromskloss 8y ago
The final result of a Bayesian analysis is the posterior probability, which is the product of the prior probability (which can incorporate previous knowledge or convictions) and the likelihood (which is determined by the data you just collected). You can leave out the prior, and just report the likelihood. Another comment mentions this: https://news.ycombinator.com/item?id=17438643 https://news.ycombinator.com/item?id=17438643

Eliezer 8y ago

For a more detailed take advocating the particular solution "report likelihoods, not posteriors or p-values", see "Likelihoods, p-values, and the replication crisis": https://arbital.com/p/likelihoods_not_pvalues/?l=4xx https://arbital.com/p/likelihoods_not_pvalues/?l=4xx

AlexCoventry 8y ago

You can abuse Bayesian methods just as easily you can hack a p-value. Maybe more easily, since fewer people would be aware of the issues. What's needed is a shift in researcher attitudes and incentives, to emphasize development of reliable knowledge instead of publication record. Just changing the rules of the game slightly will only lead to people adjusting their game slightly. http://www.stat.columbia.edu/~gelman/research/unpublished/p_hacking.pdf http://www.stat.columbia.edu/~gelman/research/unpublished/p_...

outlace 8y ago
Yes but with traditional frequentist approaches most scientists don’t understand what’s going on under the hood of the statistical tools they’re using. There are dozens and dozens of named tests like “Fisher’s exact test” and “Wilcoxon signed-rank test” and most scientists just sort of follow received wisdom about which test to use in which situation. Bayesian methods force you to actually think about how your data and model parameters are distributed, and explicitly specify a model.
- eli_gottlieb 8y ago
  >Bayesian methods force you to actually think about how your data and model parameters are distributed, and explicitly specify a model. Yep. I know some very senior scientists who get a lot of mileage out of finding places where "model-free" methods are implicitly assuming a bloody stupid model, and then attacking them.

PGenes 8y ago

Awesome article! I'll be digesting this for some time to come.