4 ms·

xkcd : Frequentists vs. Bayesians : https://xkcd.com/1132/ https://xkcd.com/1132/

by pella 11y ago

pella 11y ago
"What's wrong with XKCD's Frequentists vs. Bayesians comic?" http://stats.stackexchange.com/questions/43339/whats-wrong-with-xkcds-frequentists-vs-bayesians-comic http://stats.stackexchange.com/questions/43339/whats-wrong-w...
- ricksplat 11y ago
  The accepted answer seems wrong to me - he says that the frequentist is reasoning based on the Sun as the repeatable experiment when in fact it's the dice-roll, I would have thought ...
- deleted 11y ago
  [deleted]
  - deleted 11y ago
    [deleted]
- raverbashing 11y ago
  "The main issue is that the first experiment (Sun gone nova) is not repeatable, which makes it highly unsuitable for frequentist methodology" Is that the best they can come up with? For all we know the probabiliy of the sun going Nova is zero. And 1/36 is not impossible. Hence the detector is lying. Now, we can do the joke going the other way by thinking of another processes with conditional probabilities "expertly guessed"
  - ricksplat 11y ago
    1/36 is the probability that the detector is lying, and so when it says "yes" that means it's a 1/36 probability that the sun hasn't gone nova, not the probability that it could. $50 is only really any use if the sun hasn't gone nova (cause otherwise we're all dead), so therefore even if the odds are bad it's still a good bet. EDIT: Also, a penny just dropped after reading another of the answers. The odds against the sun having gone nova are incredibly mind-bogglingly high such that the the 1/36 probability is negligible. So even if the probability that the machine is lying is small at 1/36 it's still far far far more likely than the sun having gone nova - which is the "prior" that the Bayesian is taking into account. Another rub might also be that the frequentist's significance (P-value) is not sufficient to take into account the astronomical scale of the Nova probability but IANAS so that's just speculation.
    - raverbashing 11y ago
      Yes, what I mean is that, if you got a "yes" from the detector, it either means that: - The sun hasn't gone nova and the detector is lying (with 1/36 chance) - The sun has gone nova and the detector is not lying (negligible) Now, given the result, one can calculate the probability of which of the cases And the betting aspect of it makes sense as well
      - bonoboTP 11y ago
        The "1/36 chance" and the "negligible" aren't the corresponding/analogous probabilities. 1/36 is P(detector "yes" | not nova). The "negligible" is P(nova). What we want to calculate is P(nova | detector "yes"). Using Bayes' rule it's P(nova | detector "yes") = P(detector "yes" | nova) P(nova) / (P(detector "yes" | nova) P(nova) + P(detector "yes" | not nova) P(not nova)) = 35/36 * P_nova / (35/36 * P_nova + 1/36 * (1-P_nova)) P(nova | detector "yes") > 0.5 if P_nova > 1/36. This is also quite intuitive. Unless going nova is more likely than malfunctioning, you should bet on malfunctioning. This is very much like David Hume's quote: "no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavors to establish." (test: would you say the same amount of "holy crap! WTF!!" if it turned out that the testimony was false, compared to if it turned out that the miraculous thing actually happened). So: no detector "yes" is sufficient to establish the sun going nova, unless the detector saying "yes" be such, that it lying (1/36) would be more miraculous, than the sun going nova. It's fascinating how precisely this can be formulated and how clear it is with probability theory.
- Scarblac 11y ago
  Intuitively the main problem is that the chance of the machine lying is enormously higher than the chance of the sun going nova. The result of combining them says essentially nothing about the latter anymore. What's wrong is that it's a useless machine.
  - SideburnsOfDoom 11y ago
    I always thought that it was more about the difficulty of collecting winnings on the bet in the "sun actually gone nova" case. That holds not matter what the odds.
  - ricksplat 11y ago
    It's only useless 1/36 times ... if you ask it a few times you could make a reasonable inference.
adwn 11y ago
"Frequentists vs. Bayesians"? The problem with this comic is that it paints statisticians as belonging to one of two opposing tribes, when really frequentist and Baysian statistics are two different methods in the toolbox of a competent statistician – which of them is the appropriate one depends on the circumstances and the problem at hand. I'm not sure where this idea of an alleged divide comes from, but I think it might have to do with a certain small but vocal group on the Internet [1]. [1] cough Eliezer Yudkowsky cough
- bonoboTP 11y ago
  No, the philosophical debate exists and has a history. Also Bayesian is quite a buzzword that people use in paper titles just because it sounds like some seal of approval, even if it just means that they used Bayes' rule somewhere in the derivation of their formula. Anyway, using Bayes' formula is not the same as philosophical Bayesianism. Bayes' formula is simply a theorem of mathematics that follows from the axioms of probability theory. It's not controversial. The controversy comes from the applicability of Bayesian philosophy. In theory and ideally, it seems nice, but there are some problems with it. One well known problem is how to specifying your priors. A bigger issue that is often forgotten is that you can't calculate the conditional probability P(explanation | observation), without computing P(observation) = SUM_i [ P(observation | explanation_i) P(explanation_i) ] for the denominator of the Bayes formula. Now, this means that you have to consider all possible explanations. But systematically scanning through all possible hypotheses is really difficult. Usually it takes rare moments of random genious ideas to come up with a hypothesis that works well. You could say this is a "global optimization" problem in hypothesis space. Now, if you restrict the search for a particular class of hypotheses (a limited-flexibility model), then you can use computational methods, like Monte Carlo methods, to approximate this sum (integral) over a set of hypotheses. But you can never be sure if your modeling restrictions were valid or maybe the best explanation is outside "the box". When I hear Internet people (bloggers, commenters) saying they use Bayesianism in their everyday life it's almost always about pulling numbers out of their asses and then feeling great about how smart and rational they are. Everyday life is just too messy, there are many possibilities that are hard to capture in a model. Addendum: I also really hate how Lesswrongians hijacked and overmystify the term "Bayesian" (and even "rationalist") as if it were some panacea/silver bullet that magically turns you into some superhero. It's like a "one weird trick" BS just for smarter people.

xkcd : Frequentists vs. Bayesians : https://xkcd.com/1132/ https://xkcd.com/1132/

by pella 11y ago

xkcd : Frequentists vs. Bayesians : https://xkcd.com/1132/ https://xkcd.com/1132/

pella 11y ago

"What's wrong with XKCD's Frequentists vs. Bayesians comic?" http://stats.stackexchange.com/questions/43339/whats-wrong-with-xkcds-frequentists-vs-bayesians-comic http://stats.stackexchange.com/questions/43339/whats-wrong-w...

ricksplat 11y ago
The accepted answer seems wrong to me - he says that the frequentist is reasoning based on the Sun as the repeatable experiment when in fact it's the dice-roll, I would have thought ...
deleted 11y ago
[deleted]
- deleted 11y ago
  [deleted]
raverbashing 11y ago
"The main issue is that the first experiment (Sun gone nova) is not repeatable, which makes it highly unsuitable for frequentist methodology" Is that the best they can come up with? For all we know the probabiliy of the sun going Nova is zero. And 1/36 is not impossible. Hence the detector is lying. Now, we can do the joke going the other way by thinking of another processes with conditional probabilities "expertly guessed"
- ricksplat 11y ago
  1/36 is the probability that the detector is lying, and so when it says "yes" that means it's a 1/36 probability that the sun hasn't gone nova, not the probability that it could. $50 is only really any use if the sun hasn't gone nova (cause otherwise we're all dead), so therefore even if the odds are bad it's still a good bet. EDIT: Also, a penny just dropped after reading another of the answers. The odds against the sun having gone nova are incredibly mind-bogglingly high such that the the 1/36 probability is negligible. So even if the probability that the machine is lying is small at 1/36 it's still far far far more likely than the sun having gone nova - which is the "prior" that the Bayesian is taking into account. Another rub might also be that the frequentist's significance (P-value) is not sufficient to take into account the astronomical scale of the Nova probability but IANAS so that's just speculation.
  - raverbashing 11y ago
    Yes, what I mean is that, if you got a "yes" from the detector, it either means that: - The sun hasn't gone nova and the detector is lying (with 1/36 chance) - The sun has gone nova and the detector is not lying (negligible) Now, given the result, one can calculate the probability of which of the cases And the betting aspect of it makes sense as well
    - bonoboTP 11y ago
      The "1/36 chance" and the "negligible" aren't the corresponding/analogous probabilities. 1/36 is P(detector "yes" | not nova). The "negligible" is P(nova). What we want to calculate is P(nova | detector "yes"). Using Bayes' rule it's P(nova | detector "yes") = P(detector "yes" | nova) P(nova) / (P(detector "yes" | nova) P(nova) + P(detector "yes" | not nova) P(not nova)) = 35/36 * P_nova / (35/36 * P_nova + 1/36 * (1-P_nova)) P(nova | detector "yes") > 0.5 if P_nova > 1/36. This is also quite intuitive. Unless going nova is more likely than malfunctioning, you should bet on malfunctioning. This is very much like David Hume's quote: "no testimony is sufficient to establish a miracle, unless the testimony be of such a kind, that its falsehood would be more miraculous, than the fact, which it endeavors to establish." (test: would you say the same amount of "holy crap! WTF!!" if it turned out that the testimony was false, compared to if it turned out that the miraculous thing actually happened). So: no detector "yes" is sufficient to establish the sun going nova, unless the detector saying "yes" be such, that it lying (1/36) would be more miraculous, than the sun going nova. It's fascinating how precisely this can be formulated and how clear it is with probability theory.
Scarblac 11y ago
Intuitively the main problem is that the chance of the machine lying is enormously higher than the chance of the sun going nova. The result of combining them says essentially nothing about the latter anymore. What's wrong is that it's a useless machine.
- SideburnsOfDoom 11y ago
  I always thought that it was more about the difficulty of collecting winnings on the bet in the "sun actually gone nova" case. That holds not matter what the odds.
- ricksplat 11y ago
  It's only useless 1/36 times ... if you ask it a few times you could make a reasonable inference.

adwn 11y ago

"Frequentists vs. Bayesians"? The problem with this comic is that it paints statisticians as belonging to one of two opposing tribes, when really frequentist and Baysian statistics are two different methods in the toolbox of a competent statistician – which of them is the appropriate one depends on the circumstances and the problem at hand. I'm not sure where this idea of an alleged divide comes from, but I think it might have to do with a certain small but vocal group on the Internet [1]. [1] cough Eliezer Yudkowsky cough

bonoboTP 11y ago
No, the philosophical debate exists and has a history. Also Bayesian is quite a buzzword that people use in paper titles just because it sounds like some seal of approval, even if it just means that they used Bayes' rule somewhere in the derivation of their formula. Anyway, using Bayes' formula is not the same as philosophical Bayesianism. Bayes' formula is simply a theorem of mathematics that follows from the axioms of probability theory. It's not controversial. The controversy comes from the applicability of Bayesian philosophy. In theory and ideally, it seems nice, but there are some problems with it. One well known problem is how to specifying your priors. A bigger issue that is often forgotten is that you can't calculate the conditional probability P(explanation | observation), without computing P(observation) = SUM_i [ P(observation | explanation_i) P(explanation_i) ] for the denominator of the Bayes formula. Now, this means that you have to consider all possible explanations. But systematically scanning through all possible hypotheses is really difficult. Usually it takes rare moments of random genious ideas to come up with a hypothesis that works well. You could say this is a "global optimization" problem in hypothesis space. Now, if you restrict the search for a particular class of hypotheses (a limited-flexibility model), then you can use computational methods, like Monte Carlo methods, to approximate this sum (integral) over a set of hypotheses. But you can never be sure if your modeling restrictions were valid or maybe the best explanation is outside "the box". When I hear Internet people (bloggers, commenters) saying they use Bayesianism in their everyday life it's almost always about pulling numbers out of their asses and then feeling great about how smart and rational they are. Everyday life is just too messy, there are many possibilities that are hard to capture in a model. Addendum: I also really hate how Lesswrongians hijacked and overmystify the term "Bayesian" (and even "rationalist") as if it were some panacea/silver bullet that magically turns you into some superhero. It's like a "one weird trick" BS just for smarter people.