4 ms·

The follow-ups are also well worth reading: http://jakevdp.github.io/blog/2014/06/06/frequentism-and-bayesianism-2-when-results-differ/ http://jakevdp.github.i

The follow-ups are also well worth reading: http://jakevdp.github.io/blog/2014/06/06/frequentism-and-bayesianism-2-when-results-differ/ http://jakevdp.github.io/blog/2014/06/06/frequentism-and-bay... http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/ http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bay... http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bayesianism-4-bayesian-in-python/ http://jakevdp.github.io/blog/2014/06/14/frequentism-and-bay...

nkurz 12y ago
In case it's not clear from the beginning where he really stands on the matter, in Part 3 he offers his opinion on the relative merits of both approaches: <spoiler> The moral of the story is that frequentism and Science do not mix. Let me say it directly: you should be suspicious of the use of frequentist confidence intervals and p-values in science. In a scientific setting, confidence intervals, and closely-related p-values, provide the correct answer to the wrong question. In particular, if you ever find someone stating or implying that a 95% confidence interval is 95% certain to contain a parameter of interest, do not trust their interpretation or their results. If you happen to be peer-reviewing the paper, reject it. Their data do not back-up their conclusion. </spoiler>
- judk 12y ago
  Um, the folks at CERN don't agree with your assessment of frequentism in science. Their papers are explicitly frequentist, they celebrate results based on how many "sigmas" they get. P-value all the aym
  - nkurz 12y ago
    My interpretation? If not clear, I'm quoting the penultimate paragraph of the last post in the series, right before he thanks the reader for making it to the end: http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/#The-Moral-of-the-Story:-Frequentism-and-Science-Do-Not-Mix http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bay...
  - Houshalter 12y ago
    Yes frequentism is common in science and the author strongly objects to it. Read the whole post: https://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/ https://jakevdp.github.io/blog/2014/06/12/frequentism-and-ba...
  - bayesianhorse 12y ago
    I'm not a physicist, but as far as I know, CERN has access to tons of data with every intention of drowning any prior belief. In this setting, I would expect frequentist methods to shine, and bayesian method to be intractable. However I would not expect CERN papers to make the kinds of terminological / theoretical lapses about confidence the parent thread was talking about. Papers should be rejected for that kind of error, even if you are not a bayesian.
    - betatim 12y ago
      Exactly this. At CERN (and all of HEP elsewhere) the Bayes vs Freq wars were fought many years ago and are long over. The conclusion: when you have lots of data they converge (as they should!). In the case that they differ you almost always find that you have very few observations. I would argue that this 'difference' is not that exciting because it must be dominated by your assumptions, not your observations. After all once you accumulate enough observations the two methods tend to converge. Personal conclusion: if the methods disagree work on getting more data instead of fighting over which method is better.
anon4 12y ago
That first followup seems patently insane. "We don't know the value of p̂, so we'll assume it's 5/8". Why would you ever assume like that? The Bayesian approach appears to me exactly like the Frequentist, without the part where they assume convenient values of unknown parameters.

nkurz 12y ago

In case it's not clear from the beginning where he really stands on the matter, in Part 3 he offers his opinion on the relative merits of both approaches: <spoiler> The moral of the story is that frequentism and Science do not mix. Let me say it directly: you should be suspicious of the use of frequentist confidence intervals and p-values in science. In a scientific setting, confidence intervals, and closely-related p-values, provide the correct answer to the wrong question. In particular, if you ever find someone stating or implying that a 95% confidence interval is 95% certain to contain a parameter of interest, do not trust their interpretation or their results. If you happen to be peer-reviewing the paper, reject it. Their data do not back-up their conclusion. </spoiler>

judk 12y ago
Um, the folks at CERN don't agree with your assessment of frequentism in science. Their papers are explicitly frequentist, they celebrate results based on how many "sigmas" they get. P-value all the aym
- nkurz 12y ago
  My interpretation? If not clear, I'm quoting the penultimate paragraph of the last post in the series, right before he thanks the reader for making it to the end: http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/#The-Moral-of-the-Story:-Frequentism-and-Science-Do-Not-Mix http://jakevdp.github.io/blog/2014/06/12/frequentism-and-bay...
- Houshalter 12y ago
  Yes frequentism is common in science and the author strongly objects to it. Read the whole post: https://jakevdp.github.io/blog/2014/06/12/frequentism-and-bayesianism-3-confidence-credibility/ https://jakevdp.github.io/blog/2014/06/12/frequentism-and-ba...
- bayesianhorse 12y ago
  I'm not a physicist, but as far as I know, CERN has access to tons of data with every intention of drowning any prior belief. In this setting, I would expect frequentist methods to shine, and bayesian method to be intractable. However I would not expect CERN papers to make the kinds of terminological / theoretical lapses about confidence the parent thread was talking about. Papers should be rejected for that kind of error, even if you are not a bayesian.
  - betatim 12y ago
    Exactly this. At CERN (and all of HEP elsewhere) the Bayes vs Freq wars were fought many years ago and are long over. The conclusion: when you have lots of data they converge (as they should!). In the case that they differ you almost always find that you have very few observations. I would argue that this 'difference' is not that exciting because it must be dominated by your assumptions, not your observations. After all once you accumulate enough observations the two methods tend to converge. Personal conclusion: if the methods disagree work on getting more data instead of fighting over which method is better.

anon4 12y ago

That first followup seems patently insane. "We don't know the value of p̂, so we'll assume it's 5/8". Why would you ever assume like that? The Bayesian approach appears to me exactly like the Frequentist, without the part where they assume convenient values of unknown parameters.