5 ms·

I've probably heard 1000 Bayesians rant about the alleged Frequentist consensus and 0 Frequentists complain about Bayesian analysis. I'd need some pretty steep

I've probably heard 1000 Bayesians rant about the alleged Frequentist consensus and 0 Frequentists complain about Bayesian analysis. I'd need some pretty steep priors to interpret this as anything other than the academic equivalent of "one WEIRD trick THEY don't want you to know" marketing.

paulsutter 3y ago
If we were allowed one super-upvote per month, I would use mine right now on your comment
- selimthegrim 3y ago
  Dang is going to file this under the Bumble model of HN monetization in his desk drawer
  - causality0 3y ago
    There are times I'd be willing to pay a fiver to un-flag my comment.
  - jowea 3y ago
    Or just HN Gold
throwawaymaths 3y ago
There's always room for a first time. Bayesian statistics is sound but I suspect it's often just used to justify biases. It is technically valid to use a prior and de facto never update it, because you know I'll get around to updating my prior next week, or... eventually, cough cough let's be honest, never
- marcosdumay 3y ago
  Bayesian statistics is the one where you must state your bias explicitly, and justify them one by one under the light. It's the frequentist one that gets your biases implicitly, on the form of corrections and hypothesis formulation, so that people don't notice them.
  - derbOac 3y ago
    I don't know. I've published Bayesian methodology papers and reviewed them and although I might call myself an objective Bayesianist maybe, Bayesianism has its own implicit biases. In the very least, I think it can take advantage of implicit biases people have in interpreting analyses. One issue is that Bayes estimates are almost always produced, even if no information is coming from empirical data, and all the information is coming from a prior. So it's possible to produce results heavily influenced by the prior with Bayesian estimation that with frequentist methods would fail completely because of lack of identification of the model, sending a strong signal that something is wrong. This can all be sussed out with Bayesian methods but people often don't do it. Another more subtle issue is people aren't quite aware of how a prior can deviate from "maximal conservativism". Sometimes, for example, depending on the model, a very flat prior is actually not conservative, and is overweighting tails. There's other examples too. Basically, yes, Bayesianism forces you to be explicit with your biases, but people are really bad at interpreting the actual impact of those biases in a formal Bayesian framework, or at least, aren't any better at it than with frequentist methods that are available. If you approach statistical inference from the perspective of accuracy (as the linked paper seems to do) Bayesianism is better to the extent the priors are accurate. This is true a lot of the time empirically, but it does lead to a kind of tautology, in that you're doing the analysis because you don't really know what "truth" is. So if you're right in your priors, Bayesianism is accurate, but then you didn't really need new data as much in the first place; if you're wrong, it's more biased. Basically in the bias-variance tradeoff, Bayesianism makes a bet on reduced variance assuming that the resulting bias will be small enough. Philosophically, though, there's a completely different argument, which is one of competitive fairness. You might say this doesn't matter, but consider consequential decisions, like hiring or admissions decisions: if someone was making a prediction about you, would you want them to use a strong prior, or something that's maximally conservative and fair? This philosophy leads to frequentism basically. My preference is to be maximally conservative in a Bayesian framework, which leads to reference priors, which are often flat in many canonical situations, which is basically frequentism. In other situations you might have a different kind of prior. To me the linked paper is pretty interesting and makes a good point. On the other hand, I'd rather not make any assumptions about a new result based on past studies on other effects. I'd rather just collect lots of diverse real data and meta-analyze it. There's no substitute for data -- and that includes priors.
exe34 3y ago
Every single statistics class I've ever taken or been sent to has been 95% frequentist with a rushed Bayesian digression near the end. I've submitted papers with Bayesian work and the reviewer asked for p-values and would not budge. I gave him his bloody p-values, because I could not afford not to get the paper published that early in my career.
- jdewerd 3y ago
  There's always a more complicated model.
  - zozbot234 3y ago
    The "more complicated" version of frequentist statistics is called robust statistics. There's most likely a way to rephrase your favorite Bayesian analysis so as to make it fully kosher from a "robust+frequentist" point of view, even keeping the math unchanged. It just goes to show how silly the "controversy" is.
  - PheonixPharts 3y ago
    Bayesian statistics is fundamentally less complicated than Frequentist statistics since everything can be derived from a very simple set of first principles, rather than complex frameworks of ad hoc testing methodologies.
    - ivansavz 3y ago
      > Bayesian statistics is fundamentally less complicated than Frequentist statistics [...] I broadly agree with you, but I'm wondering if you would reconsider your qualification as "less complicated" if you consider beginner learners. E.g. someone who knows basic descriptive statistics and probability theory, and is making first contact with inferential statistics. Specifically, assume a learner who knows what an integral is, but is far from proficient with it (UGRAD student, not a GRAD student). I was reading this paper[1] recently, which highlights two difficulties of teaching Bayesian stats: 1) the mathematical complexity of understanding conditional probability distributions, and 2) the lack of well defined, broadly accepted conventions for what priors to use in specific data analysis scenarios. I think a computational approach to prob theory could mitigate 1), but 2) remains a problem—the freedom to choose priors, is also a burden... [1] https://www.stat.purdue.edu/~dsmoore/articles/BayesPedagogy.pdf https://www.stat.purdue.edu/~dsmoore/articles/BayesPedagogy....
light_hue_1 3y ago
Yes, because Frequentists won a long time ago. And are responsible for producing garbage science ever since. Of course the winners don't rant about anything. But any time we probe the consequences of Frequentist statistics they turn out to be horrific for science, our health, and our planet.
- therobots927 3y ago
  How so? What’s an example of that?
abecedarius 3y ago
The frequentist ranting was concentrated around a century ago +/- one lifetime, and they called the enemy "inverse probability" (when being polite).
- kgwgk 3y ago
  "Inverse probability" is just what statistical inference was called before frequentist ideas were introduced. It was in the second half of the last century when the "Bayesian" label was introduced - amusingly in opposition to the much newer "classical" methods.
deleted 3y ago
[deleted]

paulsutter 3y ago

If we were allowed one super-upvote per month, I would use mine right now on your comment

selimthegrim 3y ago
Dang is going to file this under the Bumble model of HN monetization in his desk drawer
- causality0 3y ago
  There are times I'd be willing to pay a fiver to un-flag my comment.
- jowea 3y ago
  Or just HN Gold

throwawaymaths 3y ago

There's always room for a first time. Bayesian statistics is sound but I suspect it's often just used to justify biases. It is technically valid to use a prior and de facto never update it, because you know I'll get around to updating my prior next week, or... eventually, cough cough let's be honest, never

marcosdumay 3y ago
Bayesian statistics is the one where you must state your bias explicitly, and justify them one by one under the light. It's the frequentist one that gets your biases implicitly, on the form of corrections and hypothesis formulation, so that people don't notice them.
- derbOac 3y ago
  I don't know. I've published Bayesian methodology papers and reviewed them and although I might call myself an objective Bayesianist maybe, Bayesianism has its own implicit biases. In the very least, I think it can take advantage of implicit biases people have in interpreting analyses. One issue is that Bayes estimates are almost always produced, even if no information is coming from empirical data, and all the information is coming from a prior. So it's possible to produce results heavily influenced by the prior with Bayesian estimation that with frequentist methods would fail completely because of lack of identification of the model, sending a strong signal that something is wrong. This can all be sussed out with Bayesian methods but people often don't do it. Another more subtle issue is people aren't quite aware of how a prior can deviate from "maximal conservativism". Sometimes, for example, depending on the model, a very flat prior is actually not conservative, and is overweighting tails. There's other examples too. Basically, yes, Bayesianism forces you to be explicit with your biases, but people are really bad at interpreting the actual impact of those biases in a formal Bayesian framework, or at least, aren't any better at it than with frequentist methods that are available. If you approach statistical inference from the perspective of accuracy (as the linked paper seems to do) Bayesianism is better to the extent the priors are accurate. This is true a lot of the time empirically, but it does lead to a kind of tautology, in that you're doing the analysis because you don't really know what "truth" is. So if you're right in your priors, Bayesianism is accurate, but then you didn't really need new data as much in the first place; if you're wrong, it's more biased. Basically in the bias-variance tradeoff, Bayesianism makes a bet on reduced variance assuming that the resulting bias will be small enough. Philosophically, though, there's a completely different argument, which is one of competitive fairness. You might say this doesn't matter, but consider consequential decisions, like hiring or admissions decisions: if someone was making a prediction about you, would you want them to use a strong prior, or something that's maximally conservative and fair? This philosophy leads to frequentism basically. My preference is to be maximally conservative in a Bayesian framework, which leads to reference priors, which are often flat in many canonical situations, which is basically frequentism. In other situations you might have a different kind of prior. To me the linked paper is pretty interesting and makes a good point. On the other hand, I'd rather not make any assumptions about a new result based on past studies on other effects. I'd rather just collect lots of diverse real data and meta-analyze it. There's no substitute for data -- and that includes priors.

exe34 3y ago

Every single statistics class I've ever taken or been sent to has been 95% frequentist with a rushed Bayesian digression near the end. I've submitted papers with Bayesian work and the reviewer asked for p-values and would not budge. I gave him his bloody p-values, because I could not afford not to get the paper published that early in my career.

jdewerd 3y ago
There's always a more complicated model.
- zozbot234 3y ago
  The "more complicated" version of frequentist statistics is called robust statistics. There's most likely a way to rephrase your favorite Bayesian analysis so as to make it fully kosher from a "robust+frequentist" point of view, even keeping the math unchanged. It just goes to show how silly the "controversy" is.
- PheonixPharts 3y ago
  Bayesian statistics is fundamentally less complicated than Frequentist statistics since everything can be derived from a very simple set of first principles, rather than complex frameworks of ad hoc testing methodologies.
  - ivansavz 3y ago
    > Bayesian statistics is fundamentally less complicated than Frequentist statistics [...] I broadly agree with you, but I'm wondering if you would reconsider your qualification as "less complicated" if you consider beginner learners. E.g. someone who knows basic descriptive statistics and probability theory, and is making first contact with inferential statistics. Specifically, assume a learner who knows what an integral is, but is far from proficient with it (UGRAD student, not a GRAD student). I was reading this paper[1] recently, which highlights two difficulties of teaching Bayesian stats: 1) the mathematical complexity of understanding conditional probability distributions, and 2) the lack of well defined, broadly accepted conventions for what priors to use in specific data analysis scenarios. I think a computational approach to prob theory could mitigate 1), but 2) remains a problem—the freedom to choose priors, is also a burden... [1] https://www.stat.purdue.edu/~dsmoore/articles/BayesPedagogy.pdf https://www.stat.purdue.edu/~dsmoore/articles/BayesPedagogy....

light_hue_1 3y ago

Yes, because Frequentists won a long time ago. And are responsible for producing garbage science ever since. Of course the winners don't rant about anything. But any time we probe the consequences of Frequentist statistics they turn out to be horrific for science, our health, and our planet.

therobots927 3y ago
How so? What’s an example of that?

abecedarius 3y ago

The frequentist ranting was concentrated around a century ago +/- one lifetime, and they called the enemy "inverse probability" (when being polite).

kgwgk 3y ago
"Inverse probability" is just what statistical inference was called before frequentist ideas were introduced. It was in the second half of the last century when the "Bayesian" label was introduced - amusingly in opposition to the much newer "classical" methods.

deleted 3y ago

[deleted]