5 ms·

I'm not much of a statistician. Can anybody explain to me why Bayesian vs Frequentist is such a political or religious issue? It seems to me they must either

by adamtj 13y ago

I'm not much of a statistician. Can anybody explain to me why Bayesian vs Frequentist is such a political or religious issue? It seems to me they must either describe the same underlying reality correctly, though differently, or else one is objectively wrong. If they are both correct but different, can we not just characterize the differences and use whichever is most appropriate? It it just a matter of taste with differing opinions on the correctness (or morality?) of the guessing that's necessary for Bayesian analysis? Can we not just figure out the error as a function of the wrongness of the guess and be done with it? Are there guesses so wrong that no amount of data is sufficient for Bayesian and Frequentist analyses to converge?

jmount 13y ago
A lot of the calculations (including the ability to use Bayes law) is common to Frequentists and Bayesians. Efron says the difference is Frequentists assume the quantity to be modeled is unknown but has a single fixed value (so you can't make probability statements about it) and the measurements have some possible variability. Bayesians model the unknown with an explicit prior distribution for the unknown quantity to be inferred and assume the measurements are fixed. So each school means something different when they state a probability (Frequentist variation over data draws, Bayesian variation over the unknown to be measured). Gelman defines Bayesian methods as the proper setting up of a generative model and computing joint probabilities of parameters and data. In his view Frequentist don't compute joint probabilities but at best conditional probabilities (which assume out modeling likely values for the unknown quantity to be inferred). The joke of the comment is: both sides tend to snipe on the other (as neither framework looks good if you assume the other). My partner has a nice article on the topic: http://www.win-vector.com/blog/2013/05/bayesian-and-frequentist-approaches-ask-the-right-question/ http://www.win-vector.com/blog/2013/05/bayesian-and-frequent...
- adamtj 13y ago
  Thanks! I got that it was a joke, I've just never understood why it was funny. The "debate" has always seemed to me like Quantumist physicists arguing with Classicalists about which method of computing a cannonball trajectory is correct. I mean, clearly the Classicalist interpretation overestimates the effectiveness of cannons by failing to account for the chance that the ball may quantum-tunnel through its target, leaving it undamaged. Your comment and that article really help convince me that there exist people who actually understand what's going on.
Fomite 13y ago
Bayesian priors aren't necessarily formed by "Guessing", and if we do advance the notion that they are "Guesses", Frequentist statistics has its own set of them.
moultano 13y ago
Frequentists and Bayesians both agree on all of the math. The difference lies in the interpretation of what the math means, and how useful different types of math are. A rough summary: Frequentist methods tend to try to characterize the question of interest in terms of a single measurable statistic. This is particularly useful in circumstances where memory/cpu is tight (i.e. every single circumstance before the last few decades.) This necessarily means though that these techniques lose much of the information in the data. They also try to make very few a priori assumptions about the underlying data. The bayesian critique of this is that the assumptions are still there, just hidden, and as a result are much more likely to be wrong. Bayesian methods tend to try to characterize the question of interest in terms of the entire distribution of outcomes. This causes there to be little information lost between the data and the conclusions, but is often computationally intractable in common circumstances. Bayesians tend to make strong a priori assumptions about the form the data will take, and to ameliorate this they make their assumptions very explicit in the model. The Frequentist critique of this is that every time you read a bayesian piece of analysis, you have to be very careful to determine whether the outcome described is merely the assumed prior + noise. Both cultures provide essential mathematical tools for any statistician's toolbox, and it's often just a matter of taste which get used. Hence the bike-shedding.

by adamtj 13y ago

jmount 13y ago

A lot of the calculations (including the ability to use Bayes law) is common to Frequentists and Bayesians. Efron says the difference is Frequentists assume the quantity to be modeled is unknown but has a single fixed value (so you can't make probability statements about it) and the measurements have some possible variability. Bayesians model the unknown with an explicit prior distribution for the unknown quantity to be inferred and assume the measurements are fixed. So each school means something different when they state a probability (Frequentist variation over data draws, Bayesian variation over the unknown to be measured). Gelman defines Bayesian methods as the proper setting up of a generative model and computing joint probabilities of parameters and data. In his view Frequentist don't compute joint probabilities but at best conditional probabilities (which assume out modeling likely values for the unknown quantity to be inferred). The joke of the comment is: both sides tend to snipe on the other (as neither framework looks good if you assume the other). My partner has a nice article on the topic: http://www.win-vector.com/blog/2013/05/bayesian-and-frequentist-approaches-ask-the-right-question/ http://www.win-vector.com/blog/2013/05/bayesian-and-frequent...

adamtj 13y ago
Thanks! I got that it was a joke, I've just never understood why it was funny. The "debate" has always seemed to me like Quantumist physicists arguing with Classicalists about which method of computing a cannonball trajectory is correct. I mean, clearly the Classicalist interpretation overestimates the effectiveness of cannons by failing to account for the chance that the ball may quantum-tunnel through its target, leaving it undamaged. Your comment and that article really help convince me that there exist people who actually understand what's going on.

Fomite 13y ago

Bayesian priors aren't necessarily formed by "Guessing", and if we do advance the notion that they are "Guesses", Frequentist statistics has its own set of them.

moultano 13y ago

Frequentists and Bayesians both agree on all of the math. The difference lies in the interpretation of what the math means, and how useful different types of math are. A rough summary: Frequentist methods tend to try to characterize the question of interest in terms of a single measurable statistic. This is particularly useful in circumstances where memory/cpu is tight (i.e. every single circumstance before the last few decades.) This necessarily means though that these techniques lose much of the information in the data. They also try to make very few a priori assumptions about the underlying data. The bayesian critique of this is that the assumptions are still there, just hidden, and as a result are much more likely to be wrong. Bayesian methods tend to try to characterize the question of interest in terms of the entire distribution of outcomes. This causes there to be little information lost between the data and the conclusions, but is often computationally intractable in common circumstances. Bayesians tend to make strong a priori assumptions about the form the data will take, and to ameliorate this they make their assumptions very explicit in the model. The Frequentist critique of this is that every time you read a bayesian piece of analysis, you have to be very careful to determine whether the outcome described is merely the assumed prior + noise. Both cultures provide essential mathematical tools for any statistician's toolbox, and it's often just a matter of taste which get used. Hence the bike-shedding.