4 ms·

I don't think this paper teaches the difference between Bayesianism and Frequentism (though in fairness to the author, that may not be the goal). The paper says

I don't think this paper teaches the difference between Bayesianism and Frequentism (though in fairness to the author, that may not be the goal). The paper says Bayesianism is 'aggressive' and 'subjective' while Frequentism is 'cautious' and aims for 'universally acceptable' conclusions. I wouldn't characterize them that way. Frequentism is also subjective (http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequently_subjective/ http://lesswrong.com/lw/1gc/frequentist_statistics_are_frequ...) and Bayesianism can be quite cautious. To me, the real difference in Bayesian and Frequentist methods are the types of questions they aim to answer. Bayesian methods try to answer the question of "What is the probability of the model" given some observed data. Frequentist methods try to answer the question of "What is the probability of the observed data" given an assumed model. In this sense, they are somewhat complementary, which why many practicing statisticians don't actually care about the Bayesianism-Frequentism debate. I am happy that Bayesian methods are becoming more popular, though, since the questions we actually care about ("what is the probability of the model given the data?") are often best suited by Bayesian methods. Also, as a physicist I'm disappointed that the paper asserts that physicists disdained statistics in 1903. For the most part, statistical mechanics was developed in the late 1800s! (https://en.wikipedia.org/wiki/Statistical_mechanics#History https://en.wikipedia.org/wiki/Statistical_mechanics#History)

bkcooper 12y ago
Also, as a physicist I'm disappointed that the paper asserts that physicists disdained statistics in 1903. For the most part, statistical mechanics was developed in the late 1800s! Sure, but Boltzmann's ideas were famously controversial at the time. I don't think they represent a good measure of mainstream practice at the time.
ced 12y ago
It's a bit of a tangent from your main point, but: Bayesian methods try to answer the question of "What is the probability of the model" given some observed data There are people arguing that P(M|D) isn't really a meaningful quantity, because the space of models is infinite, and we can't reasonably put a prior on it. I prefer talking about P(M1|D)/P(M2|D): "The data favors model 1 three times more than it favors model 2" The philosophical problem with P(M|D) is the same as distinguishing randomness from non-randomness (cf. Popperian falsifiability): there's no test to prove that something is truly random, all we can say is "We've looked for patterns over and over again, but we haven't found one yet".
gwern 12y ago
> The paper says Bayesianism is 'aggressive' and 'subjective' while Frequentism is 'cautious' and aims for 'universally acceptable' conclusions. I wouldn't characterize them that way. Worst-case vs average-case seems like the relevant distinction here. ( http://lesswrong.com/lw/f7t/beyond_bayesians_and_frequentists/ http://lesswrong.com/lw/f7t/beyond_bayesians_and_frequentist... http://lesswrong.com/lw/jne/a_fervent_defense_of_frequentist_statistics/ http://lesswrong.com/lw/jne/a_fervent_defense_of_frequentist... ) > Frequentist methods try to answer the question of "What is the probability of the observed data" given an assumed model. Isn't that more null-hypothesis testing...? Frequentism is broader.
- tedsanders 12y ago
  I'm not an expert, but my opinion mostly comes from this seminar by Michael Jordan: Are you a Bayesian or a Frequentist? http://videolectures.net/mlss09uk_jordan_bfway/ http://videolectures.net/mlss09uk_jordan_bfway/ Around the 10-15 minute mark, he talks about how the two schools of thought naturally arise from conditioning over the two arguments of the loss function, either conditioning over the models or over the data. (Incidentally, he also characterizes Bayesianism as optimistic and Frequentism as pessimistic. So maybe I'm wrong in that regard above.)

bkcooper 12y ago

Also, as a physicist I'm disappointed that the paper asserts that physicists disdained statistics in 1903. For the most part, statistical mechanics was developed in the late 1800s! Sure, but Boltzmann's ideas were famously controversial at the time. I don't think they represent a good measure of mainstream practice at the time.

ced 12y ago

It's a bit of a tangent from your main point, but: Bayesian methods try to answer the question of "What is the probability of the model" given some observed data There are people arguing that P(M|D) isn't really a meaningful quantity, because the space of models is infinite, and we can't reasonably put a prior on it. I prefer talking about P(M1|D)/P(M2|D): "The data favors model 1 three times more than it favors model 2" The philosophical problem with P(M|D) is the same as distinguishing randomness from non-randomness (cf. Popperian falsifiability): there's no test to prove that something is truly random, all we can say is "We've looked for patterns over and over again, but we haven't found one yet".

gwern 12y ago

> The paper says Bayesianism is 'aggressive' and 'subjective' while Frequentism is 'cautious' and aims for 'universally acceptable' conclusions. I wouldn't characterize them that way. Worst-case vs average-case seems like the relevant distinction here. ( http://lesswrong.com/lw/f7t/beyond_bayesians_and_frequentists/ http://lesswrong.com/lw/f7t/beyond_bayesians_and_frequentist... http://lesswrong.com/lw/jne/a_fervent_defense_of_frequentist_statistics/ http://lesswrong.com/lw/jne/a_fervent_defense_of_frequentist... ) > Frequentist methods try to answer the question of "What is the probability of the observed data" given an assumed model. Isn't that more null-hypothesis testing...? Frequentism is broader.

tedsanders 12y ago
I'm not an expert, but my opinion mostly comes from this seminar by Michael Jordan: Are you a Bayesian or a Frequentist? http://videolectures.net/mlss09uk_jordan_bfway/ http://videolectures.net/mlss09uk_jordan_bfway/ Around the 10-15 minute mark, he talks about how the two schools of thought naturally arise from conditioning over the two arguments of the loss function, either conditioning over the models or over the data. (Incidentally, he also characterizes Bayesianism as optimistic and Frequentism as pessimistic. So maybe I'm wrong in that regard above.)