I'm not an expert, but my opinion mostly comes from this seminar by Michael Jordan: Are you a Bayesian or a Frequentist? http://videolectures.net/mlss09uk_jorda

I'm not an expert, but my opinion mostly comes from this seminar by Michael Jordan: Are you a Bayesian or a Frequentist? http://videolectures.net/mlss09uk_jordan_bfway/ http://videolectures.net/mlss09uk_jordan_bfway/ Around the 10-15 minute mark, he talks about how the two schools of thought naturally arise from conditioning over the two arguments of the loss function, either conditioning over the models or over the data. (Incidentally, he also characterizes Bayesianism as optimistic and Frequentism as pessimistic. So maybe I'm wrong in that regard above.)