3 ms·

Statistics researchers in academia definitely express a preference between frequentist and Bayesian inference. So no, it's not limited to philosophers - and I'm

Statistics researchers in academia definitely express a preference between frequentist and Bayesian inference. So no, it's not limited to philosophers - and I'm pretty sure all but the most forgiving frequentists would not agree with the concept "frequentist = Bayesian with flat priors". Remember that you cannot really formulate a flat infinite (and infinitely thin) distribution in mathematical terms: you need to resort to limit calculation. While the concept "Bayesian + flat prior = frequentist" is useful to explain a high level connection between the two worlds, there is a lot more to the topic - and in my opinion, it's something that can hardly be scratched without a formal education in the field.

judk 12y ago
Why is "flat prior" here? A requentist method assumes a certain prior, for example Normal
- kgwgk 12y ago
  Maybe you mean that a frequentist method assumes a certain model, for example normal. A frequentist method will sometimes give similar results to a Bayesian analysis that uses a non-informative prior. For example, if we want to estimate a value from repeated measurements with Gaussian noise the frequentist result is equivalent to the Bayesian result if a "flat" (improper) prior is used. [http://en.wikipedia.org/wiki/Jeffreys_prior#Gaussian_distribution_with_mean_parameter http://en.wikipedia.org/wiki/Jeffreys_prior#Gaussian_distrib...]
mjw 12y ago
The other problem with the "Bayes with flat prior = frequentist maximum likelihood" idea is that, even if you ignore the issues with improper priors, the concept of a "flat prior" is inherently dependent on arbitrary choices in the way a model is parameterised. It's not possible for a prior to be "flat" with respect to all re-parameterisations of a continuous parameter in a model. E.g. a flat prior for the variance isn't flat for its inverse (precision) or its square root (the std. dev.), and the choice of which of these alternative parameterisations you use to express the unknown quantity in the model is arbitrary. In the frequentist case it doesn't affect the result of the inference; in the Bayesian case it matters which of the parameterisations you choose your prior to be flat with respect to.
- mjw 12y ago
  If this seems a bit odd (and it did to me at first!) think about it this way: Bayesian methods work by averaging over a bunch of different models / different values of the parameters. What it means to compute a mean depends on the parameterisation in which you do it: simplest example being that an arithmetic mean is not in general the same as a geometric mean, or a harmonic mean. There's no "neutral" / parameterisation-independent way to specify how this averaging is done, so if you care about the average case, you're going to have commit to doing it some particular favoured parameterisation. Choosing that parameterisation is equivalent to choosing the prior. Frequentist methods avoid the need for this decision; the price they pay is that without a prior they're unable to condition on the observed data. They must consider every parameter value and its resulting sampling distribution separately and can't average over them.

judk 12y ago

Why is "flat prior" here? A requentist method assumes a certain prior, for example Normal

kgwgk 12y ago
Maybe you mean that a frequentist method assumes a certain model, for example normal. A frequentist method will sometimes give similar results to a Bayesian analysis that uses a non-informative prior. For example, if we want to estimate a value from repeated measurements with Gaussian noise the frequentist result is equivalent to the Bayesian result if a "flat" (improper) prior is used. [http://en.wikipedia.org/wiki/Jeffreys_prior#Gaussian_distribution_with_mean_parameter http://en.wikipedia.org/wiki/Jeffreys_prior#Gaussian_distrib...]

mjw 12y ago

The other problem with the "Bayes with flat prior = frequentist maximum likelihood" idea is that, even if you ignore the issues with improper priors, the concept of a "flat prior" is inherently dependent on arbitrary choices in the way a model is parameterised. It's not possible for a prior to be "flat" with respect to all re-parameterisations of a continuous parameter in a model. E.g. a flat prior for the variance isn't flat for its inverse (precision) or its square root (the std. dev.), and the choice of which of these alternative parameterisations you use to express the unknown quantity in the model is arbitrary. In the frequentist case it doesn't affect the result of the inference; in the Bayesian case it matters which of the parameterisations you choose your prior to be flat with respect to.

mjw 12y ago
If this seems a bit odd (and it did to me at first!) think about it this way: Bayesian methods work by averaging over a bunch of different models / different values of the parameters. What it means to compute a mean depends on the parameterisation in which you do it: simplest example being that an arithmetic mean is not in general the same as a geometric mean, or a harmonic mean. There's no "neutral" / parameterisation-independent way to specify how this averaging is done, so if you care about the average case, you're going to have commit to doing it some particular favoured parameterisation. Choosing that parameterisation is equivalent to choosing the prior. Frequentist methods avoid the need for this decision; the price they pay is that without a prior they're unable to condition on the observed data. They must consider every parameter value and its resulting sampling distribution separately and can't average over them.