5 ms·
The difference between a frequentist and a Bayesian is that the latter admits that he picks a prior. A frequentist smushes together (1) the statistical assumpti
by jules 7y ago
The difference between a frequentist and a Bayesian is that the latter admits that he picks a prior. A frequentist smushes together (1) the statistical assumptions (2) the approximations that make the problem computationally tractable and (3) the mathematical derivations, into one big mess. Just because you're not stating your assumptions doesn't mean there are none. Consider maximum likelihood estimation. It is not invariant under coordinate transformations. So which coordinates you pick is an assumption. In fact, with Bayesian estimation you can do the same thing: picking a prior is equivalent to picking the uniform prior in a different coordinate system. So frequentist estimation does involve picking a prior by picking a coordinate system, even if the frequentist does not admit this.
Frequentist methods are conceptually anything but straightforward. The advantage of frequentist methods is that they are computationally tractable. Usually they are best understood as approximations to Bayesian methods. For instance, MLE can be viewed as the variational approximation to Bayes where the family of probability distributions is the family of point masses, and the prior is uniform.
- Akababa 7y agoWhat do you mean by coordinate transformation? MLE is invariant under parameter transformations because it's just the argmax of the likelihood.
- jules 7y agoIndeed, it is the argmax of the likelihood, but the likelihood is not invariant under coordinate transformations. The quantity p(x)dx is invariant, not p(x). By picking a suitable coordinate transformation you can put the MLE on any value where the likelihood is not zero.
- deleted 7y ago[deleted]
- knzhou 7y agoAdding to the other comments, you still have prior-dependence on a more subtle level, because it depends on what hypotheses are allowed. Here's an extreme example. Consider flipping an apparently fair coin and getting "THHT". The hypothesis that the coin is fair gives this result with likelihood 1/16. The hypothesis that a worldwide government conspiracy has been formed with the sole purpose of ensuring this result... has a likelihood of 1. But nobody would ever declare this the MLE, because "government conspiracy" isn't one of the allowed options. But it isn't precisely because it's unlikely, i.e. because of your prior. Of course this is an extreme example, but there are more innocuous prior-based assumptions baked in too.
- eanzenberg 7y agoAgain: Priors can and are used to mislead. Both methods can and are used to mislead. Just moving to Bayes doesn't assume the finding is free of bias all of a sudden.
- c2471 7y agoIt doesn't. But the workflow of Bayes forces you be explicit. If you try and cook the books, it will be shown for the world to see. Can you provide a paper that quoted a p value for a regression and also validated all the asymptotic conditions are close to being true in order for that p value to be even somewhat reliable?
- closed 7y agoWait, in frequentist statistics getting, say, a p-value of 1 is not a bad thing--unless you erroneously assume that value is evidence for your null hypothesis. Consider that if your data generating process really is a fair coin, then the conspiracy outcome you mention only occurs 1 our of 16 times, so 15 out of 16 times you observe a likelihood of 0. 15 out of 16 times your reject the conspiracy case. There is also a tricky component here, because the notion of sample size is not clearly defined (can we generate multiple 4-tuples of flips, and consider each one a sample? Is your example really just a funky way of discussing type II power?)
- 7y ago
- kgwgk 7y agoMLE is not invariant under parameter transformations because it's just the argmax of the likelihood! Take for example x~normal and exp(x)~lognormal. The maximum of the distribution is at mu for the former and at exp(mu-sigma^2) for the latter, instead of exp(mu).
- eanzenberg 7y agoIf anything, Bayes increases complexity because of a variety of infinite priors that can be chosen. Frequentists is more straight forward because of the removal of this bias. A constant prior along with _actually reading the study/paper_ generally is sufficient. It doesn't preclude future testing. If anything, big discoveries in science require big scrutiny. Bayes does not add anything but complexity and another lever to tune in this regard.