2 ms·
>As opposed to the frequentist approach of saying, "We have data D and hypothesis H. If the probability of D when not-H is true is below a threshold, H is true,
by BeetleB 7y ago
>As opposed to the frequentist approach of saying, "We have data D and hypothesis H. If the probability of D when not-H is true is below a threshold, H is true," which hides an awful lot of assumptions.
My textbook (what would be called a frequentist book) is very explicit: You either reject the null hypothesis, or you fail to reject the null hypothesis. A significance level test never concludes that the null hypothesis is true. It's nuanced, but is not quite the same as your statement.
- pdonis 7y agoYes, and under this description, the equivalent of the Bayesian prior is picking what the null hypothesis is. Frequentists simply fail to admit (or perhaps fail to understand) that that choice is just as much of a subjective judgment as the Bayesian choice of prior.
- BeetleB 7y ago>Frequentists simply fail to admit (or perhaps fail to understand) that that choice is just as much of a subjective judgment as the Bayesian choice of prior. Again, this claim is something I see only from self described Bayesians. I've not met a professional statistician ever fail to admit it. My textbook (written in the 90's), in at least two places, talks about it. In one place it explicitly warns against assuming the "traditional" approach is more objective, and points out that the assumptions exist in the model and the practitioner should be aware of those assumptions. And really, when it comes to p-values, I've never seen any stats textbook not give a proper description and interpretation. No statistics textbook I've read describes significance testing as a mere prescription from which you can deduce binary answers. My book discusses factors in picking your p values, and the implications, and that the appropriate value is very dependent on the problem.
- pdonis 7y agoI didn't say "picking the p-values". I said "picking the null hypothesis". (Although p-value hacking is certainly a major contributing factor to the replication crisis.) It's good that the textbooks you were taught from point out that assumptions exist and should be acknowledged.
- BeetleB 7y ago>It's good that the textbooks you were taught from point out that assumptions exist and should be acknowledged. Have you found a "standard" text book that doesn't? Not a probability textbook, but a statistics one.
- harry8 7y ago"Again, this claim is something I see only from self described Bayesians." The claim is that the prior is subjective and they're bad. The Bayesians are drawing the analogy that frequenteist assumptions are just as subjective (but maybe less explicit) to the prior selection so objecting to one and not the other is... ...in need of further justification.
- BeetleB 7y ago>The claim is that the prior is subjective and they're bad. The only such claims I've seen are ones where they disagree with the prior, which is no different that disputes amongst themselves where so-called frequentists argue whether p=0.05 or p=0.001 is appropriate. Frequentists don't pretend that their p-value is set in stone. Let me repeat what I said in another comment: I'm finding this whole discussion orthogonal to Bayesian vs frequentist statistics. The main issue frequentists have with a lot of Bayesian approaches is that Bayesians often want to assign probabilities to one off events, whereas frequentists insist only on repeatable events. That is where the accusation of "subjective" comes from. Frequentists like to believe that any question of probability can be decided by taking N samples and seeing the outcome (even if only conceptually). For problems where there is a population, and one can do sampling (i.e. repeatability), there's never a problem with using Bayesian methods. My complaint is that while there are differences amongst Bayesians and frequentists, this article does not present any, and wrongly implies things about frequentists.
- nkurz 7y agoI think it would be useful if you could apply these principles to one of the examples in the article. Assume we have a woman who has a mammogram that indicates malignancy. The question that the patient would like answered is "What is the chance that I have breast cancer?" How does frequentist statistics answer this question? What's the null hypothesis that should be used here? Can this question be answered without assuming a prior on the likelihood that the individual had breast cancer before the test was performed?
- BeetleB 7y ago>How does frequentist statistics answer this question? By applying Bayes Theorem, just as the article did. By invoking Type I and Type II errors. This is a standard problem in stats textbooks - not breast cancer but the general problem of a diagnostic measure that is 99% accurate, for a disease that has less than 1% prevalence. Frequentists don't insist on using p-values to solve all problems. >Can this question be answered without assuming a prior on the likelihood that the individual had breast cancer before the test was performed? Why should one not assume a prior? There is a base rate for breast cancer. Why should a frequentist not use that information? I've never heard a statistician who does not describe himself as a Bayesian say one shouldn't. Every textbook suggests one should. Where are people getting the idea that frequentists don't use Bayesian methods. They always have used them. I'm finding this whole discussion orthogonal to Bayesian vs frequentist statistics. The main issue frequentists have with a lot of Bayesian approaches is that Bayesians often want to assign probabilities to one off events, whereas frequentists insist only on repeatable events. That is where the accusation of "subjective" comes from. Frequentists like to believe that any question of probability can be decided by taking N samples and seeing the outcome (even if only conceptually). For problems where there is a population, and one can do sampling (i.e. repeatability), there's never a problem with using Bayesian methods.
- FabHK 7y agoGood point. It would be illustrative to get some examples where a prototypical (but competent) Bayesian and a prototypical (but competent) frequentist would give different answers. It seems to me that when there really is a known base rate, the answers would coincide. And if there is no base rate, the Bayesian would guess (but explicitly) or refuse to answer, while the frequentist would give a plausible answer, which, however, very much hinges on the unknown base rate. In the light of the replication crisis, maybe the Bayesian approach is better.
- mcguire 7y ago"In one place it explicitly warns against assuming the "traditional" approach is more objective, and points out that the assumptions exist in the model and the practitioner should be aware of those assumptions." The Bayesian approach forces you to explicitly model those assumptions. Or to admit that you are ignoring them.