2 ms·
I think this anonymous comment on the blog page is very interesting indeed: Incidentally, the usual hypothesis testing paradigm is inherently un-Bayesian even
by ScottBurson 12y ago
I think this anonymous comment on the blog page is very interesting indeed:
Incidentally, the usual hypothesis testing paradigm is inherently un-Bayesian even if posteriors are used to judge the hypothesis. Given for example hypothesis “H0: theta less than 0″ and “H1: theta great than 0″, then the full posterior P(theta|data, background) encapsulates everything the data + background has to say about theta.
If you gratuitously add another step which determines say H1 is true, and assume it’s true going forward then you’ve effectively truncated P(theta |data, background) to theta greater than zero without having any further data or other evidence for doing so. It’s an inherent violation of the sum/product rules in other words and hence un-Bayesian. In some instances this truncation will be a valid approximation to the full Bayesian version, but most of the time it wont.
The Bayesian version of hypothesis testing (Decision Theory with loss functions and all the rest) really only makes sense if you’re making final decisions. For example, if you’re programming a computer to process data and make automatic decisions about things. Otherwise the Bayesian thing to do is carry the full posterior P(theta | data, background) forward un-altered. Scientists too need to make final conclusions sometimes, but most of the time hypothesis testing is used to make piecemeal judgments along the way (such as removing a parameter from the analysis) in which you’re effectively truncating distributions without the evidence needed to do so.
So what we should be doing, if I understand this correctly, is not saying "this hypothesis is supported by the data (p < .05)" but "given such-and-such a prior, and the data, we conclude that the hypothesis is 62% likely to be true" or some such.
- tel 12y agoEven "62% true" is too weak: it summarizes the posterior by a certain integral. This drives home why we don't do either, of course: it's far more expensive.