4 ms·
One reason people might not follow this method for inference is that it depends heavily on the experimenter's personal beliefs about the world. The "improvement
by ryanmonroe 11y ago
One reason people might not follow this method for inference is that it depends heavily on the experimenter's personal beliefs about the world. The "improvements" here only lead to more accurate inference if the author's assumptions about the world are true. In business applications you often just want to get to a conclusion and make a decision, so this method makes sense. In scientific publication you want to verify results with a larger community with minimal assumptions. When you calculate a p-value and print it in publication, you might not be giving much information, but at least that information is objective and invariant to readers' personal beliefs. Making inference based on p-values you can at least say "In the long run I will incorrectly reject the null hypothesis 5% of the time with this method", while there is no such similar statement for a method that depends on the experimenter's personal beliefs.
In addition, the probability being calculated here is a little misleading in that it doesn't fit with the traditional definition of "probability of X". Despite the same s notation, P(H|D) is not the same type of probability as P(D|H). The coin is either biased or not, so there isn't actually any random process there and the statement "the probability this coin is fair is 50%" makes little sense under the traditional definition.
- ivan_k 11y ago> while there is no such similar statement for a method that depends on the experimenter's personal beliefs The posterior probability P(H|D) is exactly this kind of statement. You say "Based on the data, I am 78% sure that the coin is biased". I think this is both more direct and interpretable.
- ryanmonroe 11y agoBy "similar statement" I meant a statement about how often the method will lead you to the incorrect conclusion. If you can't quantify that, it seems to me you don't have any basis for claiming your method "works".
- ivan_k 11y agoIf that is what you meant, than you are getting in the realm of hypothesis testing. The best equivalent of a p-value is then an anabashedly named "Bayes factor" [1], which is a ratio of posterior probabilities for competing hypotheses. https://en.wikipedia.org/wiki/Bayes_factor https://en.wikipedia.org/wiki/Bayes_factor
- ryanmonroe 11y agoThanks, didn't know about this.