4 ms·

This statement is a misunderstanding of the Bayesian approach: "Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s h

This statement is a misunderstanding of the Bayesian approach: "Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement." Bayes Factor, the Bayesian alternative to a NHST, is quite a bit different than simply creating the Bayesian equivalent of a t-test. Bayes factor asks "How many times better is my Hypothesis at explaining the data than an alternate Hypothesis". So the Bayesian statement would first pit one model of the Earth's orbit against another. The Bayesian statement of the question of the Earth's orbit would be: "How much more likely is the astronomical data we've observed to have happened given that the Earth revolves around the sun than it is if the sun revolved around the Earth." For a more concrete example let's suppose that we have a coin. I think the coin has only heads and you think it is a fair coin, with a 50/50 chance of getting heads or tails. We observe three heads in a row. My hypothesis says that the probability of getting 3 heads in a row given a trick coin is 1. Your hypothesis says that the probability of getting 3 heads in a row given a fair coin is 0.5 x 0.5 x 0.5 = 0.125. My hypothesis explains the data 1/0.125 = 8 times better than your hypothesis. Now suppose the next flip is a tail. The probability of HHHT in my model is 0 and yours is 0.5 x 0.5 x 0.5 x 0.5 = 0.0625. You're hypothesis explains the model infinitely better than mine! Now we can say that our new Hypothesis is that the coin is fair. Suppose another friend comes along and claims that they thought the coin had a 75% chance of getting heads and only a 25% chance of tails. We flip the coin 5 more times and get HHTTH. Your hypothesis says 0.5^5 = 0.03125, and the friend's says 0.75^3 x 0.25^2 = 0.0263... You're hypothesis explains the data only 1.2 times better than theirs. Clearly, we need more data to feel really confident in one hypothesis over the other. If you want an even longer example, I wrote a post awhile back about "Bayesian Reasoning in the Twilight Zone" that goes into more detail (including priors)[0] [0] https://www.countbayesie.com/blog/2016/3/16/bayesian-reasoning-in-the-twilight-zone https://www.countbayesie.com/blog/2016/3/16/bayesian-reasoni...

fuckingmoron 10y ago
Will you stop it. Bayes factors are computed as a ratio of posterior model likelihoods, a likelihood ratio, which is not so different from a t-test (which as you'll recall from stats 101 is the likelihood ratio test for two Gaussians with finite samples). The difference, then, is in using the raw ratio rather than calculating its probability, and (crucially) whether you use a uniform or something more informative to generate the prior distribution that is updated with the data to yield the posterior. Frequentist == flat, weak prior (usually a dumb idea). There is no reason you couldn't use a Beta for a prior for a p-value distribution (flat would then be Beta(1,1) aka uniform) and generate a posterior p-value probability distribution based on that (take the integral of the PDF from the posterior mode on up to 1 => p-value). Not unlike a t-test! Pierre-Simone Laplace himself (who Bayes' theorem should rightly have been named after) used the "sun rising tomorrow" example to contrast naive with subjective treatments of probability as belief. (If you're a strict frequentist, a uniform prior is called for; but everyone will laugh at you because obviously the prior is a lot more pointy than that.) http://lesswrong.com/lw/774/a_history_of_bayes_theorem/ http://lesswrong.com/lw/774/a_history_of_bayes_theorem/ is a nice treatment. All this NHST bullshit came later. The important point here is how you derive the posterior likelihood, and the "sun rising tomorrow" example (whence Colquhoun derived his) neatly makes that point. So while you could generate a "Bayesian p-value" with exactly the same tools, everyone would laugh at you for throwing away all that valuable information in the posterior (how pointy is it? how much more conclusive than the prior?) and THAT (I claim) is the real difference here.
SubiculumCode 10y ago
So in the real world, are there developed techniques to build relatively complex models that account for covariates and sources of variability (e.g. random effects) and repeated measures?

fuckingmoron 10y ago

Will you stop it. Bayes factors are computed as a ratio of posterior model likelihoods, a likelihood ratio, which is not so different from a t-test (which as you'll recall from stats 101 is the likelihood ratio test for two Gaussians with finite samples). The difference, then, is in using the raw ratio rather than calculating its probability, and (crucially) whether you use a uniform or something more informative to generate the prior distribution that is updated with the data to yield the posterior. Frequentist == flat, weak prior (usually a dumb idea). There is no reason you couldn't use a Beta for a prior for a p-value distribution (flat would then be Beta(1,1) aka uniform) and generate a posterior p-value probability distribution based on that (take the integral of the PDF from the posterior mode on up to 1 => p-value). Not unlike a t-test! Pierre-Simone Laplace himself (who Bayes' theorem should rightly have been named after) used the "sun rising tomorrow" example to contrast naive with subjective treatments of probability as belief. (If you're a strict frequentist, a uniform prior is called for; but everyone will laugh at you because obviously the prior is a lot more pointy than that.) http://lesswrong.com/lw/774/a_history_of_bayes_theorem/ http://lesswrong.com/lw/774/a_history_of_bayes_theorem/ is a nice treatment. All this NHST bullshit came later. The important point here is how you derive the posterior likelihood, and the "sun rising tomorrow" example (whence Colquhoun derived his) neatly makes that point. So while you could generate a "Bayesian p-value" with exactly the same tools, everyone would laugh at you for throwing away all that valuable information in the posterior (how pointy is it? how much more conclusive than the prior?) and THAT (I claim) is the real difference here.

SubiculumCode 10y ago

So in the real world, are there developed techniques to build relatively complex models that account for covariates and sources of variability (e.g. random effects) and repeated measures?