3 ms·

Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement. S

by nkurz 10y ago

Take the proposition that the Earth goes round the Sun. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement. Sometimes I fail to follow the distinctions made in certain strains of classical statistics. How is his "conundrum" different than this one? "Roll a 6 sided die whose result we cannot see. Take the proposition that the top shows a 6. It either does or it doesn’t, so it’s hard to see how we could pick a probability for this statement." What matters to a scientific observer is how often you’ll be wrong if you claim that an effect is real, rather than being merely random. I think "scientific observer" may mean statistician here. For the scientist, what should matter is the probability that the claimed effect is real -- period. That is, unlike the statistician, the scientist isn't (shouldn't be) allowed to blame "modeling error" when it turns out that the measurements are biased, the samples are correlated, or the effects are nonlinear. False assumptions that "randomness" is the only (or main) danger can lead to unrealistic error bars and unwarranted confidence in the effectiveness of flawed models.

bbctol 10y ago
Well, it's less of a conundrum in the case of the die because it's easier to estimate the priors. That's the power and difficulty of using Bayesian reasoning; you need some estimate of how likely something is to be true before you perform the statistical test. In case of the six-sided die, we have a good physical model and years of memory to know that there should be a 1/6 chance of any number coming up, so it's easy to estimate that prior. Similarly, when doing a disease screening test, we have data on how common diseases are in the general population that can be easily used. The problem comes when using Bayesian methods on unsolved scientific problems. Suppose you don't know if the Earth goes around the Sun or the other way around, it's the early Renaissance and you've gathered some data that could indicate a probability of one or the other. What's the prior odds that one theory is correct? You have no idea, that's why you're investigating! The worry when using Bayes's theorem to replace deductive p-value methods is that the prior probabilities may just be made up out of baseless intuition, and skew the final calculation. (It can still be used effectively, you just have to get a little fancy.)
- ramidarigaz 10y ago
  I think there is a good option for choosing your priors in the Earth/Sun problem you mentioned. Just put them at 50/50. That's basically stating that you have no preference for one theory over the other, and it allows for any evidence you gather to be maximally effective in computing the posterior.
  - ikeboy 10y ago
    That quickly leads to inconsistencies. What's the probability of the earth going around some object bigger than it? What's the probability of the earth going around something bigger than it that isn't the sun? The answer to all 3 cannot all be 50/50.
    - taejo 10y ago
      That's true, but if we assume the probabilities of "Earth orbits Sun" and "Sun orbits Earth" we still get the correct conditional probabilities under the condition "either the Sun orbits the Earth or the Earth orbits the Sun", and the ratio between the two conditional probabilities is the same as the ratio between the absolute probabilities. So we can still decide which hypothesis is better, and by how much.
- nkurz 10y ago
  Well, it's less of a conundrum in the case of the die because it's easier to estimate the priors. I agree, but in the case of a die, "estimating the priors" is the same thing as "knowing the probability of a 6". So you are saying (correctly) that once we have a model that we believe in, we can assign a probability because we believe we know the probability in advance. I was pointing more to the author's emphasis on "either it does or it doesn't". One view is that probability requires replication, and we can only speak of probabilities in the long run of many trials. The other view (Bayesian to my limited understanding) is that probability can also be used to measure appropriate degree of belief in a proposition. While it's hard to quantify, and dependent on one's assumptions, I do think it's possible to speak of "the probability that the earth revolves around the sun", much as I think it's meaningful to talk about "the probability of anthropogenic warming" even though we only have a single Earth to study. I don't really understand how classical statistics disallows discussion of the first while allowing discussion of the second. Maybe it doesn't?

by nkurz 10y ago

bbctol 10y ago

Well, it's less of a conundrum in the case of the die because it's easier to estimate the priors. That's the power and difficulty of using Bayesian reasoning; you need some estimate of how likely something is to be true before you perform the statistical test. In case of the six-sided die, we have a good physical model and years of memory to know that there should be a 1/6 chance of any number coming up, so it's easy to estimate that prior. Similarly, when doing a disease screening test, we have data on how common diseases are in the general population that can be easily used. The problem comes when using Bayesian methods on unsolved scientific problems. Suppose you don't know if the Earth goes around the Sun or the other way around, it's the early Renaissance and you've gathered some data that could indicate a probability of one or the other. What's the prior odds that one theory is correct? You have no idea, that's why you're investigating! The worry when using Bayes's theorem to replace deductive p-value methods is that the prior probabilities may just be made up out of baseless intuition, and skew the final calculation. (It can still be used effectively, you just have to get a little fancy.)

ramidarigaz 10y ago
I think there is a good option for choosing your priors in the Earth/Sun problem you mentioned. Just put them at 50/50. That's basically stating that you have no preference for one theory over the other, and it allows for any evidence you gather to be maximally effective in computing the posterior.
- ikeboy 10y ago
  That quickly leads to inconsistencies. What's the probability of the earth going around some object bigger than it? What's the probability of the earth going around something bigger than it that isn't the sun? The answer to all 3 cannot all be 50/50.
  - taejo 10y ago
    That's true, but if we assume the probabilities of "Earth orbits Sun" and "Sun orbits Earth" we still get the correct conditional probabilities under the condition "either the Sun orbits the Earth or the Earth orbits the Sun", and the ratio between the two conditional probabilities is the same as the ratio between the absolute probabilities. So we can still decide which hypothesis is better, and by how much.
nkurz 10y ago
Well, it's less of a conundrum in the case of the die because it's easier to estimate the priors. I agree, but in the case of a die, "estimating the priors" is the same thing as "knowing the probability of a 6". So you are saying (correctly) that once we have a model that we believe in, we can assign a probability because we believe we know the probability in advance. I was pointing more to the author's emphasis on "either it does or it doesn't". One view is that probability requires replication, and we can only speak of probabilities in the long run of many trials. The other view (Bayesian to my limited understanding) is that probability can also be used to measure appropriate degree of belief in a proposition. While it's hard to quantify, and dependent on one's assumptions, I do think it's possible to speak of "the probability that the earth revolves around the sun", much as I think it's meaningful to talk about "the probability of anthropogenic warming" even though we only have a single Earth to study. I don't really understand how classical statistics disallows discussion of the first while allowing discussion of the second. Maybe it doesn't?