4 ms·

To my (limited) understanding, Bayesian statistics are just as subjective. If you and I start with different priors, and are then are fed the same evidence, we

by smanek 17y ago

To my (limited) understanding, Bayesian statistics are just as subjective. If you and I start with different priors, and are then are fed the same evidence, we can end up with vastly different conclusions. Since priors are defined as a function of my state of mind (or state of ignorance, as it were), it seems pretty subjective that the 'result' should depend so heavily on my initial state of mind. E.T. Jaynes' book has a example of this with regard to E.S.P. research (I believe http://omega.albany.edu:8008/ETJ-PS/cc5d.ps http://omega.albany.edu:8008/ETJ-PS/cc5d.ps is the relevant chapter - but it's been a year or two since I've read the book). A real statistician should feel free to correct me though - I'm more of an algebra guy ...

mnemonicsloth 17y ago
To my (limited) understanding, Bayesian statistics are just as subjective. I'm still reading, but that seems to be the point. As Yudkowsky puts it, "probability is ignorance and ignorance is a state of mind"
yummyfajitas 17y ago
The argument is that Bayesian statistics are less subjective because while we won't come to the same conclusion, we will perform the same update to our prior probabilities. An example: consider a medical diagnostic test with a 1% false positive rate and no false negatives. If I understand him correctly, Eliezer is arguing that the scientist (i.e. the lab running the test) should present an update rule, not the posterior probability. I.e., the lab report should say: "Whatever your prior estimate of P(sick) is, your posterior should now be P(sick)/(0.01 P(not sick) + P(sick))" instead of saying "the patient has an x% chance of being sick". Suppose before the test I believe the odds the patient is sick are 1% and you think they are 10%. Once the test comes out positive, I've updated my probability estimate to 50%, you to 92%. The update rule provided by Bayesian statistics is objective, even if the posterior is subjective. The frequentists have nothing comparable which is objective. See also this link: http://www.overcomingbias.com/2009/02/share-likelihood-ratios-not-posterior-beliefs.html http://www.overcomingbias.com/2009/02/share-likelihood-ratio...
- smanek 17y ago
  Ah, that makes sense - thanks.
jules 17y ago
The difference is that bayesians acknowledge this explicitly. You publish the way to calculate the posterior given someones prior so that everyone can fill in their own priors to determine their posteriors. This calculation is objective. With frequentist statistics everything is subjective, including the results you publish.
btilly 17y ago
Your understanding is incomplete. Bayesian statistics are not subjective at all. Instead they objectively describe the correlation between prior belief and conclusion. People may draw different conclusions for subjective reasons, but that subjectivity is in the people, not the statistics. Now it seems you are objecting to the fact that your initial state of mind affects your conclusion. But it is unavoidable that there is no way to make sense of observation except through the lens of prior beliefs. You can be explicit about it and draw conclusions with a correct methodology about it as in Bayesian statistics. You can be implicit about it and draw conclusions with an incorrect methodology as in frequentist statistics. (See my other post in this thread for an explanation of why frequentist methodology is incorrect.) Let me offer a simple example. Suppose a pregnant woman you know gives birth to a boy. Is that evidence that babies are more likely to be boys than girls? Obviously it is. Is it strong evidence? Obviously not. Should we upon observing that conclude that boys are more likely than girls? Obviously not. Now suppose that you have no prior knowledge other than we see about similar numbers of boys and girls. I know that of the last 100,000 babies born in the USA, 51,157 are boys. Suppose we are both told that at the local hospital, 92 of the last 200 babies born were boys. I submit that we both will and should wind up with different conclusions about the relative likelyhood of boys and girls. Why? Because different prior knowledge lead to different prior beliefs, and those prior beliefs when modified by identical evidence lead to different conclusions.
- smanek 17y ago
  You have to admit that your example is pretty contrived though - usually setting priors isn't so clean. How do you 'properly' set the prior probability that someone really has E.S.P.? That a researcher is secretly colluding with a test subject? That a researcher is flat out fabricating their results? That this is just an instance of the Hans effect? Or a thousand other possible hypotheses ...
  - btilly 17y ago
    I picked my example so that the proper influence of the prior would be clear. You are absolutely right that there are many cases where different people with equivalent information have different beliefs. But being faced with complications like this makes correct reasoning more important, and not less. When you try to sweep the proper influence of the prior under the carpet, people will manipulate their statistics to draw the conclusions that they want. Worse still, if each is using subjective techniques that they believe are objective, the unexamined assumption will lead to them talking past each other. By contrast with correct reasoning you can show each why they continue to believe what they believe while showing them how the other side is not going to have their opinions changed. In my experience this opens up a bigger possibility of useful dialog and changed opinions.

by smanek 17y ago

mnemonicsloth 17y ago

To my (limited) understanding, Bayesian statistics are just as subjective. I'm still reading, but that seems to be the point. As Yudkowsky puts it, "probability is ignorance and ignorance is a state of mind"

yummyfajitas 17y ago

The argument is that Bayesian statistics are less subjective because while we won't come to the same conclusion, we will perform the same update to our prior probabilities. An example: consider a medical diagnostic test with a 1% false positive rate and no false negatives. If I understand him correctly, Eliezer is arguing that the scientist (i.e. the lab running the test) should present an update rule, not the posterior probability. I.e., the lab report should say: "Whatever your prior estimate of P(sick) is, your posterior should now be P(sick)/(0.01 P(not sick) + P(sick))" instead of saying "the patient has an x% chance of being sick". Suppose before the test I believe the odds the patient is sick are 1% and you think they are 10%. Once the test comes out positive, I've updated my probability estimate to 50%, you to 92%. The update rule provided by Bayesian statistics is objective, even if the posterior is subjective. The frequentists have nothing comparable which is objective. See also this link: http://www.overcomingbias.com/2009/02/share-likelihood-ratios-not-posterior-beliefs.html http://www.overcomingbias.com/2009/02/share-likelihood-ratio...

smanek 17y ago
Ah, that makes sense - thanks.

jules 17y ago

The difference is that bayesians acknowledge this explicitly. You publish the way to calculate the posterior given someones prior so that everyone can fill in their own priors to determine their posteriors. This calculation is objective. With frequentist statistics everything is subjective, including the results you publish.

btilly 17y ago

Your understanding is incomplete. Bayesian statistics are not subjective at all. Instead they objectively describe the correlation between prior belief and conclusion. People may draw different conclusions for subjective reasons, but that subjectivity is in the people, not the statistics. Now it seems you are objecting to the fact that your initial state of mind affects your conclusion. But it is unavoidable that there is no way to make sense of observation except through the lens of prior beliefs. You can be explicit about it and draw conclusions with a correct methodology about it as in Bayesian statistics. You can be implicit about it and draw conclusions with an incorrect methodology as in frequentist statistics. (See my other post in this thread for an explanation of why frequentist methodology is incorrect.) Let me offer a simple example. Suppose a pregnant woman you know gives birth to a boy. Is that evidence that babies are more likely to be boys than girls? Obviously it is. Is it strong evidence? Obviously not. Should we upon observing that conclude that boys are more likely than girls? Obviously not. Now suppose that you have no prior knowledge other than we see about similar numbers of boys and girls. I know that of the last 100,000 babies born in the USA, 51,157 are boys. Suppose we are both told that at the local hospital, 92 of the last 200 babies born were boys. I submit that we both will and should wind up with different conclusions about the relative likelyhood of boys and girls. Why? Because different prior knowledge lead to different prior beliefs, and those prior beliefs when modified by identical evidence lead to different conclusions.

smanek 17y ago
You have to admit that your example is pretty contrived though - usually setting priors isn't so clean. How do you 'properly' set the prior probability that someone really has E.S.P.? That a researcher is secretly colluding with a test subject? That a researcher is flat out fabricating their results? That this is just an instance of the Hans effect? Or a thousand other possible hypotheses ...
- btilly 17y ago
  I picked my example so that the proper influence of the prior would be clear. You are absolutely right that there are many cases where different people with equivalent information have different beliefs. But being faced with complications like this makes correct reasoning more important, and not less. When you try to sweep the proper influence of the prior under the carpet, people will manipulate their statistics to draw the conclusions that they want. Worse still, if each is using subjective techniques that they believe are objective, the unexamined assumption will lead to them talking past each other. By contrast with correct reasoning you can show each why they continue to believe what they believe while showing them how the other side is not going to have their opinions changed. In my experience this opens up a bigger possibility of useful dialog and changed opinions.