3 ms·

A good Bayesian should be able to make confident decisions based on information available at the moment... No. A good Bayesian should be able to come to decis

by btilly 7y ago

A good Bayesian should be able to make confident decisions based on information available at the moment... No. A good Bayesian should be able to come to decisions like, "I am 70% confident that Osama bin Laden is in that compound." While the Bayesian next says, "I am only 50% confident that Osama bin Laden is in that compound." With both knowing that there is a difference of opinion, but no disagreement on basic facts or reasoning method. It is very rare for a good Bayesian to be absolutely confident of any prediction. And if you are often so confident, you're probably not thinking very well. I mean that quite literally - the process of analyzing probabilities well requires being able to make the case both for what you think will happen, and what you think won't. Because only then can you start putting probabilities on the key assumptions. For example, a leader can be absolutely confident that shelter-in-place is the best decision based on the available information, while acknowledging that there is missing information that would drastically change this assessment. Really? https://news.ycombinator.com/item?id=22750790 https://news.ycombinator.com/item?id=22750790 is a discussion that I was in recently about whether on a cost benefit analysis it is better to crash the economy by shutting things down, or to keep things open and let lots of people die. The decision wasn't nearly as clear in the end as I would have expected it to be. (That all options are horrible was clear. But we knew that.)

bo1024 7y ago
> No. A good Bayesian should be able to come to decisions like, "I am 70% confident that Osama bin Laden is in that compound." That's not a decision though. That's an assessment, what I would put in the same category as predictions. A decision would be whether or not to bomb the compound. Your post seems to miss my point, that predictions and decisions are very different and one can be uncertain about predictions while being certain about decisions. For example, I completely agree with this: > It is very rare for a good Bayesian to be absolutely confident of any prediction. Here's a simple example to think through the difference. You have a sophisticated weather model that predicts 40% chance of rain today. You hate getting wet, so you take your umbrella. In fact, you would take your umbrella even if the chance were only 10%. So you are really uncertain about whether it's going to rain (your forecast), but absolutely certain that taking your umbrella is the optimal decision given the information at hand.
- btilly 7y ago
  You are right that I had not paid close attention to decisions. I see no particular reason why Bayesians should be better at being decisive. They should make better judgments given the available information. But they are not necessarily any better at making decisions and moving on.
  - btilly 7y ago
    Expanding on this, the skill of figuring out the odds of bin Laden being in the compound is unrelated to the skill of figuring out how to handle both outcomes, and whether that is a worthwhile risk to take. So a good Bayesian can inform a good decision maker, but the Bayesian is not necessarily a good decision maker. Similarly in the book, one of the superforecasters made the point that listening to well-informed experts who might be bad at making decisions was very useful. Because the expert really did have a good grasp of the current situation and could explain it clearly, which was a great starting place for the Bayesian who lacked background. Preparing background and making predictions are both required, but the combination of skills need not start in the same brain.

by btilly 7y ago

bo1024 7y ago

> No. A good Bayesian should be able to come to decisions like, "I am 70% confident that Osama bin Laden is in that compound." That's not a decision though. That's an assessment, what I would put in the same category as predictions. A decision would be whether or not to bomb the compound. Your post seems to miss my point, that predictions and decisions are very different and one can be uncertain about predictions while being certain about decisions. For example, I completely agree with this: > It is very rare for a good Bayesian to be absolutely confident of any prediction. Here's a simple example to think through the difference. You have a sophisticated weather model that predicts 40% chance of rain today. You hate getting wet, so you take your umbrella. In fact, you would take your umbrella even if the chance were only 10%. So you are really uncertain about whether it's going to rain (your forecast), but absolutely certain that taking your umbrella is the optimal decision given the information at hand.

btilly 7y ago
You are right that I had not paid close attention to decisions. I see no particular reason why Bayesians should be better at being decisive. They should make better judgments given the available information. But they are not necessarily any better at making decisions and moving on.
- btilly 7y ago
  Expanding on this, the skill of figuring out the odds of bin Laden being in the compound is unrelated to the skill of figuring out how to handle both outcomes, and whether that is a worthwhile risk to take. So a good Bayesian can inform a good decision maker, but the Bayesian is not necessarily a good decision maker. Similarly in the book, one of the superforecasters made the point that listening to well-informed experts who might be bad at making decisions was very useful. Because the expert really did have a good grasp of the current situation and could explain it clearly, which was a great starting place for the Bayesian who lacked background. Preparing background and making predictions are both required, but the combination of skills need not start in the same brain.