3 ms·

Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is th

Doctors observe a result of the test, and know the basic probabilities (in the example, 99% test accuracy, 1% of population have the disease). The problem is that they [often] draw incorrect conclusions from those observations (99% test accuracy and you tested positive? well then you likely - 99% - have the disease, right?). The question formed as 'your one patient tested positively' is more immediately relevant, I'd think. The correspondence with actual practice is obvious. The question formed as 'out of 10000 ...' could be remembered as a quirk of statistics, but not actually recalled when someone tests positively for cancer.

Jabbles 11y ago
Of course, doctors do not randomly assign tests to patients. Their prior that a patient has a disease is a lot higher than the background frequency of it occurring. Getting them to estimate their prior would be interesting.
- hyperpape 11y ago
  Note that the frequency with which people who don't have the condition take the test is a major (but not the only) component of a good prior.
- TeMPOraL 11y ago
  It depends. Some tests are applied without prior suspicion, so you deal with exactly the background frequency. With others, the disease in question is so rare that false positives will dominate even if the doctor has serious suspicions. The second case is the reason for the "think horses not zebras" aphorism. Doing a few explicit Bayesian calculations can help one internalize just how much important is that often forgotten P(A) factor is. The same applies, by the way, to the antiterrorism security theater - many support it just because they have no intuition (or idea) about base rates.
- URSpider94 11y ago
  Except when they do. For example, when 100% of men above a certain age are screened for prostate cancer, or 100% of women above a certain age are screened for breast cancer. Both cases spawned major public health campaigns to encourage screening, followed years later by recommendations AGAINST 100% screening, based on the high degree of false positives and unnecessary treatment. Other cases that come to mind: -- doctors who offer "full body scans" as a part of an executive physical; you're pretty much guaranteed to turn up something that is 2 sigma away from the population norm, somewhere in the body, on such a scan -- spinal x-rays for back pain. Doctors almost always find something abnormal, and use that to justify the back pain and treat aggressively. But, we don't really have a good prior; if you x-rayed 1000 people off the street, would we find similar abnormalities frequently?
hyperpape 11y ago
I certainly agree that thinking in terms of percentages is more familiar. But it's familiarity that doesn't help most people get the right answer.

Jabbles 11y ago

Of course, doctors do not randomly assign tests to patients. Their prior that a patient has a disease is a lot higher than the background frequency of it occurring. Getting them to estimate their prior would be interesting.

hyperpape 11y ago
Note that the frequency with which people who don't have the condition take the test is a major (but not the only) component of a good prior.
TeMPOraL 11y ago
It depends. Some tests are applied without prior suspicion, so you deal with exactly the background frequency. With others, the disease in question is so rare that false positives will dominate even if the doctor has serious suspicions. The second case is the reason for the "think horses not zebras" aphorism. Doing a few explicit Bayesian calculations can help one internalize just how much important is that often forgotten P(A) factor is. The same applies, by the way, to the antiterrorism security theater - many support it just because they have no intuition (or idea) about base rates.
URSpider94 11y ago
Except when they do. For example, when 100% of men above a certain age are screened for prostate cancer, or 100% of women above a certain age are screened for breast cancer. Both cases spawned major public health campaigns to encourage screening, followed years later by recommendations AGAINST 100% screening, based on the high degree of false positives and unnecessary treatment. Other cases that come to mind: -- doctors who offer "full body scans" as a part of an executive physical; you're pretty much guaranteed to turn up something that is 2 sigma away from the population norm, somewhere in the body, on such a scan -- spinal x-rays for back pain. Doctors almost always find something abnormal, and use that to justify the back pain and treat aggressively. But, we don't really have a good prior; if you x-rayed 1000 people off the street, would we find similar abnormalities frequently?

hyperpape 11y ago

I certainly agree that thinking in terms of percentages is more familiar. But it's familiarity that doesn't help most people get the right answer.