2 ms·
Every time mistakes in cancer tests come up, I have to bring up this textbook counter-intuitive probabilistic result that happens when one tries to detect an a
by blackbear_ 3y ago
Every time mistakes in cancer tests come up, I have to bring up this textbook counter-intuitive probabilistic result that happens when one tries to detect an a priori very rare condition.
It follows from a straightforward application of Bayes formula:
- Suppose a person has 1% chance of getting cancer, so p(cancer) = 0.01
- Suppose that the test has 99% sensitivity and specificity, i.e., p(positive | cancer) = p(negative | no cancer) = 0.99 (no test is perfect in the real world, in this case the test is wrong only one time every 100).
- To use Bayes we first need the probability of a positive result regardless of disease status, i.e. p(positive) = p(positive | cancer) * p(cancer) + p(positive | no cancer) * p(no cancer) = 0.99 * 0.01 + 0.01 * 0.99 = 0.0198
- Then by Bayes we have p(cancer | positive) = p(positive | cancer) * p(cancer) / p(positive) = 0.99 * 0.01 / 0.0198 = 0.5
And things get worse the rarer the condition is. For example, when p(cancer) = 0.001 then p(cancer | positive) = 0.09, while for p(cancer) = 0.5 the computations above give p(cancer | positive) = 0.99.
In other words, the rarer the condition, the more precise tests have to be to have confidence in its results.
- red0point 3y agoThe article is about a simple misconfiguration on their email campaign, nothing to do with their actual tests. It‘s a misleading title for sure.
- blackbear_ 3y agoYes, I know. The point that I was trying to make is that this is bound to happen again, even if everything is set-up correctly.
- Dylan16807 3y agoIf it's "set up correctly", then they'll only do a test like that if they have a reasonable follow-up test. In which case they will probably send a good and valid email and "this" is not what will happen.
- veltas 3y agoIn a sense this comment is still relevant, because from the grand perspective this email error could be considered a random failure on the part of the test in its full practical context. And it's not impossible it will happen again (in fact I expect it to). But it's certainly not characteristic of the test itself in theory, but is nevertheless characteristic of any such test that might involve bad IT. It's almost like test false-positive background radiation.
- deleted 3y ago[deleted]
- CatWChainsaw 3y agoI first skimmed over "a priori" and thought you went to talking about prions; I don't think I could think about the statistics of those with a cool head much less cancer, however rare..