4 ms·
It's not a sensitivity / specificity thing; it's a base rate thing [1]. Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000
by gwd 4y ago
It's not a sensitivity / specificity thing; it's a base rate thing [1].
Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000 (i.e., then "base rate" is 1 in 1000). (Turns out this is actually on the same order of magnitude as real number [2].). And suppose this test has 99% sensitivity and 99% specificity.
And suppose you test 1,000,000 people.
Of those 1,000,000 people, 1,000 will actually have prostate cancer, and 999,000 will not.
Of those 1000 that actually have it, 990 will have a positive test (true positive), and 10 will have a negative test (false negative).
Of the 999,000 people who don't actually have it, 989,010 will have a negative test (true negative), and 9,990 will have a positive test (false positive).
So even with a test of 99% accuracy, if you get a positive result, your chances of actually having prostate cancer are still only 990 / 10980, or about 9%; 91% of the positives will be false positives.
And of course, the more rare the cancer, the worse it gets.
EDIT2: So, to follow on with GGP's point: "Near perfect accuracy" isn't very specific, but colloquially would imply that if you have a positive test, you have a high chance of actually having cancer. To get that number to 95% you'd need to have only 52 false positives, would require a specificity closer to 99.995%.
EDIT: Fixed some math
[1] https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy
[2] https://www.cancer.org/cancer/prostate-cancer/about/key-statistics.html https://www.cancer.org/cancer/prostate-cancer/about/key-stat...
- nerdponx 4y agoRight, the "lots of false positives" conclusion is still the correct analysis. I just wanted to point out that both "P(Detect | Positive)" and "P(Not Detect | Negative)" are both high, since GP only mentioned one and not the other.