3 ms·
I would be curious to see the positive predictive value (PPV) of this test (i.e. given that you have a positive result, what is the probability of having the di
by MontyCarloHall 5y ago
I would be curious to see the positive predictive value (PPV) of this test (i.e. given that you have a positive result, what is the probability of having the disease?), which is a very different quantity from sensitivity, which measures the probability of having a positive result, given that you have the disease. The answer to this question will depend on whom the test is administered to.
For example, given a test with 95% sensitivity and specificity for a disease that affects 1% of the population, if we administer the test uniformly to the population (i.e. without accounting for any prior probability of having the disease), the probability of actually having the disease given a positive test result is quite low, since the number of healthy people who get a positive test will be much greater than the number of sick people who get a positive test, owing to the fact that the vast majority of people are healthy.
The math for this is just Bayes’s rule:
p(disease|pos) = p(disease and pos)/p(pos) = p(disease and pos)/(p(disease and pos) + p(healthy and pos))
The highlighted term in the denominator is what lowers our PPV. We expect it to be much larger than p(disease and pos), since the fraction of healthy people in the population is much larger than the fraction of diseased population.
Expanding the joint probabilities, we have:
p(disease and pos)/(p(disease and pos) + p(healthy and pos)) = p(pos|disease)p(disease)/(p(pos|disease)p(disease) + p(pos|healthy)p(healthy))
Since our sensitivity and specificity are both 0.95, we have
p(pos|disease)=0.95
p(pos|healthy)=0.05
and since we administer the test uniformly, the prior probability of having the disease is the overall incidence level in the population, p(disease) = 0.01, so p(healthy) = 0.99
Plugging these numbers in, we get
0.95*0.01/(0.95*0.01 + 0.05*0.99) = 0.16
In other words, you only have a 16% chance of actually having the disease if you get a positive test.
The only way to improve on this number is to (a) construct a test with much higher specificity (hard!), or (b) only administer the test to people who are already more likely to have the disease than the baseline probability (i.e. pick a better prior, much easier!).
Given the figures in this article (99.5% specificity, aggregate 50% sensitivity) and an overall annual incidence of new cancer cases of ~450 per 100000 [0], we get a PPV of ~30% for GRAIL. This is probably fine for certain cancers, which can be followed up with minimally invasive additional screens, but too low for others, which would require invasive biopsies. I assume the actual PPV is much higher, since this test presumably will only be administered to a subset of people with a higher probability of having cancer, whose incidence rate is much greater than 450/100000. As I said before, I’d love to see what that number actually is.
[0] https://www.cancer.gov/about-cancer/understanding/statistics https://www.cancer.gov/about-cancer/understanding/statistics
- wgolsen 5y agoAn interim analysis of Grail’s Pathfinder study [1] suggests a PPV of 43% [2]. This may be a mixed population of their high and normal risk cohorts - it’s not clear to me from the abstract. They call this a conservative estimate, which seems quite fair, given that the “discordant” positive cases are really only potential false positives at this point. These patients may actually harbor tumors that become detectable further into the study or on follow up. Even with a mixed risk study population, a lower bound estimate of PPV in the 40s seems like a big win for Grail. 1 https://clinicaltrials.gov/ct2/show/NCT04241796 https://clinicaltrials.gov/ct2/show/NCT04241796 2 https://ascopubs.org/doi/abs/10.1200/JCO.2021.39.15_suppl.3070 https://ascopubs.org/doi/abs/10.1200/JCO.2021.39.15_suppl.30...