3 ms·

it says "In a clinical study, around 4 out of 10 people with this result were diagnosed with cancer after diagnostic testing." so is my arithmetic correct, in

it says "In a clinical study, around 4 out of 10 people with this result were diagnosed with cancer after diagnostic testing." so is my arithmetic correct, in that it's a 60% false positive rate?

dekhn 3y ago
That corresponds to the publication, https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(23)01700-2/fulltext?rss=yes https://www.thelancet.com/journals/lancet/article/PIIS0140-6... "A cancer signal was detected in 92 (1·4%) of 6621 participants with analysable results. 35 (38%) participants were diagnosed with cancer (true positives) and 57 (62%) had no cancer diagnosis (false positives)." I am surprised that they were approved based on their study results, it will be interesting to see the results from PATHFINDER 2, which is running now.
- nradov 3y ago
  If there a legal reason that the test shouldn't be approved? Although the false positive rate is high, it's ultimately up to the patient and their doctor about whether to order the test and what to do with the results. I do understand the concerns over false positives unnecessarily scaring patients and leading to wasteful additional diagnostic tests, but I don't think the FDA really accounts for that in the approval process. Detecting early stage cancer in a liquid biopsy is extremely difficult so it's impressive that they made it work at all.
  - dekhn 3y ago
    It's policy not law (the law give the FDA authority to set policy and makes decisions). And they definitely would consider the sensitivity and specificity and PPV of a test. TBH it's amazing it got approved. But the product leadership and founding team definitely knew how to play the FDA game. Yes, early cancer tests are fantastic. However, tests need to have extremely good sensitivity and specificity for exactly the reason you said.
tzs 3y ago
Assuming that you calculated that by subtracting 4 from 10, then divided the results by 10, and expressed that as a percentage, then yes, your arithmetic is correct. It's just not the right arithmetic for computing the false positive rate. :-) What you computed is the false discovery rate. Let FP = number of false positives TP = number of true positives FN = number of false negative TN = number of true negative What you computed is FP/(FP+TP). The false positive rate is FP/(FP + TN). To compute that we need to know TN. Example. Suppose 1000 people were screened, with 985 true negatives, 5 false negatives and 10 positives, and of those 10 positives only 4 were true positives. Then the false positive rate would be 6/(6+985) = 0.605%. That names for some of the various things you can compute from FP, TP, FN, and TN are given here [1]. With medical screening you are often have the case that only a small fraction of the people being screened have the condition. Even if a test is wrong only a small fraction of the time, (large fraction of people without the condition) x (small chance of them getting a false positive) can easily be as big as (small fraction of people with the condition) x (large chance of getting a true positive). Continuing with the example from above, what screening does is take you from "We've got 1000 people and probably around 10 have the condition" to "here's 10 people and probably around 4 have the condition". You can then use more expensive tests that wouldn't be feasible to do on 1000 people on those 10 to find the ones who really have it. [1] https://en.wikipedia.org/wiki/Confusion_matrix https://en.wikipedia.org/wiki/Confusion_matrix
- AlbertCory 3y ago
  OK, you're right, and I have done this before. My bad. However, the inverse but highly relevant questions the patient asks are: Given that I got a positive result, what's the chance that I actually have cancer? Given that I got a negative result, what's the chance that I actually have it anyway? I could attempt this, but right now I have something else to do. Will check back later and probably find that someone's answered those!

it says "In a clinical study, around 4 out of 10 people with this result were diagnosed with cancer after diagnostic testing." so is my arithmetic correct, in that it's a 60% false positive rate?

dekhn 3y ago

That corresponds to the publication, https://www.thelancet.com/journals/lancet/article/PIIS0140-6736(23)01700-2/fulltext?rss=yes https://www.thelancet.com/journals/lancet/article/PIIS0140-6... "A cancer signal was detected in 92 (1·4%) of 6621 participants with analysable results. 35 (38%) participants were diagnosed with cancer (true positives) and 57 (62%) had no cancer diagnosis (false positives)." I am surprised that they were approved based on their study results, it will be interesting to see the results from PATHFINDER 2, which is running now.

nradov 3y ago
If there a legal reason that the test shouldn't be approved? Although the false positive rate is high, it's ultimately up to the patient and their doctor about whether to order the test and what to do with the results. I do understand the concerns over false positives unnecessarily scaring patients and leading to wasteful additional diagnostic tests, but I don't think the FDA really accounts for that in the approval process. Detecting early stage cancer in a liquid biopsy is extremely difficult so it's impressive that they made it work at all.
- dekhn 3y ago
  It's policy not law (the law give the FDA authority to set policy and makes decisions). And they definitely would consider the sensitivity and specificity and PPV of a test. TBH it's amazing it got approved. But the product leadership and founding team definitely knew how to play the FDA game. Yes, early cancer tests are fantastic. However, tests need to have extremely good sensitivity and specificity for exactly the reason you said.

tzs 3y ago

Assuming that you calculated that by subtracting 4 from 10, then divided the results by 10, and expressed that as a percentage, then yes, your arithmetic is correct. It's just not the right arithmetic for computing the false positive rate. :-) What you computed is the false discovery rate. Let FP = number of false positives TP = number of true positives FN = number of false negative TN = number of true negative What you computed is FP/(FP+TP). The false positive rate is FP/(FP + TN). To compute that we need to know TN. Example. Suppose 1000 people were screened, with 985 true negatives, 5 false negatives and 10 positives, and of those 10 positives only 4 were true positives. Then the false positive rate would be 6/(6+985) = 0.605%. That names for some of the various things you can compute from FP, TP, FN, and TN are given here [1]. With medical screening you are often have the case that only a small fraction of the people being screened have the condition. Even if a test is wrong only a small fraction of the time, (large fraction of people without the condition) x (small chance of them getting a false positive) can easily be as big as (small fraction of people with the condition) x (large chance of getting a true positive). Continuing with the example from above, what screening does is take you from "We've got 1000 people and probably around 10 have the condition" to "here's 10 people and probably around 4 have the condition". You can then use more expensive tests that wouldn't be feasible to do on 1000 people on those 10 to find the ones who really have it. [1] https://en.wikipedia.org/wiki/Confusion_matrix https://en.wikipedia.org/wiki/Confusion_matrix

AlbertCory 3y ago
OK, you're right, and I have done this before. My bad. However, the inverse but highly relevant questions the patient asks are: Given that I got a positive result, what's the chance that I actually have cancer? Given that I got a negative result, what's the chance that I actually have it anyway? I could attempt this, but right now I have something else to do. Will check back later and probably find that someone's answered those!