6 ms·

What would be the harm? It’s a blood test for a broad spectrum of cancers. So any doctor and patient should know that there is a non-zero false positive rate. S

by mo_42 3y ago

What would be the harm? It’s a blood test for a broad spectrum of cancers. So any doctor and patient should know that there is a non-zero false positive rate. So the harm could be CT scans, biopsies, colonoscopies, … Also some kind of psychological trauma about the positive test result. I would have liked to see the real error rates and how many tests they're selling. It seems like this is not part of the false positive rate of the medical test itself. But it is part of the overall false positive rate of the organization.

eska 3y ago
The problem is the same as what destroyed the diagnosis through expert systems the first time around in the 80s/90s.. the software adds an asterisk behind every diagnosis, whether positive or negative, and says “..but talk to your doctor to be sure”. The consequence is that patients flood doctors, and you might as well remove the software from the equation.
- mo_42 3y ago
  Every test has some failure rate aka asterisk. A priori a we cannot dismiss a test for a high failure rate. It depends on how severe the condition is. So for really nasty cancers, we accept higher false positive rates because of the severe consequences. I don’t know about these expert systems. Might very well be that they were overall useless. Another scenario where it’s plausible is skin cancer. Let’s say there’s an app with a false positive rate of 10%. Now worried patients flood dermatologists. They quickly double check and send them home or treat them. Don’t nail me down on numbers here. But it seems like an acceptable situation given that more and more people are dying of skin cancer.
  - eska 3y ago
    I think you may be underestimating the importance of this. Say you have a test that detects skin cancer 60% of the time. ..flipping a coin would do it 50% of the time. Such a test is pretty useless. Remember that you’re applying this diagnosis to the broad population, not somebody who came to your practice with an initial suspicion.
  - DanBC 3y ago
    There's a difference between diagnostic testing of individuals who have symptoms and population level screening. Both involve statistics and probability, and the public (and a surprising number of health care professionals) really struggle with both. To take your number: a disease exists. There's a test for that disease. The test is good, but not perfect. If you have the disease there is a 99% probability that the test will return "Positive". But if you do not have the disease there is a 10% chance that the test will also return "positive". 4 people in 100,000 people have the disease. Hypothetical Bob takes the test. The test shows "positive". What is the probability that Bob actually has the disease? Most of the public, and lots of healthcare providers, cannot even begin to answer this problem. They simply lack the math skill to start to work out what the answer is. 4 people out of 100,000 people have a disease. There is a test for the disease. If you have the disease the test will, 99 times out of 100, say "positive". If you do not have the disease the test will, 10 times out of 100, (100 out of 1,000, 1,000 out of 10,000 or 10,000 out of 100,000) say "negative". Bob has had the test. It said positive. What's the chances he has the disease? Now people can see that only 4 people in 100,000 have the disease and most of those are going to get a positive result, but also 10,000 people who don't have the disease will also get a positive result. We'd be overwhelming diagnostic services with people who do not have cancer because our test is terrible. Of course, all of this changes if Bob has a mole that has changed shape, because the testing converts from a screening test to a diagnostic test.
    - mo_42 3y ago
      Thanks for pointing it out so clearly. Bayesian statistics is the right tool for this analysis as we have a base rate in the population (prior) and a test accuracy (conditional probability).
TylerE 3y ago
> So the harm could be CT scan Guess what CT scans cause (in aggregate populations): Cancer! It's literally a buttload of xrays. To put some concrete numbers on it, "They estimated that one full-body CT scan in a 45-year-old person confers an increased lifetime risk of cancer death of 0.08%, which is approximately 1 in 1250 people. Moreover, annual scans from ages 45 to 75 years could result in an increased risk of cancer mortality of 1.9%, or approximately 1 in 50 people." Read More: https://www.ajronline.org/doi/10.2214/AJR.12.9226 https://www.ajronline.org/doi/10.2214/AJR.12.9226
- steveBK123 3y ago
  Yes, annual is overkill more likely than not, but there are countries that do periodic CT scans above a certain age. Knowing a number of people in their 30s-50s who passed or had close calls with slow moving but symptomless cancer caught early only because of an unrelated accident.. I'd think CT scans every 5 years or something past say 40ish probably has a good ROI for the individual. That said, it doesn't have a great ROI for insurance companies, which is why it doesn't happen. There are numerous types of internal organ cancers that are incredibly slow moving (10-20+ years) but by the time they give you symptoms you are in stage 4. And many of these have essentially no alternative screenings for early warning - thyroid / liver / pancreas / kidney / etc.
DanBC 3y ago
> So the harm could be CT scans, biopsies, colonoscopies Colonoscopy is low risk, but not no risk, and there are plenty of people who have a colonoscopy that goes wrong and bamn they're taken for surgery, anaesthetised, have part of their colon removed, and maybe have a (sometimes temporary) stoma fitted. Anaesthesia is not low risk. Over-testing leads to over-diagnosis which causes over-treatment, and that causes harm.
sokoloff 3y ago
You might be significantly overestimating what “any patient” knows.

by mo_42 3y ago

eska 3y ago

The problem is the same as what destroyed the diagnosis through expert systems the first time around in the 80s/90s.. the software adds an asterisk behind every diagnosis, whether positive or negative, and says “..but talk to your doctor to be sure”. The consequence is that patients flood doctors, and you might as well remove the software from the equation.

mo_42 3y ago
Every test has some failure rate aka asterisk. A priori a we cannot dismiss a test for a high failure rate. It depends on how severe the condition is. So for really nasty cancers, we accept higher false positive rates because of the severe consequences. I don’t know about these expert systems. Might very well be that they were overall useless. Another scenario where it’s plausible is skin cancer. Let’s say there’s an app with a false positive rate of 10%. Now worried patients flood dermatologists. They quickly double check and send them home or treat them. Don’t nail me down on numbers here. But it seems like an acceptable situation given that more and more people are dying of skin cancer.
- eska 3y ago
  I think you may be underestimating the importance of this. Say you have a test that detects skin cancer 60% of the time. ..flipping a coin would do it 50% of the time. Such a test is pretty useless. Remember that you’re applying this diagnosis to the broad population, not somebody who came to your practice with an initial suspicion.
- DanBC 3y ago
  There's a difference between diagnostic testing of individuals who have symptoms and population level screening. Both involve statistics and probability, and the public (and a surprising number of health care professionals) really struggle with both. To take your number: a disease exists. There's a test for that disease. The test is good, but not perfect. If you have the disease there is a 99% probability that the test will return "Positive". But if you do not have the disease there is a 10% chance that the test will also return "positive". 4 people in 100,000 people have the disease. Hypothetical Bob takes the test. The test shows "positive". What is the probability that Bob actually has the disease? Most of the public, and lots of healthcare providers, cannot even begin to answer this problem. They simply lack the math skill to start to work out what the answer is. 4 people out of 100,000 people have a disease. There is a test for the disease. If you have the disease the test will, 99 times out of 100, say "positive". If you do not have the disease the test will, 10 times out of 100, (100 out of 1,000, 1,000 out of 10,000 or 10,000 out of 100,000) say "negative". Bob has had the test. It said positive. What's the chances he has the disease? Now people can see that only 4 people in 100,000 have the disease and most of those are going to get a positive result, but also 10,000 people who don't have the disease will also get a positive result. We'd be overwhelming diagnostic services with people who do not have cancer because our test is terrible. Of course, all of this changes if Bob has a mole that has changed shape, because the testing converts from a screening test to a diagnostic test.
  - mo_42 3y ago
    Thanks for pointing it out so clearly. Bayesian statistics is the right tool for this analysis as we have a base rate in the population (prior) and a test accuracy (conditional probability).

TylerE 3y ago

> So the harm could be CT scan Guess what CT scans cause (in aggregate populations): Cancer! It's literally a buttload of xrays. To put some concrete numbers on it, "They estimated that one full-body CT scan in a 45-year-old person confers an increased lifetime risk of cancer death of 0.08%, which is approximately 1 in 1250 people. Moreover, annual scans from ages 45 to 75 years could result in an increased risk of cancer mortality of 1.9%, or approximately 1 in 50 people." Read More: https://www.ajronline.org/doi/10.2214/AJR.12.9226 https://www.ajronline.org/doi/10.2214/AJR.12.9226

steveBK123 3y ago
Yes, annual is overkill more likely than not, but there are countries that do periodic CT scans above a certain age. Knowing a number of people in their 30s-50s who passed or had close calls with slow moving but symptomless cancer caught early only because of an unrelated accident.. I'd think CT scans every 5 years or something past say 40ish probably has a good ROI for the individual. That said, it doesn't have a great ROI for insurance companies, which is why it doesn't happen. There are numerous types of internal organ cancers that are incredibly slow moving (10-20+ years) but by the time they give you symptoms you are in stage 4. And many of these have essentially no alternative screenings for early warning - thyroid / liver / pancreas / kidney / etc.

DanBC 3y ago

> So the harm could be CT scans, biopsies, colonoscopies Colonoscopy is low risk, but not no risk, and there are plenty of people who have a colonoscopy that goes wrong and bamn they're taken for surgery, anaesthetised, have part of their colon removed, and maybe have a (sometimes temporary) stoma fitted. Anaesthesia is not low risk. Over-testing leads to over-diagnosis which causes over-treatment, and that causes harm.

sokoloff 3y ago

You might be significantly overestimating what “any patient” knows.