5 ms·
I'm a tedious broken record about the fact that the base rate of most cancers means that extraordinarily-accurate-seeming screening tests have surprisingly unte
by tptacek 1y ago
I'm a tedious broken record about the fact that the base rate of most cancers means that extraordinarily-accurate-seeming screening tests have surprisingly untenable false positive rates. Like, a 99% accurate test for liver cancer might be almost worthless: because the base rate is so low, 99% of positives will be false.
And a false positive screening result is not innocuous: it incurs costs in a variety of different ways, including human health.
- para_parolu 1y agoWhat if you just do these testa every few days? Would this minimize error rate?
- Nifty3929 1y agoProbably not. They are not likely independent. Whatever it is about you that caused the false positive yesterday might likely cause it tomorrow.
- throwup238 1y agoNo, the factors causing the false positives are usually correlated. The most common cause is mutations causing unrelated proteins to mimic an epitope (binding site) of the biomarker protein. Another cause is analytical interference where some other molecule absorbs the wavelength used to measure the assay. Running the test using multiple different labs helps eliminate contamination and handling errors but most false positives are due to genetics and long term environmental factors.
- raffraffraff 1y agoBut if you got one test saying "Detected a fragment of pancreatic cancer DNA" two days later another saying "Didn't see any cancerous DNA fragments" what would the actual effect on the doctor or patient be? Who'll take the risk and say "the first was a false positive"? Also, these tests are a grand a pop if I'm reading it directly (which I may not be)
- para_parolu 1y agoI think it can always be blind process. Only notify when good probability. Plus it can be funnel to other non invasive (possibly) tests. They do cost a lot but that may change with time and scale.
- DavidSJ 1y agoThe article seems to suggest the false positive rate is only 38%: The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed. (It also had a false negative rate of 1%:) The test correctly ruled out cancer in over 99% of those who tested negative.
- hn_throwaway_99 1y agoIf the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test, but they are highly dependent on the underlying prevalence of the disease, and why broad-based testing for relatively rare diseases often results in horrible PVP and PVN metrics. Based on your quoted sections, we can infer: 1. About 250 people got a positive result ("nearly one in 100") 2. Of those 250 people, 155 (62%) actually had cancer, 95 did not. 3. About 24,750 people got a negative test result. 4. Assuming a false negative rate of 1% (the quote says "over 99%") it means of those 24,750 people, about 248 actually did have cancer, while about 24,502 did not. When you write it out like that (and I know I'm making some rounding assumptions on the numbers), it means the test missed the majority of people who had cancer while subjecting over 1/3 of those who tested positive to fear and further expense.
- dv_dt 1y agoso possibly saving lives and late stage cancer care level medical expenses 2/3 of positive results vs fear and lighter medical care 1/3 of the time. is this not a win?
- thaumasiotes 1y ago> If the stats were as good as the hyperbole in the article, it would clearly state the only 2 metrics that really matter: predictive value positive (what's the actual probability that you really have cancer if you test positive) and predictive value negative (what's the actual probability that you're cancer free if you test negative). As tptacek points out, these metrics don't just depend on the sensitivity and specificity of the test This is a bizarre thing to say in response to... a clear statement of the positive and negative predictive value. PPV is 62% and NPV is "over 99%". Your calculations don't appear to have any connection to your criticism. You're trying to back into sensitivity ("the test missed the majority of people who had cancer") from reported PPV and NPV, while complaining that sensitivity is misleading and honest reporting would have stated the PPV and NPV.
- Nifty3929 1y agoThis is an important and unintuitive reality about all kinds of tests. You have to take into account the base rate - the likelihood that you have the disease given what you know prior to the test. If you're otherwise healthy and would have a 1/1,000,000 chance of having the disease before the test, and then you test positive with a test that is 99% accurate, you are ~100x more likely to have the disease than before - but that's still only 1/10,000 - not at all 99% likely, even though the test was "99% accurate" That said, I think with this knowledge the test still confers helpful information. I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative. Depends on how wealthy I am, and how serious the disease is, and what the treatments for it are.
- smt88 1y ago> I might decide to spend $1000 on an additional diagnostic, even knowing that I'm still very likely to be negative. The problem is that diagnostics aren't necessarily risk-free. For example, there's a non-zero risk of death while getting a colonoscopy, to the point that false positives from unnecessary testing can increase all-cause mortality for patients.
- tptacek 1y agoThere's also a real psychic cost to even a tentative positive result, which is why I'm much more likely to get an actual colonoscopy than a Cologuard test.
- genghisjahn 1y agoI had the same thought. I got the colonoscopy and I don’t regret it at all. And for anyone thinking about it or stressing about it, the prep was not that bad. If you’re over 45 just do it.
- alyandon 1y agoI had put it off for several years because everyone kept telling me how awful the prep was. My doctor kept pestering me to agree to do it so I eventually relented. Turns out, I needed to go to the bathroom frequently during the day which was an annoyance but I never had anything close to an "accident" nor did I feel any strong urge to evacuate at night. So the whole experience turned out to be a huge nothing-burger and I had a few polyps that got sniped that weren't cancerous - so now I have peace of mind that I didn't have before.
- mmooss 1y agoDefinitely keep at it. Also, we should know the accuracy of comparable tests. Reading the article, I'm still not sure about the accuracy, and don't have the time to carefully parse the whole article. I see at least the following statements (there may be more): The trial followed 25,000 adults from the US and Canada over a year, with nearly one in 100 getting a positive result. For 62% of these cases, cancer was later confirmed. and The test correctly ruled out cancer in over 99% of those who tested negative.
- SkyPuncher 1y agoThe doctors who perform these tests are trained to understand this type of stuff. They're not just blindly accepting any single test result
- smt88 1y agoThere are Silicon Valley startups increasingly selling these directly to consumers, so I wouldn't assume tests like this are always intermediated by a medical professional. Edit: There is in fact another comment on this thread of someone doing exactly this: https://news.ycombinator.com/item?id=45652535 https://news.ycombinator.com/item?id=45652535
- rottencupcakes 1y agoOne of the multitude of things that makes me frustrated about the medical establishment in the US is how they insist on gatekeeping test results.
- smt88 1y agoDo you feel that we shouldn't have gatekeepers for prescription drugs either?
- rottencupcakes 1y agoThis isn't perscription drugs, this is information? To answer your question directly I don't have strong feelings about it. I would prefer if they were freely accessible, other than maybe things like antibiotics that have clear externalities. I'm okay with insurance companies saying they won't pay for them unless a doctor writes a script, but to gate keep relatively harmless drugs the way we do isn't my favorite policy. There are many times I would love to get a drug but don't want to talk to a doctor about it. This is one of the reasons companies like KHealth exist - they're just rent seeking on a silly system. You can get anything you want, you just have to jump through hoops, but all those hoops just make doctors richer and health care cost more.
- 1y ago
- HWR_14 1y agoIn this case, it is worthwhile. Adjusting for the base rate, the test is correct when it indicates cancer 62% of the time. That's almost certainly worth the false positives.
- gadders 1y agoThis website is a good analysis of various diagnostic tests: https://thennt.com/thennt-explained/ https://thennt.com/thennt-explained/ Also why I don't bother with PSA tests until I start getting symptoms: https://thennt.com/nnt/psa-test-to-screen-for-prostate-cancer-2/ https://thennt.com/nnt/psa-test-to-screen-for-prostate-cance...