8 ms·

The real false positive rate is much higher than the 5% reported. The false positive rate reported is ~5% in a sample with an equal number of people that did a

The real false positive rate is much higher than the 5% reported. The false positive rate reported is ~5% in a sample with an equal number of people that did and did not develop cancer, while the ratio in reality is 1/122 (i.e., 1000 people developed cancer, 122000 did not). Could still be a useful development, but it does make me wonder about the cost/benefits at this stage.

deleted 6y ago
[deleted]
patall 6y ago
Eventually, this is all about cost. cfDNA assays are targetted to in the end cost less than 100 bucks per sample which for the detection of a single cancer puts you in the range of $12000 for one tumor diagnosis. Ideally, you can now decrease the therapy cost of that one diagnosis while also boosting survival chances. And $12000 is not much in tumor therapy. The key however here is the advanced prognosis that may allow totally different therapeutic approaches, extend the understanding risk factors etc. and the fact that you do not need to have a specialist on side, taking a blood sample can in principle be done at home and the rest is automatic. Edit: Oh, and 5% false positives is not that bad actually, I do not have numbers is my head but would expect other essays like breast cancer screening to be also in that range.
- helsinkiandrew 6y ago
  5% false positives on people who have some symptoms of cancer isn’t bad. But 5% of a ‘high risk group’ could be a lot - particularly if the cancer is too early to confirm detection any other way yet and possibly not treatable for several years.
  - minerjoe 6y ago
    They'll find something to sell in this case.
    - ethbro 6y ago
      Time provides options in cancer. Even if the first option is "get more regular checkups over the next year+, to characterize the growth rate." You don't have to nuke an organ from orbit, if you have historical data that indicates a tumor isn't aggressive. Or you would, if you've been watching it and it suddenly takes off.
      - minerjoe 6y ago
        "You don't" but many doctors sure seem happy to treat early and to be blind to the potential harm that the treatement entails.
        ethbro 6y ago
        And many software developers don't properly use a CVS. But that doesn't make GitHub a bad idea.
        batty_alex 6y ago
        The problem is that you don’t want to treat every cancer like it’s potentially life-threatening (many aren’t) because you can cause other problems with detection/treatment. I’d want any early detection to be pretty damn solid if we’re going to start talking monthly checkups for something that could resolve on its own. Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week.
        ethbro 6y ago
        > Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week. You haven't worked at a shop that mandated Team Foundation Server, have you? ;)
  - refurb 6y ago
    But in this example - stomach cancer - you could just skip the test and do a routine endoscopy and get some tissue samples. You’d have a much higher confidence in the result. Reminds me a bit of the PSA test for prostate cancer. More of an early warning, but with the weak predictability, doctors are moving away from routine PSA tests.
    - jjk166 6y ago
      If option A is regular blood tests with an endoscopy to confirm should the blood test come back positive and option B is getting endoscopies on a regular basis, option A is a no brainer. While it might not be super invasive, I'd definitely like to keep the number of times a tube has to be shoved down my throat to a minimum.
      - refurb 6y ago
        But if the false positive rate is high, just skip the test and do the endoscopy.
        jjk166 6y ago
        No, the false positive rate is irrelevant. Let's say you want to check for cancer annually and the blood test has a 50% false positive rate: If you only get endoscopies after positive results, you get an endoscopy every other year. If you skip the test and go straight to the endoscopy, you get one every year. Thus you get half as many tubes shoved down your throat if you get the blood test. For a more reasonable false positive rate like 5%, you would have a 50/50 chance of getting a false positive any given decade. I'd much rather get 4 endoscopies than 40. Indeed, for any false positive rate less than 100% (which isn't really a test to begin with) still on average decreases the number of endoscopies you need. Really the metric that's important is the false negatives: if that's too high then I can't safely forego the endoscopy just because this year's blood test was negative. Even then though, if the blood test has a long lead time then maybe it's safe to wait and see how next year's blood test goes.
- 488643689 6y ago
  The problem is with breast cancer screening the next step probably isn't a full body CT. If you detect cancer in the blood, next step would be actually finding it. A full body CT would actually increase your cancer risk and add the chances for another false positive which might involve more invasive diagnosis. The real risk/benefit calculations quickly become tricky. All that said, I'd rather risk dying while propofol'ed, than from say colon cancer. I mean all this is about how we die, not if we have to die.
  - fqwefqwef902 6y ago
    Coming from the field of cognitive neuroscience, where MRI scans are commonplace (because you can get an estimate of brain activity by doing a continuous, "functional" MRI scan), CT feels like ancient technology to me. There are highly specific case where it has benefits, so it has a place. But if I went into the hospital and they sent me in for CT rather than MRI to get an anatomical scan, they better have a damn good reason for it.
    - carbocation 6y ago
      CT and MRI are simply different imaging modalities that provide different information, usually used to answer different questions. There are tradeoffs for each. By the way, they were both invented around the same time (CT is only older by like ~5 years).
    - hylaride 6y ago
      Excuse my ignorance, but I've always been told that CTs are significantly cheaper and faster to administer. According to the famous XKCD radiation graph, a full CT head scan is about 50% of the normal natural yearly background radiation you'll get. Not something to cheer over, but nothing to lose sleep over either. If a CT scan would suffice, why tie up the MRI machine?
      - hda111 6y ago
        The dosage is in a shorter amount of time (within seconds instead of months). I don’t think the radiation averaged over half a year is comparable to the head CT.
- HeadsUpHigh 6y ago
  If you need 12k$ for a single tumor diagnosis, then we should consider MRI imaging. Those cost significantly less than that. And as of right now, the positive effects of chemotherapy on such early cancers would be debatable. Still a useful development though.
  - 488643689 6y ago
    Worth noting an MRT is of low value for prevention. Small lesions would be invisible and you could very well progress from nothing visible to seriously sick in 6 month. We really need some marker for before we get unhindered growth. Even if we had 100% precise AI assisted image analysis, once a year fullbody MRI... the MRI resolution would still not catch the "before things escalate quickly" pre stage of cancer. With imagery catching _most cancers early is inherently a thing of mostly luck.
  - patall 6y ago
    You misunderstand. 12k$ (or some number in that ballpark) would be per postive test for testing everyone. Your 12k$ for MRI would check a single person. That's something totally different. The point is that treating a cancer patient easily costs tens or of hundreds of thousands plus dire consequences for the patient themselves. So there is a lot of room for improvement if the diagnosis is made that much earlier.
- noknownsender 6y ago
  Another potential benefit of this sort of early detection would be the ability to more effectively test interventions that might have little effect on more advanced cancers. That could help direct research into the development of cancers at its earliest stages, and potentially pave the way for more effective prevention.
blablabla123 6y ago
Blood tests for testing cancer markers during therapy are already standard since a long time. A family member of mine got that testing regularly about 10 years ago but the predictive value was not good. Still, if the accuracy improves it's a good development and could be revolutionary once high enough since some types of cancer are very likely to be curable if detected early enough. But yes, a false positive result can make things worse, especially for people that are in a risk group.
- edwhitesell 6y ago
  > but the predictive value was not good My understanding is that there is some dependency on the type of cancer, but mostly it depends on your immune system response. Doctors are pretty clear that just having a cold can affect the numbers.
blago 6y ago
Maybe it can be used as a component/signal in a broader basket of diagnostic tests and symptoms?
mrfusion 6y ago
Could you take it multiple times to help? Or it could be a guide to what additional screening you do?
- DangerousPie 6y ago
  That would only help if the false positive occur completely at random. If the error is systematic (for example because the test is detecting something else in the blood) you'd just keep getting false positives.
  - jjk166 6y ago
    presumably these other things that would cause repeated false positives would be identified over time and could be ruled out with other tests - ie you either have cancer or a serious iron deficiency, let's check if you have an iron deficiency.
- noknownsender 6y ago
  It probably depends on what is causing the false positive. If it is just some variance in the test, then multiple testings could help (And the test could probably be improved to reduce false positives pretty easily) But if there is something intrinsic to those tested (say, some hormone level or other) Repeat tests will give repeat false positives. AND it may be harder to eliminate false positives by refining the test
DangerousPie 6y ago
The false positive rate doesn't depend on the number of true positives in the population. However, as you rightly point out the metric that actually matters for these kinds of screening is the precision/positive predictive value, and that one does.
- Beltiras 6y ago
  96% specificity. That's really good for a screening test.
  - DangerousPie 6y ago
    As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.
    - deleted 6y ago
      [deleted]
    - bananaface 6y ago
      What kind of prevalance do you need for that to be bad?
      - hn_throwaway_99 6y ago
        The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
        bananaface 6y ago
        Perfect! Thank you.
      - blamestross 6y ago
deleted 6y ago
[deleted]
abcc8 6y ago
Yes, the positive predictive value of the test may be low, with the PPV being equal to the number of true positives divide by the number of true positives plus the number of false positives. Since this test is designed to identify cancer very early, a likely solution to the low PPV problem would be to re-test the individual, perhaps even multiple times. The likelihood of multiple consecutive FP test results, barring a biological reason for the FP, would at some point warrant a more accurate test, be it a screen or diagnostic test depending on tumor type. Also keep in mind that the PPV of the test would increase with a person's age - an 80 year old is at a greater risk of developing cancer than is a 30 year old. This test would also be useful for people with hereditary cancer syndromes. The cost benefit would be huge - imagine if everyone could identify their cancer 6 months or even years earlier. Mortality would greatly decrease, as would spending on treatments, etc.
- mbreese 6y ago
  The cost benefit for someone with a cancer diagnosis could be huge. But for the people with false positives, that could also be very harmful to people. A (false) cancer diagnosis would be incredibly stressful. It’s tricky to find that balance.
  - abcc8 6y ago
    This test is a screening test and thus can never be used to diagnose someone. Only a diagnostic test can be used to diagnose someone. Screening tests like these will represent a paradigm shift in cancer surveillance and as such will require those administering the test, most likely a pcp, to properly educate each patient about what the test results mean and what steps should be taken next.
  - taylorlunt 6y ago
    This seems like a failure in how test results are communicated to patients. Patients should just been told that they have been "flagged for further testing", rather than being told they have tested positive for cancer.
    - 6y ago
m463 6y ago
Personally I'd like to know I'm in really early stages. I would take positive steps waiting for a retest, such as cutting out refined carbs, saunas or other heat therapies, and eliminating stress and other harmful situations. false positive? probably still a better lifestyle will have it's own benefits. (IMHO)

deleted 6y ago

[deleted]

patall 6y ago

Eventually, this is all about cost. cfDNA assays are targetted to in the end cost less than 100 bucks per sample which for the detection of a single cancer puts you in the range of $12000 for one tumor diagnosis. Ideally, you can now decrease the therapy cost of that one diagnosis while also boosting survival chances. And $12000 is not much in tumor therapy. The key however here is the advanced prognosis that may allow totally different therapeutic approaches, extend the understanding risk factors etc. and the fact that you do not need to have a specialist on side, taking a blood sample can in principle be done at home and the rest is automatic. Edit: Oh, and 5% false positives is not that bad actually, I do not have numbers is my head but would expect other essays like breast cancer screening to be also in that range.

helsinkiandrew 6y ago
5% false positives on people who have some symptoms of cancer isn’t bad. But 5% of a ‘high risk group’ could be a lot - particularly if the cancer is too early to confirm detection any other way yet and possibly not treatable for several years.
- minerjoe 6y ago
  They'll find something to sell in this case.
  - ethbro 6y ago
    Time provides options in cancer. Even if the first option is "get more regular checkups over the next year+, to characterize the growth rate." You don't have to nuke an organ from orbit, if you have historical data that indicates a tumor isn't aggressive. Or you would, if you've been watching it and it suddenly takes off.
    - minerjoe 6y ago
      "You don't" but many doctors sure seem happy to treat early and to be blind to the potential harm that the treatement entails.
      - ethbro 6y ago
        And many software developers don't properly use a CVS. But that doesn't make GitHub a bad idea.
        batty_alex 6y ago
        The problem is that you don’t want to treat every cancer like it’s potentially life-threatening (many aren’t) because you can cause other problems with detection/treatment. I’d want any early detection to be pretty damn solid if we’re going to start talking monthly checkups for something that could resolve on its own. Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week.
        ethbro 6y ago
        > Using an SCM tool isn’t going to cause additional problems except for reminding yourself of all the dumb mistakes you made last week. You haven't worked at a shop that mandated Team Foundation Server, have you? ;)
- refurb 6y ago
  But in this example - stomach cancer - you could just skip the test and do a routine endoscopy and get some tissue samples. You’d have a much higher confidence in the result. Reminds me a bit of the PSA test for prostate cancer. More of an early warning, but with the weak predictability, doctors are moving away from routine PSA tests.
  - jjk166 6y ago
    If option A is regular blood tests with an endoscopy to confirm should the blood test come back positive and option B is getting endoscopies on a regular basis, option A is a no brainer. While it might not be super invasive, I'd definitely like to keep the number of times a tube has to be shoved down my throat to a minimum.
    - refurb 6y ago
      But if the false positive rate is high, just skip the test and do the endoscopy.
      - jjk166 6y ago
        No, the false positive rate is irrelevant. Let's say you want to check for cancer annually and the blood test has a 50% false positive rate: If you only get endoscopies after positive results, you get an endoscopy every other year. If you skip the test and go straight to the endoscopy, you get one every year. Thus you get half as many tubes shoved down your throat if you get the blood test. For a more reasonable false positive rate like 5%, you would have a 50/50 chance of getting a false positive any given decade. I'd much rather get 4 endoscopies than 40. Indeed, for any false positive rate less than 100% (which isn't really a test to begin with) still on average decreases the number of endoscopies you need. Really the metric that's important is the false negatives: if that's too high then I can't safely forego the endoscopy just because this year's blood test was negative. Even then though, if the blood test has a long lead time then maybe it's safe to wait and see how next year's blood test goes.
488643689 6y ago
The problem is with breast cancer screening the next step probably isn't a full body CT. If you detect cancer in the blood, next step would be actually finding it. A full body CT would actually increase your cancer risk and add the chances for another false positive which might involve more invasive diagnosis. The real risk/benefit calculations quickly become tricky. All that said, I'd rather risk dying while propofol'ed, than from say colon cancer. I mean all this is about how we die, not if we have to die.
- fqwefqwef902 6y ago
  Coming from the field of cognitive neuroscience, where MRI scans are commonplace (because you can get an estimate of brain activity by doing a continuous, "functional" MRI scan), CT feels like ancient technology to me. There are highly specific case where it has benefits, so it has a place. But if I went into the hospital and they sent me in for CT rather than MRI to get an anatomical scan, they better have a damn good reason for it.
  - carbocation 6y ago
    CT and MRI are simply different imaging modalities that provide different information, usually used to answer different questions. There are tradeoffs for each. By the way, they were both invented around the same time (CT is only older by like ~5 years).
  - hylaride 6y ago
    Excuse my ignorance, but I've always been told that CTs are significantly cheaper and faster to administer. According to the famous XKCD radiation graph, a full CT head scan is about 50% of the normal natural yearly background radiation you'll get. Not something to cheer over, but nothing to lose sleep over either. If a CT scan would suffice, why tie up the MRI machine?
    - hda111 6y ago
      The dosage is in a shorter amount of time (within seconds instead of months). I don’t think the radiation averaged over half a year is comparable to the head CT.
HeadsUpHigh 6y ago
If you need 12k$ for a single tumor diagnosis, then we should consider MRI imaging. Those cost significantly less than that. And as of right now, the positive effects of chemotherapy on such early cancers would be debatable. Still a useful development though.
- 488643689 6y ago
  Worth noting an MRT is of low value for prevention. Small lesions would be invisible and you could very well progress from nothing visible to seriously sick in 6 month. We really need some marker for before we get unhindered growth. Even if we had 100% precise AI assisted image analysis, once a year fullbody MRI... the MRI resolution would still not catch the "before things escalate quickly" pre stage of cancer. With imagery catching _most cancers early is inherently a thing of mostly luck.
- patall 6y ago
  You misunderstand. 12k$ (or some number in that ballpark) would be per postive test for testing everyone. Your 12k$ for MRI would check a single person. That's something totally different. The point is that treating a cancer patient easily costs tens or of hundreds of thousands plus dire consequences for the patient themselves. So there is a lot of room for improvement if the diagnosis is made that much earlier.
noknownsender 6y ago
Another potential benefit of this sort of early detection would be the ability to more effectively test interventions that might have little effect on more advanced cancers. That could help direct research into the development of cancers at its earliest stages, and potentially pave the way for more effective prevention.

blablabla123 6y ago

Blood tests for testing cancer markers during therapy are already standard since a long time. A family member of mine got that testing regularly about 10 years ago but the predictive value was not good. Still, if the accuracy improves it's a good development and could be revolutionary once high enough since some types of cancer are very likely to be curable if detected early enough. But yes, a false positive result can make things worse, especially for people that are in a risk group.

edwhitesell 6y ago
> but the predictive value was not good My understanding is that there is some dependency on the type of cancer, but mostly it depends on your immune system response. Doctors are pretty clear that just having a cold can affect the numbers.

blago 6y ago

Maybe it can be used as a component/signal in a broader basket of diagnostic tests and symptoms?

mrfusion 6y ago

Could you take it multiple times to help? Or it could be a guide to what additional screening you do?

DangerousPie 6y ago
That would only help if the false positive occur completely at random. If the error is systematic (for example because the test is detecting something else in the blood) you'd just keep getting false positives.
- jjk166 6y ago
  presumably these other things that would cause repeated false positives would be identified over time and could be ruled out with other tests - ie you either have cancer or a serious iron deficiency, let's check if you have an iron deficiency.
noknownsender 6y ago
It probably depends on what is causing the false positive. If it is just some variance in the test, then multiple testings could help (And the test could probably be improved to reduce false positives pretty easily) But if there is something intrinsic to those tested (say, some hormone level or other) Repeat tests will give repeat false positives. AND it may be harder to eliminate false positives by refining the test

DangerousPie 6y ago

The false positive rate doesn't depend on the number of true positives in the population. However, as you rightly point out the metric that actually matters for these kinds of screening is the precision/positive predictive value, and that one does.

Beltiras 6y ago
96% specificity. That's really good for a screening test.
- DangerousPie 6y ago
  As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.
  - deleted 6y ago
    [deleted]
  - bananaface 6y ago
    What kind of prevalance do you need for that to be bad?
    - hn_throwaway_99 6y ago
      The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
      - bananaface 6y ago
        Perfect! Thank you.
    - blamestross 6y ago

deleted 6y ago

[deleted]

abcc8 6y ago

Yes, the positive predictive value of the test may be low, with the PPV being equal to the number of true positives divide by the number of true positives plus the number of false positives. Since this test is designed to identify cancer very early, a likely solution to the low PPV problem would be to re-test the individual, perhaps even multiple times. The likelihood of multiple consecutive FP test results, barring a biological reason for the FP, would at some point warrant a more accurate test, be it a screen or diagnostic test depending on tumor type. Also keep in mind that the PPV of the test would increase with a person's age - an 80 year old is at a greater risk of developing cancer than is a 30 year old. This test would also be useful for people with hereditary cancer syndromes. The cost benefit would be huge - imagine if everyone could identify their cancer 6 months or even years earlier. Mortality would greatly decrease, as would spending on treatments, etc.

mbreese 6y ago
The cost benefit for someone with a cancer diagnosis could be huge. But for the people with false positives, that could also be very harmful to people. A (false) cancer diagnosis would be incredibly stressful. It’s tricky to find that balance.
- abcc8 6y ago
  This test is a screening test and thus can never be used to diagnose someone. Only a diagnostic test can be used to diagnose someone. Screening tests like these will represent a paradigm shift in cancer surveillance and as such will require those administering the test, most likely a pcp, to properly educate each patient about what the test results mean and what steps should be taken next.
- taylorlunt 6y ago
  This seems like a failure in how test results are communicated to patients. Patients should just been told that they have been "flagged for further testing", rather than being told they have tested positive for cancer.
  - 6y ago

m463 6y ago

Personally I'd like to know I'm in really early stages. I would take positive steps waiting for a retest, such as cutting out refined carbs, saunas or other heat therapies, and eliminating stress and other harmful situations. false positive? probably still a better lifestyle will have it's own benefits. (IMHO)