9 ms·

I detest this mindset. It is incredibly counter productive. "Our tests are bad, so lets not test" is not a thought worthy of respect. This states that you know

I detest this mindset. It is incredibly counter productive. "Our tests are bad, so lets not test" is not a thought worthy of respect. This states that you know about a problem but want to continue to ignore that problem. Reprehensible. No, we should take the exact opposite approach. Test everyone constantly until such methods become both cheap and powerful. So yeah we all have some cancer load but if the diagnostic or treatment systems can't deal with that reality then those systems need to change.

jacquesm 5y ago
The problem is not my mindset, but your understanding of the subject material. Patient outcomes have very little to do with the quality of the tests, even a 100% accurate test would not necessarily lead to improved patient outcomes because these are defined independent of the tests. You are treating this like a software problem, but it isn't, it's a medical problem, and medical problems tend to be complex because they have a ton of confounding variables that make it hard to have a one-size-fits-all method for dealing with medical issues. What needs to change is that people need to realize that they have a field of expertise and that the medical domain has its own experts who typically dedicate a lifetime to their profession, their general consensus is that improved tests are welcome but in and of themselves are not enough to guarantee improved patient outcomes. Yes, this is unfortunate, but it is also a simple reality, you can either accept that or not, that's up to you but if you want to make a change there than you probably should join the medical profession. Most likely by the time that you have completed the requirements you will have shifted your viewpoint away from the software domain's mindset that all bugs can be found and squashed by the next sprint. Which by the way judging by the general quality of software out there is also something that doesn't work out in practice as we believe it should in theory.
- plank_time 5y ago
  This is an absolutely absurd thing to say. “We’ve never had blood tests that detect pancreatic cancer early, therefore we should never use them because when we used other means to try to detect early, it didn’t lead to better outcomes.” If you can detect colon, kidney pancreatic or liver cancer early, you might be able to do surgery or develop treatments at the stage 1 stage. Right now we don’t have anything except MRIs and CT scans that are too hard and expensive to do frequently. And if you are diagnose with pancreatic cancer it’s usually so late that you will basically die in weeks. You’re basically saying “give up. Even if we detect early you all die anyway” which is frankly stupid. You’re discounting the possibility that early detection by means of a blood test adds whole new layers of possibilities to fight those particularly dangerous cancers.
  - jacquesm 5y ago
    But: that surgery is not without risk, and if there are no symptoms there is a fair chance that there never will be symptoms. Depending on your age and your genetic make-up you might be more or less at risk. There is such a thing as spontaneous remission and so on. So no, I'm not 'basically giving up', and no you won't die anyway (well, unless you take that in the most abstract way), in particular likely not from cancer. Even for those cancers where we do screen (such as for instance breast cancer) it is not a given that the increased frequency of detection has led to better patient outcomes. But once you know someone's genetic disposition increased frequency of testing might be advantageous.
    - BallyBrain 5y ago
      I don't see why early detection means early treatment. What if after detected, the illness is monitored instead. If it then gets to a point where the treatment is no longer considered risky for the stage of the illness, then the treatment can proceed. It would also give the person the opportunity to change lifestyle to perhaps prolong the time that the illness will become a problem, or perhaps halt its progress altogether. Additionally, the person who is aware of such illness, can keep an eye out for symptoms related to it, that might otherwise be ignored as something else. At which point normal cancer treatment can progress.
      - jacquesm 5y ago
        This is a tricky bit. There is a lot of interplay here between medical professionals and the general public, and not all of that is either rational or ethical. But absent symptoms non-treatment is better than treatment. Data about unnecessary procedures is relatively plentiful, which is of course sad, whether that's driven by commercial incentives or the need for 'something to be done' is not something that I have any grip on but it certainly is problematic.
      - BallyBrain 5y ago
        Sounds to me like the issues with this early detection is more of a regulation and education problem then. That could be fixed.
- laputan_machine 5y ago
  For someone who has no medical background, you are certainly spouting a lot of nonsense as if it was fact. The WHO, for example, disagree. https://www.who.int/activities/promoting-cancer-early-diagnosis https://www.who.int/activities/promoting-cancer-early-diagno...
  - jacquesm 5y ago
    The fact that you link to that and apparently do not understand it combined with a personal attack is enough to disregard your comment for me. But let me stress this in case it wasn't clear to you: screening and early diagnosis are not the same thing. If you screen a large population for cancer you will turn up a lot of cancers that may never become a problem, or that may even end up being re-absorbed without the need for intervention, as well as a large number of false positives. Early diagnosis means that there are already symptoms. Please don't confuse the two.
    - laputan_machine 5y ago
      You come in and flame other users, but then take deep offense when you get it back. Treat others with respect and in turn you'll receive it.
      - jacquesm 5y ago
        I think I've spent more than enough time in this thread going out of my way to explain things in as clear and simple a manner as I know how to and if have offended anybody then I apologize for that, it certainly wasn't my intention.
      - pessimizer 5y ago
        Disagreement is not "flaming." Jacquesm has been generally respectful and factual. The closest they have come to rudeness is when people refer to "facts" not in evidence or link to things that they haven't read or understood well, and even that impatience was explanatory. This comment is simply a personal attack.
- wrcwill 5y ago
  this just reads to me as "our tests don't test for the right things, since when we act on those tests (even if they were 100% accurate), it doesn't lead to better patient outcomes". I feel like a better conclusion is that we need better tests, that detect things that when acted upon improve patient outcome. Of course, we're nowhere near that yet, but do you really think in 1000 years we will still wait for patients to be responsible for correctly noticing symptoms and going to the doctor? Of course not. It's good that we have studies that show we should move with caution in this territory, but completely ignoring it forever seems absurd.
- dekhn 5y ago
  I think you should stop trying to derate people who you think know less than you. It's very unattractive. Be aware there are folks with extensive public health experience (that would be me) watching you insist you're right. While I totally appreciate what you're trying to do (help software engineers understand why medicine is complex), please do understand that GRAIL is tightly integrated with the PH community, did their work from a good-faith perspective, and came up with a product that does "do" something. Now it's up to the medical community to evaluate whether it truly provides something that doesn't just end up costing us more money and not helping people.
  - jacquesm 5y ago
    That's perfectly accurate: the problem is that people tend to run with things like this as though they are the miracle that everybody has been waiting for and that is not the case, it is a very important development but in the right hands and not as a tool that will lead to - yet another - round of disappointment in the 'war on cancer'. The whole thread has devolved into people arguing from hope rather than from facts. This set of tests is a very useful thing to have. But the 50% true positive rate, 18% true positive rate for stage I cancers and the false positive rate combined make it at present - as far as I understand this - a tool that could when used as a mass screening tool easily do more harm than good. I'm more than willing to be convinced otherwise but with relevant data. FWIW I've been following this particular development closely because it has direct implications for a start-up that I have had a lot of contact with that is also in the early detection space and they were adamant that the combination of factors is such - and so complex - that test accuracy is trumped significantly by absence of symptoms in otherwise healthy patients. Your input on this would be greatly appreciated.
    - dekhn 5y ago
      my input is that I've been involved with GRAIL since the beginning and I think they've actually done something. We don't have the data to say one way or the other. I am not arguing with most of your points, it's just that you're not being epistemically humble. I find it's far better to just act humble even when you aare schooling people who persist on believing in magic.
- iandanforth 5y ago
  You don't know what you're talking about. Before my software career I spent a few years working in a diagnostic medical field. Specifically osteoporosis testing. I worked both in a research capacity at Stanford looking at osteoporosis in older men (not pretending to be the PI here) and in a day to day testing clinic. So I've seen exactly what happens when you test a cross-sectional asymptomatic sample of the populous and what happens during the normal course of referred testing. Low bone mass at the spine, hip, heel, and forearm as measured by DXA are correlated with increased risk of fracture, but it's only a correlation. Some people have resilient architecture which looks porous on an x-ray but only leads to serious fracture much later in life. Because the current diagnostic tests are set up with levels like 'osteopenia' and 'osteoporosis' the reaction to clinical referrals was most often treatment. Some of those treatments have serious side effects like necrotizing impacts on the jaw. However the reaction to testing in a large asymptomatic population was much more likely to be an increase in preventative behavior (exercise) or no treatment except in extreme cases. While our study was exploratory and didn't have a treatment cohort (we cared about the impact of sleep quality on bone mass) we got to see a lot of older men who if they were referred to a clinic might have received treatment because that's what clinics do. Instead we had sufficient data to discuss what's normal and what isn't. For a time we were the leading experts in the world on what "normal" meant. Because I've conducted these tests myself and seen data from hundreds of experimental and clinical patients I feel comfortable contrasting the two. The problem is the clinical medical field reacting to a lack of data with over-prescription of treatments.
  - jacquesm 5y ago
    Sure, but that's a completely different setting than the one the OP described: mass testing for presence of cancers by non-specialist labs. And that's the thing that triggered my response. The outcome of such an approach could easily be a negative one. FWIW I too have some experience with medical diagnostic systems (specifically: cancer testing), and one of the main reasons why I'm still skeptical about this test is that for many cancer types tested for the base rate is much lower than the false positive rate.
elbasti 5y ago
This is not a question of mindset, it's a question of statistics when you're trying to identify anything that is rare. Take prostate cancer as an example. It's a relatively common cancer, with an incidence of .1% [0]. If you have an _incredible_ screening test with a 99% specificity (ie, 1% false positive rate), then: If you test 100,000 people, you will correctly identify 100 people with cancer, and misdiagnose one thousand people as false positives. There are consequences to having a false positive: unnecessary interventions, surgery "just to be safe", stress, etc. This is why you limit testing to populations that have a higher probability of being ill (namely, show some sort of symptom). Now imagine being tested for 100 things: the odds of coming up with a false positive for something start to approach 1!
- captainmuon 5y ago
  But the tests are not binary sick or healty. If your test gets more sensitive, then you have to shift the, say, PSA value that you use for a diagnosis. I really hope that false positives become a regular thing. For one it shows that the tests are working according to statistics. Second, getting comfortable with false positives means that you can more easily hold off treatment, if in your specific scenario the treatment is not beneficial - think an old test that doesn't find the cancer plus a new test that does; and the new test is shown to lead to overtreatment. We need to learn when to hold on and not to treat. And third, if testing for 10 things will find something that you don't want to know and harm your psyche, when you test for 100 things you will have almost certainty that there is a false positive and this can give you back some of the "blissfull ignorance" I believe (while the doctor can still give you the statistically best treatment).
TaupeRanger 5y ago
The only problematic mindset is yours, and you clearly have not engaged with the subject of iatrogenics in any meaningful way. The policy you advocate for would cause real harm to real people, likely to no benefit. Please understand how wrong you are.
athenot 5y ago
> "Our tests are bad, so lets[sic] not test" is not a thought worthy of respect. Correct. But that's not the issue here. Current tests can detect benign tumors, and do. People go in for their regular tests, hear "cancer" and of course want to get treatment. But that treatment itself is not idempotent. You want to accept the negative side-effects if the alternative is an aggressive cancer that will end your life soon. But you don't want that if there is no cancer. As in many fields, it's a matter of trade-offs and risk assessment. Here's one article detailing some of the issues surrounding overdiagnosis of colorectal cancer: https://www.gastrojournal.org/article/S0016-5085%2818%2934825-X/pdf https://www.gastrojournal.org/article/S0016-5085%2818%293482... (2018)
pessimizer 5y ago
You're advocating for a policy that may kill or hurt more people than the alternative, and insisting that doing the math is reprehensible. The enemy is not cancer, the enemy is ill-health and death. You could cure all deaths from cancer by poisoning everyone with cyanide.
colinmhayes 5y ago
The public health literature has been over this again and again. Over zealous non-symptomatic testing consistently leads to worse outcomes. You can get as philosophical as you want, the fact remains that it lowers total utility.

jacquesm 5y ago

The problem is not my mindset, but your understanding of the subject material. Patient outcomes have very little to do with the quality of the tests, even a 100% accurate test would not necessarily lead to improved patient outcomes because these are defined independent of the tests. You are treating this like a software problem, but it isn't, it's a medical problem, and medical problems tend to be complex because they have a ton of confounding variables that make it hard to have a one-size-fits-all method for dealing with medical issues. What needs to change is that people need to realize that they have a field of expertise and that the medical domain has its own experts who typically dedicate a lifetime to their profession, their general consensus is that improved tests are welcome but in and of themselves are not enough to guarantee improved patient outcomes. Yes, this is unfortunate, but it is also a simple reality, you can either accept that or not, that's up to you but if you want to make a change there than you probably should join the medical profession. Most likely by the time that you have completed the requirements you will have shifted your viewpoint away from the software domain's mindset that all bugs can be found and squashed by the next sprint. Which by the way judging by the general quality of software out there is also something that doesn't work out in practice as we believe it should in theory.

plank_time 5y ago
This is an absolutely absurd thing to say. “We’ve never had blood tests that detect pancreatic cancer early, therefore we should never use them because when we used other means to try to detect early, it didn’t lead to better outcomes.” If you can detect colon, kidney pancreatic or liver cancer early, you might be able to do surgery or develop treatments at the stage 1 stage. Right now we don’t have anything except MRIs and CT scans that are too hard and expensive to do frequently. And if you are diagnose with pancreatic cancer it’s usually so late that you will basically die in weeks. You’re basically saying “give up. Even if we detect early you all die anyway” which is frankly stupid. You’re discounting the possibility that early detection by means of a blood test adds whole new layers of possibilities to fight those particularly dangerous cancers.
- jacquesm 5y ago
  But: that surgery is not without risk, and if there are no symptoms there is a fair chance that there never will be symptoms. Depending on your age and your genetic make-up you might be more or less at risk. There is such a thing as spontaneous remission and so on. So no, I'm not 'basically giving up', and no you won't die anyway (well, unless you take that in the most abstract way), in particular likely not from cancer. Even for those cancers where we do screen (such as for instance breast cancer) it is not a given that the increased frequency of detection has led to better patient outcomes. But once you know someone's genetic disposition increased frequency of testing might be advantageous.
  - BallyBrain 5y ago
    I don't see why early detection means early treatment. What if after detected, the illness is monitored instead. If it then gets to a point where the treatment is no longer considered risky for the stage of the illness, then the treatment can proceed. It would also give the person the opportunity to change lifestyle to perhaps prolong the time that the illness will become a problem, or perhaps halt its progress altogether. Additionally, the person who is aware of such illness, can keep an eye out for symptoms related to it, that might otherwise be ignored as something else. At which point normal cancer treatment can progress.
    - jacquesm 5y ago
      This is a tricky bit. There is a lot of interplay here between medical professionals and the general public, and not all of that is either rational or ethical. But absent symptoms non-treatment is better than treatment. Data about unnecessary procedures is relatively plentiful, which is of course sad, whether that's driven by commercial incentives or the need for 'something to be done' is not something that I have any grip on but it certainly is problematic.
    - BallyBrain 5y ago
      Sounds to me like the issues with this early detection is more of a regulation and education problem then. That could be fixed.
laputan_machine 5y ago
For someone who has no medical background, you are certainly spouting a lot of nonsense as if it was fact. The WHO, for example, disagree. https://www.who.int/activities/promoting-cancer-early-diagnosis https://www.who.int/activities/promoting-cancer-early-diagno...
- jacquesm 5y ago
  The fact that you link to that and apparently do not understand it combined with a personal attack is enough to disregard your comment for me. But let me stress this in case it wasn't clear to you: screening and early diagnosis are not the same thing. If you screen a large population for cancer you will turn up a lot of cancers that may never become a problem, or that may even end up being re-absorbed without the need for intervention, as well as a large number of false positives. Early diagnosis means that there are already symptoms. Please don't confuse the two.
  - laputan_machine 5y ago
    You come in and flame other users, but then take deep offense when you get it back. Treat others with respect and in turn you'll receive it.
    - jacquesm 5y ago
      I think I've spent more than enough time in this thread going out of my way to explain things in as clear and simple a manner as I know how to and if have offended anybody then I apologize for that, it certainly wasn't my intention.
    - pessimizer 5y ago
      Disagreement is not "flaming." Jacquesm has been generally respectful and factual. The closest they have come to rudeness is when people refer to "facts" not in evidence or link to things that they haven't read or understood well, and even that impatience was explanatory. This comment is simply a personal attack.
wrcwill 5y ago
this just reads to me as "our tests don't test for the right things, since when we act on those tests (even if they were 100% accurate), it doesn't lead to better patient outcomes". I feel like a better conclusion is that we need better tests, that detect things that when acted upon improve patient outcome. Of course, we're nowhere near that yet, but do you really think in 1000 years we will still wait for patients to be responsible for correctly noticing symptoms and going to the doctor? Of course not. It's good that we have studies that show we should move with caution in this territory, but completely ignoring it forever seems absurd.
dekhn 5y ago
I think you should stop trying to derate people who you think know less than you. It's very unattractive. Be aware there are folks with extensive public health experience (that would be me) watching you insist you're right. While I totally appreciate what you're trying to do (help software engineers understand why medicine is complex), please do understand that GRAIL is tightly integrated with the PH community, did their work from a good-faith perspective, and came up with a product that does "do" something. Now it's up to the medical community to evaluate whether it truly provides something that doesn't just end up costing us more money and not helping people.
- jacquesm 5y ago
  That's perfectly accurate: the problem is that people tend to run with things like this as though they are the miracle that everybody has been waiting for and that is not the case, it is a very important development but in the right hands and not as a tool that will lead to - yet another - round of disappointment in the 'war on cancer'. The whole thread has devolved into people arguing from hope rather than from facts. This set of tests is a very useful thing to have. But the 50% true positive rate, 18% true positive rate for stage I cancers and the false positive rate combined make it at present - as far as I understand this - a tool that could when used as a mass screening tool easily do more harm than good. I'm more than willing to be convinced otherwise but with relevant data. FWIW I've been following this particular development closely because it has direct implications for a start-up that I have had a lot of contact with that is also in the early detection space and they were adamant that the combination of factors is such - and so complex - that test accuracy is trumped significantly by absence of symptoms in otherwise healthy patients. Your input on this would be greatly appreciated.
  - dekhn 5y ago
    my input is that I've been involved with GRAIL since the beginning and I think they've actually done something. We don't have the data to say one way or the other. I am not arguing with most of your points, it's just that you're not being epistemically humble. I find it's far better to just act humble even when you aare schooling people who persist on believing in magic.
iandanforth 5y ago
You don't know what you're talking about. Before my software career I spent a few years working in a diagnostic medical field. Specifically osteoporosis testing. I worked both in a research capacity at Stanford looking at osteoporosis in older men (not pretending to be the PI here) and in a day to day testing clinic. So I've seen exactly what happens when you test a cross-sectional asymptomatic sample of the populous and what happens during the normal course of referred testing. Low bone mass at the spine, hip, heel, and forearm as measured by DXA are correlated with increased risk of fracture, but it's only a correlation. Some people have resilient architecture which looks porous on an x-ray but only leads to serious fracture much later in life. Because the current diagnostic tests are set up with levels like 'osteopenia' and 'osteoporosis' the reaction to clinical referrals was most often treatment. Some of those treatments have serious side effects like necrotizing impacts on the jaw. However the reaction to testing in a large asymptomatic population was much more likely to be an increase in preventative behavior (exercise) or no treatment except in extreme cases. While our study was exploratory and didn't have a treatment cohort (we cared about the impact of sleep quality on bone mass) we got to see a lot of older men who if they were referred to a clinic might have received treatment because that's what clinics do. Instead we had sufficient data to discuss what's normal and what isn't. For a time we were the leading experts in the world on what "normal" meant. Because I've conducted these tests myself and seen data from hundreds of experimental and clinical patients I feel comfortable contrasting the two. The problem is the clinical medical field reacting to a lack of data with over-prescription of treatments.
- jacquesm 5y ago
  Sure, but that's a completely different setting than the one the OP described: mass testing for presence of cancers by non-specialist labs. And that's the thing that triggered my response. The outcome of such an approach could easily be a negative one. FWIW I too have some experience with medical diagnostic systems (specifically: cancer testing), and one of the main reasons why I'm still skeptical about this test is that for many cancer types tested for the base rate is much lower than the false positive rate.

elbasti 5y ago

This is not a question of mindset, it's a question of statistics when you're trying to identify anything that is rare. Take prostate cancer as an example. It's a relatively common cancer, with an incidence of .1% [0]. If you have an _incredible_ screening test with a 99% specificity (ie, 1% false positive rate), then: If you test 100,000 people, you will correctly identify 100 people with cancer, and misdiagnose one thousand people as false positives. There are consequences to having a false positive: unnecessary interventions, surgery "just to be safe", stress, etc. This is why you limit testing to populations that have a higher probability of being ill (namely, show some sort of symptom). Now imagine being tested for 100 things: the odds of coming up with a false positive for something start to approach 1!

captainmuon 5y ago
But the tests are not binary sick or healty. If your test gets more sensitive, then you have to shift the, say, PSA value that you use for a diagnosis. I really hope that false positives become a regular thing. For one it shows that the tests are working according to statistics. Second, getting comfortable with false positives means that you can more easily hold off treatment, if in your specific scenario the treatment is not beneficial - think an old test that doesn't find the cancer plus a new test that does; and the new test is shown to lead to overtreatment. We need to learn when to hold on and not to treat. And third, if testing for 10 things will find something that you don't want to know and harm your psyche, when you test for 100 things you will have almost certainty that there is a false positive and this can give you back some of the "blissfull ignorance" I believe (while the doctor can still give you the statistically best treatment).

TaupeRanger 5y ago

The only problematic mindset is yours, and you clearly have not engaged with the subject of iatrogenics in any meaningful way. The policy you advocate for would cause real harm to real people, likely to no benefit. Please understand how wrong you are.

athenot 5y ago

> "Our tests are bad, so lets[sic] not test" is not a thought worthy of respect. Correct. But that's not the issue here. Current tests can detect benign tumors, and do. People go in for their regular tests, hear "cancer" and of course want to get treatment. But that treatment itself is not idempotent. You want to accept the negative side-effects if the alternative is an aggressive cancer that will end your life soon. But you don't want that if there is no cancer. As in many fields, it's a matter of trade-offs and risk assessment. Here's one article detailing some of the issues surrounding overdiagnosis of colorectal cancer: https://www.gastrojournal.org/article/S0016-5085%2818%2934825-X/pdf https://www.gastrojournal.org/article/S0016-5085%2818%293482... (2018)

pessimizer 5y ago

You're advocating for a policy that may kill or hurt more people than the alternative, and insisting that doing the math is reprehensible. The enemy is not cancer, the enemy is ill-health and death. You could cure all deaths from cancer by poisoning everyone with cyanide.

colinmhayes 5y ago

The public health literature has been over this again and again. Over zealous non-symptomatic testing consistently leads to worse outcomes. You can get as philosophical as you want, the fact remains that it lowers total utility.