4 ms·

I disagree with that article as it makes no sense. Their assertion that tests === diagnostics that map 1:1 to a disease was baseless. Tests don't answer the que

I disagree with that article as it makes no sense. Their assertion that tests === diagnostics that map 1:1 to a disease was baseless. Tests don't answer the question "doe patient have diabetes?", they answer the question "what is the patients glucose level?" 115 mg/dl. Maybe the patient forgot to fast, maybe the test was a fluke? Either way a consistent average of 85 would put that out of range. SO I found the five thirty eight article stupid so I guess I misunderstood why more health data would be bad.

deleted 10y ago
[deleted]
kevinnk 10y ago
The 538 article actually makes a lot of sense and is a pretty decent introduction to a common public health issue. > Their assertion that tests === diagnostics that map 1:1 to a disease was baseless. Tests don't answer the question "doe patient have diabetes?", they answer the question "what is the patients glucose level?" 115 mg/dl. Medical tests that aren't actionable aren't useful. So while you're correct in a very narrow sense, you have to use the results to make a decision about treatment. If you're going to make treatment decisions based on tests that have some amount of inherent error, you may end up over or under-treating patients. > Maybe the patient forgot to fast, maybe the test was a fluke? Either way a consistent average of 85 would put that out of range. Medical tests have both random and systematic error. What you're talking about is random error (i.e. a single test result being bad). However, many tests have systematic error (i.e. the result being wrong every time the test is taken) just due to personal variation in what's "normal". The article makes the point that if more tests lead to worse outcomes[1] (due to overtreatment) then adding more tests is useless or worse than useless. This effect is particularly pronounced when testing across large populations, even if the test is highly accurate. [1] And in many cases they do. For example, early mammogram screenings result in worse outcomes for patients when done across a large population. See http://annals.org/article.aspx?articleid=2480757 http://annals.org/article.aspx?articleid=2480757
- cperciva 10y ago
  Medical tests that aren't actionable aren't useful. True, but screening tests are very useful. If you can separate the population into "definitely diabetic", "definitely not diabetic", and "needs more testing" quickly and cheaply, that's a win. The fact that it doesn't always give you an actionable result doesn't make it useless.
  - kevinnk 10y ago
    > If you can separate the population into "definitely diabetic", "definitely not diabetic", and "needs more testing" quickly and cheaply, that's a win. That's the fundamental problem. It turns out with a reasonably accurate test, a low incidence rate, and a large population, the "definitely X" class will be filled with people who aren't actually X. Going back and trying to separate the two ends up costing more and causing more harm to the non-Xs then just waiting until there's a problem. I'd encourage you to take a look at the link I posted since it takes a very clear look at the problem in the context of mammograms and breast cancer.
KingMob 10y ago
One way to look at it is, as you point out, our estimates of a measure are more accurate when you take more measures. But the major issue with more data is Bayes' Law. For decades, running tests and collecting data was expensive, so we only ran them on people who were sick or suspected of being so. For prostate cancer, this data would tell us the probability that one would have a high PSA measurement given that they have prostate cancer (p(high PSA|prostate cancer). E.g., we know that 70% of prostate cancer patients have high PSA. But, as Bayes' Law points out, this does NOT tell us the probability that you have prostate cancer given that you come in for a normal checkup, and we find a high PSA (p(prostate cancer|high PSA)). E.g., we do NOT know that 70% of people with high PSA have prostate cancer (absent other symptoms). In order to determine that, you need to know the probability of having a high PSA in general. And the point is, NOBODY KNOWS these population meaures yet. Collecting lots of data is a good way to establish these baselines, but it's not clear it will help with diagnoses at all. Another real example: we've gotten better and better at detecting tiny tumors early. But once we started looking at the data, we noticed that a large fraction of tiny tumors simply disappear over time. So, when the tumor is small enough, it's not clear whether we should treat it with surgery/radiation/chemo (which have their own risks/downsides) or just wait-and-see.

deleted 10y ago

[deleted]

kevinnk 10y ago

The 538 article actually makes a lot of sense and is a pretty decent introduction to a common public health issue. > Their assertion that tests === diagnostics that map 1:1 to a disease was baseless. Tests don't answer the question "doe patient have diabetes?", they answer the question "what is the patients glucose level?" 115 mg/dl. Medical tests that aren't actionable aren't useful. So while you're correct in a very narrow sense, you have to use the results to make a decision about treatment. If you're going to make treatment decisions based on tests that have some amount of inherent error, you may end up over or under-treating patients. > Maybe the patient forgot to fast, maybe the test was a fluke? Either way a consistent average of 85 would put that out of range. Medical tests have both random and systematic error. What you're talking about is random error (i.e. a single test result being bad). However, many tests have systematic error (i.e. the result being wrong every time the test is taken) just due to personal variation in what's "normal". The article makes the point that if more tests lead to worse outcomes[1] (due to overtreatment) then adding more tests is useless or worse than useless. This effect is particularly pronounced when testing across large populations, even if the test is highly accurate. [1] And in many cases they do. For example, early mammogram screenings result in worse outcomes for patients when done across a large population. See http://annals.org/article.aspx?articleid=2480757 http://annals.org/article.aspx?articleid=2480757

cperciva 10y ago
Medical tests that aren't actionable aren't useful. True, but screening tests are very useful. If you can separate the population into "definitely diabetic", "definitely not diabetic", and "needs more testing" quickly and cheaply, that's a win. The fact that it doesn't always give you an actionable result doesn't make it useless.
- kevinnk 10y ago
  > If you can separate the population into "definitely diabetic", "definitely not diabetic", and "needs more testing" quickly and cheaply, that's a win. That's the fundamental problem. It turns out with a reasonably accurate test, a low incidence rate, and a large population, the "definitely X" class will be filled with people who aren't actually X. Going back and trying to separate the two ends up costing more and causing more harm to the non-Xs then just waiting until there's a problem. I'd encourage you to take a look at the link I posted since it takes a very clear look at the problem in the context of mammograms and breast cancer.

KingMob 10y ago

One way to look at it is, as you point out, our estimates of a measure are more accurate when you take more measures. But the major issue with more data is Bayes' Law. For decades, running tests and collecting data was expensive, so we only ran them on people who were sick or suspected of being so. For prostate cancer, this data would tell us the probability that one would have a high PSA measurement given that they have prostate cancer (p(high PSA|prostate cancer). E.g., we know that 70% of prostate cancer patients have high PSA. But, as Bayes' Law points out, this does NOT tell us the probability that you have prostate cancer given that you come in for a normal checkup, and we find a high PSA (p(prostate cancer|high PSA)). E.g., we do NOT know that 70% of people with high PSA have prostate cancer (absent other symptoms). In order to determine that, you need to know the probability of having a high PSA in general. And the point is, NOBODY KNOWS these population meaures yet. Collecting lots of data is a good way to establish these baselines, but it's not clear it will help with diagnoses at all. Another real example: we've gotten better and better at detecting tiny tumors early. But once we started looking at the data, we noticed that a large fraction of tiny tumors simply disappear over time. So, when the tumor is small enough, it's not clear whether we should treat it with surgery/radiation/chemo (which have their own risks/downsides) or just wait-and-see.