4 ms·

96% specificity. That's really good for a screening test.

DangerousPie 6y ago
As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.
- deleted 6y ago
  [deleted]
- bananaface 6y ago
  What kind of prevalance do you need for that to be bad?
  - hn_throwaway_99 6y ago
    The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
    - bananaface 6y ago
      Perfect! Thank you.
  - blamestross 6y ago
    Lets whip out Bayes' rule: P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general) Lets say 1% true positive rate is a "useless test". 0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity) X = 0.0005 Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014 [cite](https://www.cancer.org/cancer/stomach-cancer/about/key-statistics.html#:~:text=Stomach%20cancer%20mostly%20affects%20older,is%20about%201%20in%2095. https://www.cancer.org/cancer/stomach-cancer/about/key-stati...) Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that. This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.
koheripbal 6y ago
The issue is that there's no follow-up test, at the moment. The cancer is otherwise undetectable. ...and the test does not determine location, so it's impossible to treat even with a positive test. So from a clinical standpoint it's nearly useless, except to schedule patients for more frequent traditional testing. ...the counter point is that the stress caused to a patient is probably not trivial, even from a physical health perspective. ...but from a scientific standpoint, it's awesome and well worth further study.
- IshKebab 6y ago
  > except to schedule patients for more frequent traditional testing That sounds very useful to me!
- psychometry 6y ago
  You actually could potentially determine location by examining mutation patterns, both by looking more closely at methylation and doing further sequencing of cell-free DNA.
- abcc8 6y ago
  I imagine that there will be a longitudinal trial for this test. Individuals will be tested at set intervals and the test results will be correlated with clinical outcomes. It is possible that enough blood samples have already been collected such these same correlates could be identified retrospectively.
- mNovak 6y ago
  >> Nearly useless Being able to catch a cancer early on because of increased surveillance is hugely valuable. Part of why cancer is so dangerous is because it often grows/spreads for years before you get any symptoms. If you're in the unlucky cohort of <50 years old and have an asymptomatic cancer, it might go decades before anyone thinks to check. And mind, increased surveillance typically means once a year tests or scans, so it's not so overly onerous. It's possible even that generally being in frequent contact with doctors and more seriously minding your health could have net benefits in preventing other maladies as well.

As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.

deleted 6y ago
[deleted]
bananaface 6y ago
What kind of prevalance do you need for that to be bad?
- hn_throwaway_99 6y ago
  The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
  - bananaface 6y ago
    Perfect! Thank you.
- blamestross 6y ago
  Lets whip out Bayes' rule: P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general) Lets say 1% true positive rate is a "useless test". 0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity) X = 0.0005 Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014 [cite](https://www.cancer.org/cancer/stomach-cancer/about/key-statistics.html#:~:text=Stomach%20cancer%20mostly%20affects%20older,is%20about%201%20in%2095. https://www.cancer.org/cancer/stomach-cancer/about/key-stati...) Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that. This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.

koheripbal 6y ago

The issue is that there's no follow-up test, at the moment. The cancer is otherwise undetectable. ...and the test does not determine location, so it's impossible to treat even with a positive test. So from a clinical standpoint it's nearly useless, except to schedule patients for more frequent traditional testing. ...the counter point is that the stress caused to a patient is probably not trivial, even from a physical health perspective. ...but from a scientific standpoint, it's awesome and well worth further study.

IshKebab 6y ago
> except to schedule patients for more frequent traditional testing That sounds very useful to me!
psychometry 6y ago
You actually could potentially determine location by examining mutation patterns, both by looking more closely at methylation and doing further sequencing of cell-free DNA.
abcc8 6y ago
I imagine that there will be a longitudinal trial for this test. Individuals will be tested at set intervals and the test results will be correlated with clinical outcomes. It is possible that enough blood samples have already been collected such these same correlates could be identified retrospectively.
mNovak 6y ago
>> Nearly useless Being able to catch a cancer early on because of increased surveillance is hugely valuable. Part of why cancer is so dangerous is because it often grows/spreads for years before you get any symptoms. If you're in the unlucky cohort of <50 years old and have an asymptomatic cancer, it might go decades before anyone thinks to check. And mind, increased surveillance typically means once a year tests or scans, so it's not so overly onerous. It's possible even that generally being in frequent contact with doctors and more seriously minding your health could have net benefits in preventing other maladies as well.