3 ms·

As the parent comment said - whether or not that's "good" enough for a screening test really depends on the prevalence of the disease it's testing for.

deleted 6y ago
[deleted]
bananaface 6y ago
What kind of prevalance do you need for that to be bad?
- hn_throwaway_99 6y ago
  The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
  - bananaface 6y ago
    Perfect! Thank you.
- blamestross 6y ago
  Lets whip out Bayes' rule: P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general) Lets say 1% true positive rate is a "useless test". 0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity) X = 0.0005 Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014 [cite](https://www.cancer.org/cancer/stomach-cancer/about/key-statistics.html#:~:text=Stomach%20cancer%20mostly%20affects%20older,is%20about%201%20in%2095. https://www.cancer.org/cancer/stomach-cancer/about/key-stati...) Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that. This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.

deleted 6y ago

[deleted]

bananaface 6y ago

What kind of prevalance do you need for that to be bad?

hn_throwaway_99 6y ago
The wiki article on positive predictive value has the formula for calculating it based on the sensitivity and specificity of a test, and the prevalence: https://en.m.wikipedia.org/wiki/Positive_and_negative_predictive_values https://en.m.wikipedia.org/wiki/Positive_and_negative_predic...
- bananaface 6y ago
  Perfect! Thank you.
blamestross 6y ago
Lets whip out Bayes' rule: P(Having cancer given positive) = P(Positive given having cancer) * P(Having Cancer) / P(Having a Positive in general) Lets say 1% true positive rate is a "useless test". 0.01 = 0.88 (Detection rate from paper) * X / 0.05 (inverse of specificity) X = 0.0005 Incident rate of Stomach Cancer (most common) is 27,600 per year in US. The denominator for this fraction is iffy because there are a lot of biases on current detection rates. If we go ahead with "adults in the US" then the rate of cancer is 0.00014 [cite](https://www.cancer.org/cancer/stomach-cancer/about/key-statistics.html#:~:text=Stomach%20cancer%20mostly%20affects%20older,is%20about%201%20in%2095. https://www.cancer.org/cancer/stomach-cancer/about/key-stati...) Basically, if we applied this test to all adults without other diagnostic factors. This would be worse than 1% true positive detection rate. I don't know about how effective intersecting with other diagnostic criteria would be on increasing the odds a patient actually have the disease and it might or might not still be useful after that. This sort of study is mostly bullshit anyway. They took a relatively small pile of data, made a small classifier with high accuracy (maybe overtrained, maybe not) and really can't validate it.