3 ms·

> They found none of the cancer-derived mutations among blood samples of 44 healthy individuals. Is 98% specificity adequate for a cancer test?

epistasis 9y ago
Certainly not for a general screen, but then we just know that the specificity is in the range of (98%, 100%) now, not the exact specificity. And we don't know what other refinements might improve upon what they did here. But their data is too valuable to hold back until they had the final result of a test that could be taken to the FDA and approved for general use. This is valuable science, that impacts the directions that others are taking with their research. I'm not saying you're doing this, but it's a common trend on HN and for other non-scientist technical people to forget that scientific journals show the process of science, not completed science. A journal is not a textbook that one can pick up and read a complete, final story, where all the false starts and uninteresting tangents are skipped. Journals are a way for scientists to communicate about their current progress, and that's an essential part of the scientific process. There has to be room to say: "here's where we are." Especially when they are able to get to the point of where they are right now. Also, this was published in a very high profile journal, on a highly competitive topic, so take it all with a huge grain of salt. The senior author of this paper does good work, and many people are talking about similar preliminary results, but it takes time to prove these things.
- AlexCoventry 9y ago
  > I'm not saying you're doing this, but it's a common trend on HN and for other non-scientist technical people to forget that scientific journals show the process of science, not completed science How much would it have cost them to increase the number of controls? How did they choose 44?
  - doctoring 9y ago
    It costs a lot, unfortunately. Sequencing is still fairly expensive, and the methods in OP are fairly complex on top of that. There's also the interesting question of: if the test came back positive in a seemingly-healthy control, how would you know if that was a true positive or a false positive? It's still early in the game as to how to work up a cancer detected only in blood. The gold standard is tissue biopsy, but how far are you willing to go to hunt that down? It's a tough one. Source: My spouse is one of the authors of the study in question.
    - AlexCoventry 9y ago
      > if the test came back positive in a seemingly-healthy control, how would you know if that was a true positive or a false positive? You can estimate the likelihood from cancer prevalance.
mnw21cam 9y ago
Where did the 98% figure come from? 95% confidence intervals from the binomial distribution are for the specificity to be between 92% and 100%.

epistasis 9y ago

Certainly not for a general screen, but then we just know that the specificity is in the range of (98%, 100%) now, not the exact specificity. And we don't know what other refinements might improve upon what they did here. But their data is too valuable to hold back until they had the final result of a test that could be taken to the FDA and approved for general use. This is valuable science, that impacts the directions that others are taking with their research. I'm not saying you're doing this, but it's a common trend on HN and for other non-scientist technical people to forget that scientific journals show the process of science, not completed science. A journal is not a textbook that one can pick up and read a complete, final story, where all the false starts and uninteresting tangents are skipped. Journals are a way for scientists to communicate about their current progress, and that's an essential part of the scientific process. There has to be room to say: "here's where we are." Especially when they are able to get to the point of where they are right now. Also, this was published in a very high profile journal, on a highly competitive topic, so take it all with a huge grain of salt. The senior author of this paper does good work, and many people are talking about similar preliminary results, but it takes time to prove these things.

AlexCoventry 9y ago
> I'm not saying you're doing this, but it's a common trend on HN and for other non-scientist technical people to forget that scientific journals show the process of science, not completed science How much would it have cost them to increase the number of controls? How did they choose 44?
- doctoring 9y ago
  It costs a lot, unfortunately. Sequencing is still fairly expensive, and the methods in OP are fairly complex on top of that. There's also the interesting question of: if the test came back positive in a seemingly-healthy control, how would you know if that was a true positive or a false positive? It's still early in the game as to how to work up a cancer detected only in blood. The gold standard is tissue biopsy, but how far are you willing to go to hunt that down? It's a tough one. Source: My spouse is one of the authors of the study in question.
  - AlexCoventry 9y ago
    > if the test came back positive in a seemingly-healthy control, how would you know if that was a true positive or a false positive? You can estimate the likelihood from cancer prevalance.

mnw21cam 9y ago

Where did the 98% figure come from? 95% confidence intervals from the binomial distribution are for the specificity to be between 92% and 100%.