4 ms·

The submission title has taken the liberty of heavily editorializing. This is a submission to a minor journal, of a complicated material, attempting to do some

The submission title has taken the liberty of heavily editorializing. This is a submission to a minor journal, of a complicated material, attempting to do something that is clinically very difficult. I'm not trying to dismiss this research, but please don't expect this to ever be clinically available.

Pinegulf 4y ago
Is it a minor journal? They claim impact factor over 12. This is, of course, field related, but it's not nothing.
- JPLeRouzic 4y ago
  If I understand well the WP article about 'impact factor', it tells that in 2016 Nature was graded at 38 and Plos One at 3, so 'impact factor' might not mean too much. And the criticism section of the WP article is quite long: https://en.wikipedia.org/wiki/Impact_factor https://en.wikipedia.org/wiki/Impact_factor
nxpnsv 4y ago
The devil is in the details. Near perfect should be weighted against prevalence….
radg0wu570ghk 4y ago
I agree that the HN title is editorialised, but can we talk about the actual paper? The paper’s title and abstract aren’t unreasonable, why is it that it will never be clinically viable?
birriel 4y ago
Please consider that the researchers reported a 99% detection rate of those particular cancers in normal people, using a urine test. One would be hard-pressed to come up with a less editorialized title, given HN's constraints.
- tptacek 4y ago
  I didn't read the paper, but in case you did: is it "near perfect accuracy", or is it 99%? The base rate of pancreatic cancer is very low, so a test with 99% specificity is going to overwhelmingly generate false positives.
  - nerdponx 4y ago
    The abstract claims that both sensitivity and specificity are high: > The developed platform successfully classified the human prostate and pancreatic cancer urines in a label-free method supported by two types of deep learning networks, with high clinical sensitivity and specificity. I don't have access to the full article so I can't see the numbers. It's not on the hub of science yet either.
    - gwd 4y ago
      It's not a sensitivity / specificity thing; it's a base rate thing [1]. Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000 (i.e., then "base rate" is 1 in 1000). (Turns out this is actually on the same order of magnitude as real number [2].). And suppose this test has 99% sensitivity and 99% specificity. And suppose you test 1,000,000 people. Of those 1,000,000 people, 1,000 will actually have prostate cancer, and 999,000 will not. Of those 1000 that actually have it, 990 will have a positive test (true positive), and 10 will have a negative test (false negative). Of the 999,000 people who don't actually have it, 989,010 will have a negative test (true negative), and 9,990 will have a positive test (false positive). So even with a test of 99% accuracy, if you get a positive result, your chances of actually having prostate cancer are still only 990 / 10980, or about 9%; 91% of the positives will be false positives. And of course, the more rare the cancer, the worse it gets. EDIT2: So, to follow on with GGP's point: "Near perfect accuracy" isn't very specific, but colloquially would imply that if you have a positive test, you have a high chance of actually having cancer. To get that number to 95% you'd need to have only 52 false positives, would require a specificity closer to 99.995%. EDIT: Fixed some math [1] https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy [2] https://www.cancer.org/cancer/prostate-cancer/about/key-statistics.html https://www.cancer.org/cancer/prostate-cancer/about/key-stat...
nerdponx 4y ago
I think maybe the most interesting thing about it from my perspective is that deep learning is being integrated into this kind of technology. For all of the corporate immorality of Google and Facebook, the value to society of Tensorflow and Torch being free software cannot be overstated.
josalhor 4y ago
From the paper and for contextualisation: > In the same manner, the trained deep learning for pancreatic cancer urine dataset (Figs. S13a–c) clearly showed superior classification performance with a sensitivity of 98.6% and a specificity of 100% (99.3% accuracy, 0.9892 AUC, 59 epochs).
- ludvigk 4y ago
  Seems strange to me that they put the epoch number there. How many people did they test?
  - avs733 4y ago
    My suspicion in reading the abstract and intro is that this is a translation issue. Inference informed by: 1) All the authors have South Korean institutional affiliations 2) "Detection of human biofluids such as blood, tears, saliva, sweat, and urine is important for clinical analysis of various physiological patterns" (First sentence, seems to be missing a word) 3) "differentiate patients from the normal group with high sensitivity and specificity" (abstract, patients and normal group is an odd way of phrasing this) That being said...it had never occurred to me that nueral networks might be a useful way of interpreting spectroscopy data, that is a really cool insight
    - survirtual 4y ago
      Neural nets operating on mel sprectograms (spectrograms shifted to bias frequencies that are of interest to humans) have remarkable ability for audio classification and synthesis. It stands to reason it would not be difficult to adapt methods towards nearly anything which can be mapped to an image in a similar way. I don’t think its particularly novel in concept and kind of surprised it isn’t significantly further along. I think we are at the technology level to really make medicine significantly more affordable and available, at the cost of many high paying doctor’s jobs. Same can be said for lawyers. These are powerful groups that will not take being automated lightly.
  - josalhor 4y ago
dang 4y ago
We've changed the title. Submitted title was "Urine Test Detects Prostate and Pancreatic Cancers with Near-Perfect Accuracy". Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html If the original title won't fit HN's 80 char limit, it can usually be shortened, as we did here.

Pinegulf 4y ago

Is it a minor journal? They claim impact factor over 12. This is, of course, field related, but it's not nothing.

JPLeRouzic 4y ago
If I understand well the WP article about 'impact factor', it tells that in 2016 Nature was graded at 38 and Plos One at 3, so 'impact factor' might not mean too much. And the criticism section of the WP article is quite long: https://en.wikipedia.org/wiki/Impact_factor https://en.wikipedia.org/wiki/Impact_factor

nxpnsv 4y ago

The devil is in the details. Near perfect should be weighted against prevalence….

radg0wu570ghk 4y ago

I agree that the HN title is editorialised, but can we talk about the actual paper? The paper’s title and abstract aren’t unreasonable, why is it that it will never be clinically viable?

birriel 4y ago

Please consider that the researchers reported a 99% detection rate of those particular cancers in normal people, using a urine test. One would be hard-pressed to come up with a less editorialized title, given HN's constraints.

tptacek 4y ago
I didn't read the paper, but in case you did: is it "near perfect accuracy", or is it 99%? The base rate of pancreatic cancer is very low, so a test with 99% specificity is going to overwhelmingly generate false positives.
- nerdponx 4y ago
  The abstract claims that both sensitivity and specificity are high: > The developed platform successfully classified the human prostate and pancreatic cancer urines in a label-free method supported by two types of deep learning networks, with high clinical sensitivity and specificity. I don't have access to the full article so I can't see the numbers. It's not on the hub of science yet either.
  - gwd 4y ago
    It's not a sensitivity / specificity thing; it's a base rate thing [1]. Suppose that the percentage of people with prostate cancer at any one time is 1 in 1000 (i.e., then "base rate" is 1 in 1000). (Turns out this is actually on the same order of magnitude as real number [2].). And suppose this test has 99% sensitivity and 99% specificity. And suppose you test 1,000,000 people. Of those 1,000,000 people, 1,000 will actually have prostate cancer, and 999,000 will not. Of those 1000 that actually have it, 990 will have a positive test (true positive), and 10 will have a negative test (false negative). Of the 999,000 people who don't actually have it, 989,010 will have a negative test (true negative), and 9,990 will have a positive test (false positive). So even with a test of 99% accuracy, if you get a positive result, your chances of actually having prostate cancer are still only 990 / 10980, or about 9%; 91% of the positives will be false positives. And of course, the more rare the cancer, the worse it gets. EDIT2: So, to follow on with GGP's point: "Near perfect accuracy" isn't very specific, but colloquially would imply that if you have a positive test, you have a high chance of actually having cancer. To get that number to 95% you'd need to have only 52 false positives, would require a specificity closer to 99.995%. EDIT: Fixed some math [1] https://en.wikipedia.org/wiki/Base_rate_fallacy https://en.wikipedia.org/wiki/Base_rate_fallacy [2] https://www.cancer.org/cancer/prostate-cancer/about/key-statistics.html https://www.cancer.org/cancer/prostate-cancer/about/key-stat...

nerdponx 4y ago

I think maybe the most interesting thing about it from my perspective is that deep learning is being integrated into this kind of technology. For all of the corporate immorality of Google and Facebook, the value to society of Tensorflow and Torch being free software cannot be overstated.

josalhor 4y ago

From the paper and for contextualisation: > In the same manner, the trained deep learning for pancreatic cancer urine dataset (Figs. S13a–c) clearly showed superior classification performance with a sensitivity of 98.6% and a specificity of 100% (99.3% accuracy, 0.9892 AUC, 59 epochs).

ludvigk 4y ago
Seems strange to me that they put the epoch number there. How many people did they test?
- avs733 4y ago
  My suspicion in reading the abstract and intro is that this is a translation issue. Inference informed by: 1) All the authors have South Korean institutional affiliations 2) "Detection of human biofluids such as blood, tears, saliva, sweat, and urine is important for clinical analysis of various physiological patterns" (First sentence, seems to be missing a word) 3) "differentiate patients from the normal group with high sensitivity and specificity" (abstract, patients and normal group is an odd way of phrasing this) That being said...it had never occurred to me that nueral networks might be a useful way of interpreting spectroscopy data, that is a really cool insight
  - survirtual 4y ago
    Neural nets operating on mel sprectograms (spectrograms shifted to bias frequencies that are of interest to humans) have remarkable ability for audio classification and synthesis. It stands to reason it would not be difficult to adapt methods towards nearly anything which can be mapped to an image in a similar way. I don’t think its particularly novel in concept and kind of surprised it isn’t significantly further along. I think we are at the technology level to really make medicine significantly more affordable and available, at the cost of many high paying doctor’s jobs. Same can be said for lawyers. These are powerful groups that will not take being automated lightly.
- josalhor 4y ago

dang 4y ago

We've changed the title. Submitted title was "Urine Test Detects Prostate and Pancreatic Cancers with Near-Perfect Accuracy". Submitters: "Please use the original title, unless it is misleading or linkbait; don't editorialize." https://news.ycombinator.com/newsguidelines.html https://news.ycombinator.com/newsguidelines.html If the original title won't fit HN's 80 char limit, it can usually be shortened, as we did here.