6 ms·

My company has used ML to create synthetic cancer data to train classifiers to augment doctors/specialists who are looking for cancer. This work has greatly inc

My company has used ML to create synthetic cancer data to train classifiers to augment doctors/specialists who are looking for cancer. This work has greatly increased accuracy in diagnosis, saving lives. To say it's only for music generation or generating waifus is a bit unfair.

sillysaurusx 6y ago
That's actually wonderful to hear! Is there some way to assist with that work? Doing something with a human impact is appealing. (To be clear, my argument wasn't that ML isn't useful -- but rather that individual lone hackers are less likely to be using ML to achieve superman-type powers than I originally thought. Supermen do exist, but they are firmly in the ranks of DeepMind et al, and must pursue projects collectively rather than individually.)
- yorwba 6y ago
  > individual lone hackers are less likely to be using ML to achieve superman-type powers than I originally thought For a single individual to have "superhuman" impact with ML, they need not only generic ML knowledge, but also specialized knowledge of some domain they want to impact. Actually, because ML has become so generic (just grab a pre-trained model, maybe fine-tune it, and push your data through it) a very shallow understanding of the fundamentals is probably enough, and in-depth domain knowledge much more important. That doesn't mean generic ML research isn't important, it's just that it has an average impact on everything, not a huge impact in one specific area. (I suspect many hobbyist ML projects are about generating entertaining content because everyone has experience with entertainment, even ML researchers.)
  - np_tedious 6y ago
    They also likely need a lot of training data and compute resources
  - counters 6y ago
    !00% nailed it. ML/AI are tools. As in any other exercise, tools can make it easier for beginners/amateurs to engage with a project, but they don't replace 10,000 hours of experience and deep domain experience and understanding. It's the master craftsmen and domain experts who will create the most value with these tools, but that may not be in obvious or clearly visible ways.
- rscho 6y ago
  Yes, if you carry some weight in the field then the most useful contribution would be to push for better automated gathering of quality clinical data. This is the most limiting factor currently. Of course policy activism is far less sexy than building new shiny things, so there's little interest in that.
bosie 6y ago
Can you expand on how you create that synthetic data and how the evaluation (increased accuracy in diagnosis) works?
- deleted 6y ago
  [deleted]
ramraj07 6y ago
It has affected a small number of fields for sure. Imaging based diagnosis might be better off due to ML, but imaging based diagnosis isn't going to cure cancer or be helpful for every single disease. Glad it's helping but the authors point is only reinforced. Unless you can make a case that we will cure basically everything with ML that is.
- TaupeRanger 6y ago
  ML diagnosis could actually be worse for us overall, as we might find more harmless cancers and subject people to more unnecessary tests and treatments. Iatrogenic harms are real, especially when ML gives us only diagnostics, and never any treatments.
- yread 6y ago
  Imaging based diagnosis is not going to cure cancer, but it can guide treatment - based on what the AI reads from the images patients can get drugs that are very effective for their particular cancer. We have very effective drugs nowadays, large part of treating cancer is figuring which drug to give. Imaging based diagnosis could read presence or absence of particular gene mutations from the images so that the genes can be silenced by the drugs. Imaging based diagnosis could also figure out whether a particular cancer precursor is going to develop into invasive cancer and do it better than the experts we have now (otherwise we wouldn't use the AI). This can also be done cheaper than paying consultants to figure it out and it can be done in locations where they don't have the specialists. Some companies working in the field (some already have tools approved for use on patients): https://analogintelligence.com/artificial-intelligence-ai-startups-pathology-venture-meta-review-analysis/ https://analogintelligence.com/artificial-intelligence-ai-st...
  - timy2shoes 6y ago
    > Imaging based diagnosis could read presence or absence of particular gene mutations from the images so that the genes can be silenced by the drugs. >Imaging based diagnosis could also figure out whether a particular cancer precursor is going to develop into invasive cancer and do it better than the experts we have now (otherwise we wouldn't use the AI). Where is the evidence for these claims, other than a VC hype sheet? Like real clinical trials. These claims also show a fundamental misunderstanding of what this data can tell us. Imaging data doesn't give you tumor genetic profiles. It can give you tumor phenotype, which is associated with specific mutations. To get the true genetic profile you need to do deep sequencing at tens of thousands of dollars per tumor, and even then you have the problem of tumor heterogeneity, which lets the cancer evade the treatment. A major concern I have working in this space is that we're selling people on grand promises of far off possibilities rather than what we can actually deliver right now.
polm23 6y ago
Regina Barzilay has done some work in this area; I posted slides from a great talk she gave years back a few years ago. The slides seem to be gone and not on Internet Archive sadly... https://news.ycombinator.com/item?id=20019355 https://news.ycombinator.com/item?id=20019355 This Twitter thread also has a lot of good stuff. https://twitter.com/maite_taboada/status/1086415051127308288 https://twitter.com/maite_taboada/status/1086415051127308288
- polm23 6y ago
  Ah, found the PDF - the URL changed at some point before disappearing. https://web.archive.org/web/20190527041657/http://people.csail.mit.edu/regina/talks/CNLP.pdf https://web.archive.org/web/20190527041657/http://people.csa...
lasagnaphil 6y ago
I think ML's currently temporarily useful in fields that have been making decisions mostly based on intuition and heuristics. The medical field's one example, even with some knowledge on biology and anatomy it's hard to diagnose and treat patients only with deductive reasoning, a lot of guesswork and "experience" is involved. In that case ML might be able to perform better than humans, but I think this will have its limits. Above a certain point, I think biological simulation (as in physics simulation) would be a much more useful tool for doctors to understand the human body.
- deleted 6y ago
  [deleted]
- hderms 6y ago
  I don't know, if you ever look at a flowchart of biochemical processes, realizing that what we've mapped out is only a tiny sliver of what actually occurs, you'd be more pessimistic about simulation in the near term. We can simulate things all we want but the hard part is rooting the simulation in hard evidence, something which requires massive capital and time investment. Epigenetics complicate even further.
  - imbnwa 6y ago
    Would you happen to have any links to share further explaining the limits of our knowledge of biochem processes? How does epigentics complicate this further, is it that it wides the number of inputs into a biochem system
    - hderms 6y ago
      Yeah it would increase the conditionality or contextuality of any given observation because even if the genomes were identical, you have differing levels of gene expression based on environmental stimuli. So observing that gene X impacts biochemical pathway in some way Y is already really difficult when there are tons of other genes at play. Add on the fact that these genes could be triggered to stop expressing themselves in certain conditions and it makes the whole process of figuring out what is really going on that much more difficult. Even if we can make some observation, there are tons of contextual situations which would potentially invalidate that observation.
TaupeRanger 6y ago
Do you have any papers published to support such a strong claim (one that directly contradicts the sentiment of almost every single oncologist and pharmacologist I know that isn't trying to generate profits for a biotech company)?
- ganstyles 6y ago
  edit: I posted that I have internal data, but also realized I said a little bit too much about the process. The below point someone is making is a totally fair one. Editing this though for Reasons while trying to keep the part of the comment that led to the below dismissal, and also to clarify my definition of "internal data" to be more expansive than "internal testing on datasets" which is what I realize it might sound like.
  - TaupeRanger 6y ago
    That's not how this works. The only way to show an actual reduction in all cause mortality as the result of an intervention, treatment, or screening process is through a randomized controlled clinical trial. If none have been performed, you don't have evidence that lives have been saved. Extraordinary claims require extraordinary evidence.
    - hadsed 6y ago
      i think focusing entirely on one type of evidence is a little unimaginative. if these guys have data on doctors performing some task with and without their tool, they're in a good place to measure the difference. they can take that all the way to the bank, and to me that would contribute to what id call evidence.
      - rscho 6y ago
        Until they have to really show that it's working in day-to-day practice. Where it most likely won't, unfortunately.
    - jhowell 6y ago
rscho 6y ago
> This work has greatly increased accuracy in diagnosis, saving lives. As an MD with a special interest in statistics, color me skeptical. I'd love to be proven wrong though, so please provide references. Edit: yeah, so the way this whole thread is developing really goes to show (yet again) that medical AI hype is relying as strongly as ever on the fantasies of people who've never seen any clinical work.
- BadInformatics 6y ago
  I know of at least one Canadian hospital that's incorporated ML into 100% of their ED triage. Sure it's not some state of the art deep learning architecture, but it's definitely a step above the old crop of heuristic-based systems you see so often in medical software. "Medical AI" is a stupid term that's been co-opted by more hucksters than legitimate practitioners, so I prefer to talk about more concrete (and less fanciful) applications like patient chart OCR or capacity forecasting.
  - rscho 6y ago
    > I know of at least one Canadian hospital that's incorporated ML into 100% of their ED triage. Cool! Which hospital is that? Is the clinical staff happy with the results? Personally, I've never seen any medical ML application that made my job easier. But it would be nice to see.
- Thebroser 6y ago
  Maybe not hard data, but EPIC's software (which is used in about ~25% of hospitals for EHR) has over the years been utilizing patient data to be used for treatment recommendation purposes. Again, difficult to weigh the impact if we don't know how many doctors are relying on these types of recommendations and acting on them but it is definitely out there in the real world at the moment. >>https://www.epic.com/software#AI https://www.epic.com/software#AI
  - rscho 6y ago
    > we don't know how many doctors are relying on these types of recommendations and acting on them I can answer that: close to zero. Clinicians don't want stuff that makes recommendations, as good as they may be. They want a bycicle for the mind: something that helps them visualize, understand the big picture and anticipate better. And also ensure that trivial stuff to do is not forgotten (now that's the place a recommender engine could fit in). That's a fundamental misunderstanding of what a clinician's job is that is unfortunately very common. What do you ask of your software tooling? Do you want something that just tells you what to write? No, you want a flexible debugger. A compiler with precise error messages. You want a profiler with a zillion detailed charts allowing you to understand how everything fits together and why such and such is not the way you anticipated. Same thing for medicine until the day machines will actually do better than humans, which is not tomorrow nor the day after.
pdimitar 6y ago
What did your company to do share those findings with the doctors worldwide? Did your company reduce mortality? Otherwise it's kind of like, I have invented SkyNet in my garage but I am only using it to become richer through the stock market. It's admirable that you are working on saving human lives. But are human lives actually saved?
- edouard-harris 6y ago
  > Did your company reduce mortality? From the parent: > This work has greatly increased accuracy in diagnosis, saving lives.
  - rscho 6y ago
    ... which is apparently pure speculation
jhowell 6y ago
Earlier detection. Is this new tech, or just a sharper hammer?
- rscho 6y ago
  We don't know yet. What's sure is that it has greatly increased the number of cancer surgeries, especially lungs. We don't know if that's a good thing yet.

sillysaurusx 6y ago

That's actually wonderful to hear! Is there some way to assist with that work? Doing something with a human impact is appealing. (To be clear, my argument wasn't that ML isn't useful -- but rather that individual lone hackers are less likely to be using ML to achieve superman-type powers than I originally thought. Supermen do exist, but they are firmly in the ranks of DeepMind et al, and must pursue projects collectively rather than individually.)

yorwba 6y ago
> individual lone hackers are less likely to be using ML to achieve superman-type powers than I originally thought For a single individual to have "superhuman" impact with ML, they need not only generic ML knowledge, but also specialized knowledge of some domain they want to impact. Actually, because ML has become so generic (just grab a pre-trained model, maybe fine-tune it, and push your data through it) a very shallow understanding of the fundamentals is probably enough, and in-depth domain knowledge much more important. That doesn't mean generic ML research isn't important, it's just that it has an average impact on everything, not a huge impact in one specific area. (I suspect many hobbyist ML projects are about generating entertaining content because everyone has experience with entertainment, even ML researchers.)
- np_tedious 6y ago
  They also likely need a lot of training data and compute resources
- counters 6y ago
  !00% nailed it. ML/AI are tools. As in any other exercise, tools can make it easier for beginners/amateurs to engage with a project, but they don't replace 10,000 hours of experience and deep domain experience and understanding. It's the master craftsmen and domain experts who will create the most value with these tools, but that may not be in obvious or clearly visible ways.
rscho 6y ago
Yes, if you carry some weight in the field then the most useful contribution would be to push for better automated gathering of quality clinical data. This is the most limiting factor currently. Of course policy activism is far less sexy than building new shiny things, so there's little interest in that.

bosie 6y ago

Can you expand on how you create that synthetic data and how the evaluation (increased accuracy in diagnosis) works?

deleted 6y ago
[deleted]

ramraj07 6y ago

It has affected a small number of fields for sure. Imaging based diagnosis might be better off due to ML, but imaging based diagnosis isn't going to cure cancer or be helpful for every single disease. Glad it's helping but the authors point is only reinforced. Unless you can make a case that we will cure basically everything with ML that is.

TaupeRanger 6y ago
ML diagnosis could actually be worse for us overall, as we might find more harmless cancers and subject people to more unnecessary tests and treatments. Iatrogenic harms are real, especially when ML gives us only diagnostics, and never any treatments.
yread 6y ago
Imaging based diagnosis is not going to cure cancer, but it can guide treatment - based on what the AI reads from the images patients can get drugs that are very effective for their particular cancer. We have very effective drugs nowadays, large part of treating cancer is figuring which drug to give. Imaging based diagnosis could read presence or absence of particular gene mutations from the images so that the genes can be silenced by the drugs. Imaging based diagnosis could also figure out whether a particular cancer precursor is going to develop into invasive cancer and do it better than the experts we have now (otherwise we wouldn't use the AI). This can also be done cheaper than paying consultants to figure it out and it can be done in locations where they don't have the specialists. Some companies working in the field (some already have tools approved for use on patients): https://analogintelligence.com/artificial-intelligence-ai-startups-pathology-venture-meta-review-analysis/ https://analogintelligence.com/artificial-intelligence-ai-st...
- timy2shoes 6y ago
  > Imaging based diagnosis could read presence or absence of particular gene mutations from the images so that the genes can be silenced by the drugs. >Imaging based diagnosis could also figure out whether a particular cancer precursor is going to develop into invasive cancer and do it better than the experts we have now (otherwise we wouldn't use the AI). Where is the evidence for these claims, other than a VC hype sheet? Like real clinical trials. These claims also show a fundamental misunderstanding of what this data can tell us. Imaging data doesn't give you tumor genetic profiles. It can give you tumor phenotype, which is associated with specific mutations. To get the true genetic profile you need to do deep sequencing at tens of thousands of dollars per tumor, and even then you have the problem of tumor heterogeneity, which lets the cancer evade the treatment. A major concern I have working in this space is that we're selling people on grand promises of far off possibilities rather than what we can actually deliver right now.

polm23 6y ago

Regina Barzilay has done some work in this area; I posted slides from a great talk she gave years back a few years ago. The slides seem to be gone and not on Internet Archive sadly... https://news.ycombinator.com/item?id=20019355 https://news.ycombinator.com/item?id=20019355 This Twitter thread also has a lot of good stuff. https://twitter.com/maite_taboada/status/1086415051127308288 https://twitter.com/maite_taboada/status/1086415051127308288

polm23 6y ago
Ah, found the PDF - the URL changed at some point before disappearing. https://web.archive.org/web/20190527041657/http://people.csail.mit.edu/regina/talks/CNLP.pdf https://web.archive.org/web/20190527041657/http://people.csa...

lasagnaphil 6y ago

I think ML's currently temporarily useful in fields that have been making decisions mostly based on intuition and heuristics. The medical field's one example, even with some knowledge on biology and anatomy it's hard to diagnose and treat patients only with deductive reasoning, a lot of guesswork and "experience" is involved. In that case ML might be able to perform better than humans, but I think this will have its limits. Above a certain point, I think biological simulation (as in physics simulation) would be a much more useful tool for doctors to understand the human body.

deleted 6y ago
[deleted]
hderms 6y ago
I don't know, if you ever look at a flowchart of biochemical processes, realizing that what we've mapped out is only a tiny sliver of what actually occurs, you'd be more pessimistic about simulation in the near term. We can simulate things all we want but the hard part is rooting the simulation in hard evidence, something which requires massive capital and time investment. Epigenetics complicate even further.
- imbnwa 6y ago
  Would you happen to have any links to share further explaining the limits of our knowledge of biochem processes? How does epigentics complicate this further, is it that it wides the number of inputs into a biochem system
  - hderms 6y ago
    Yeah it would increase the conditionality or contextuality of any given observation because even if the genomes were identical, you have differing levels of gene expression based on environmental stimuli. So observing that gene X impacts biochemical pathway in some way Y is already really difficult when there are tons of other genes at play. Add on the fact that these genes could be triggered to stop expressing themselves in certain conditions and it makes the whole process of figuring out what is really going on that much more difficult. Even if we can make some observation, there are tons of contextual situations which would potentially invalidate that observation.

TaupeRanger 6y ago

Do you have any papers published to support such a strong claim (one that directly contradicts the sentiment of almost every single oncologist and pharmacologist I know that isn't trying to generate profits for a biotech company)?

ganstyles 6y ago
edit: I posted that I have internal data, but also realized I said a little bit too much about the process. The below point someone is making is a totally fair one. Editing this though for Reasons while trying to keep the part of the comment that led to the below dismissal, and also to clarify my definition of "internal data" to be more expansive than "internal testing on datasets" which is what I realize it might sound like.
- TaupeRanger 6y ago
  That's not how this works. The only way to show an actual reduction in all cause mortality as the result of an intervention, treatment, or screening process is through a randomized controlled clinical trial. If none have been performed, you don't have evidence that lives have been saved. Extraordinary claims require extraordinary evidence.
  - hadsed 6y ago
    i think focusing entirely on one type of evidence is a little unimaginative. if these guys have data on doctors performing some task with and without their tool, they're in a good place to measure the difference. they can take that all the way to the bank, and to me that would contribute to what id call evidence.
    - rscho 6y ago
      Until they have to really show that it's working in day-to-day practice. Where it most likely won't, unfortunately.
  - jhowell 6y ago

rscho 6y ago

> This work has greatly increased accuracy in diagnosis, saving lives. As an MD with a special interest in statistics, color me skeptical. I'd love to be proven wrong though, so please provide references. Edit: yeah, so the way this whole thread is developing really goes to show (yet again) that medical AI hype is relying as strongly as ever on the fantasies of people who've never seen any clinical work.

BadInformatics 6y ago
I know of at least one Canadian hospital that's incorporated ML into 100% of their ED triage. Sure it's not some state of the art deep learning architecture, but it's definitely a step above the old crop of heuristic-based systems you see so often in medical software. "Medical AI" is a stupid term that's been co-opted by more hucksters than legitimate practitioners, so I prefer to talk about more concrete (and less fanciful) applications like patient chart OCR or capacity forecasting.
- rscho 6y ago
  > I know of at least one Canadian hospital that's incorporated ML into 100% of their ED triage. Cool! Which hospital is that? Is the clinical staff happy with the results? Personally, I've never seen any medical ML application that made my job easier. But it would be nice to see.
Thebroser 6y ago
Maybe not hard data, but EPIC's software (which is used in about ~25% of hospitals for EHR) has over the years been utilizing patient data to be used for treatment recommendation purposes. Again, difficult to weigh the impact if we don't know how many doctors are relying on these types of recommendations and acting on them but it is definitely out there in the real world at the moment. >>https://www.epic.com/software#AI https://www.epic.com/software#AI
- rscho 6y ago
  > we don't know how many doctors are relying on these types of recommendations and acting on them I can answer that: close to zero. Clinicians don't want stuff that makes recommendations, as good as they may be. They want a bycicle for the mind: something that helps them visualize, understand the big picture and anticipate better. And also ensure that trivial stuff to do is not forgotten (now that's the place a recommender engine could fit in). That's a fundamental misunderstanding of what a clinician's job is that is unfortunately very common. What do you ask of your software tooling? Do you want something that just tells you what to write? No, you want a flexible debugger. A compiler with precise error messages. You want a profiler with a zillion detailed charts allowing you to understand how everything fits together and why such and such is not the way you anticipated. Same thing for medicine until the day machines will actually do better than humans, which is not tomorrow nor the day after.

pdimitar 6y ago

What did your company to do share those findings with the doctors worldwide? Did your company reduce mortality? Otherwise it's kind of like, I have invented SkyNet in my garage but I am only using it to become richer through the stock market. It's admirable that you are working on saving human lives. But are human lives actually saved?

edouard-harris 6y ago
> Did your company reduce mortality? From the parent: > This work has greatly increased accuracy in diagnosis, saving lives.
- rscho 6y ago
  ... which is apparently pure speculation

jhowell 6y ago

Earlier detection. Is this new tech, or just a sharper hammer?

rscho 6y ago
We don't know yet. What's sure is that it has greatly increased the number of cancer surgeries, especially lungs. We don't know if that's a good thing yet.