3 ms·

In my PhD research (on computer vision at University of Tokyo), one of the topics I was exploring was reliability of models to synthetic data. Using generative

In my PhD research (on computer vision at University of Tokyo), one of the topics I was exploring was reliability of models to synthetic data. Using generative models, we could show that models which were built on binary data sources were somewhat more reliable than others. Consider different geometries of rashes on skin & different skin location - the model could still be closer to ground truth (i.e model performance on real skin lesions) However when other variables came into the mix (different tones, ethnicity, participant age etc), the model predictions were up for tosses. Now, medical data is much more harder in ML. The variability is quite extreme since all our bodies are little different from one another. In my thesis, I very firmly proposed that generative models should never ever be used in synthetic medical information as part of the thesis proposal - and there were those field studies to back it up. (The thesis is under non-disclosure at University of Tokyo until September 2023 since an embargo is involved from participating hospitals & data sources. I would have been eager to share otherwise). I am glad people aren't wildly trusting if task A, B & C are doable, task AB, CA, ABC, BC etc should be similarly tractable.

deadmutex 5y ago
> I very firmly proposed that generative models should never ever be used Isn't this a bit strong? E.g. just because it is not a good idea for your use case right now, it may not mean that in the future something else happens and it could invalidate your firm proposal.
- srvmshr 5y ago
  Apologies. Yes it seems a bit strong, but going by the understanding of field-data variability - I and few others in the area, feel that it would take nothing short of an absolute miracle to get to human-level error rate in diagnostic medicine. Hence, the confidence for the near future (5-10 years) and a disclaimer against the false notion of improving ML based outcomes. Sometimes negative results paint much truer pictures than amazing ones. Maybe if the paradigm changes for how we design generator networks, perhaps one day this presumption will be invalidated as you correctly pointed. The whole craze about my application i.e. dermatology successes in ML spurred from the 2017 Nature paper, where skin cancer was detected as good or better than dermatologists. But technically, such experiments have design problems: We had apriori knowledge of the dataset (White N American Melanoma data) & hence we could ascertain the model performance. Real world data is much more variable. Further, later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions. 3 years later my experiments could model reliably only on 10 very common diseases & of a very uniform skin type and ethnicity. Those results were nowhere close to perfect. The proposal to keep Human-in-the-loop is a much fairer alternative in ML aided medicine, than end to end machine learning. Most direction of research is headed that way. Physician assistance is much more reliable than potential replacement. The trouble with synthetic data is that it doesn't address the extended variability in real population & generalization will always suffer. Also, doctors take multi-path decision, choice by elimination, past cases - based on several diagnostic inputs & even gut intuition. At that scale of input multimodality, Type I & II errors are at a scale higher than correct identifications. And we don't know how to teach intuition or imagination to machine models well enough. Those fall back to rule based methods & edge cases.
  - overkalix 5y ago
    > later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions ... excuse me?
    - srvmshr 5y ago
      The dermatologists make small dots/arrow markings on the positively identified lesions. What the Nature paper apparently didn't do was remove the small markings in their training. It is probably an innocuous flaw since gradient-based investigation/ interpretation only took off later than their publication (2017). As a result, instead of positively identifying the lesion based on disease pathology, it identified overwhelmingly based on presence or absence of medical marks. There was following up discussion in a certain paper of this experimental design (I can't remember the exact name), but they did gradient based activation mapping and those pointed to the marks as the identifying feature. It felt quite a revelation of why this worked so well. More information: 1.https://jamanetwork.com/journals/jamadermatology/fullarticle/2740808 https://jamanetwork.com/journals/jamadermatology/fullarticle... 2.Swetter (2020) .Novel Technologies to Improve Melanoma Detection and Care Focusing on Artificial Intelligence 3.ISIC Workshop 2019 at CVPR (Slides online) 4.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8074854/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8074854/

deadmutex 5y ago

> I very firmly proposed that generative models should never ever be used Isn't this a bit strong? E.g. just because it is not a good idea for your use case right now, it may not mean that in the future something else happens and it could invalidate your firm proposal.

srvmshr 5y ago
Apologies. Yes it seems a bit strong, but going by the understanding of field-data variability - I and few others in the area, feel that it would take nothing short of an absolute miracle to get to human-level error rate in diagnostic medicine. Hence, the confidence for the near future (5-10 years) and a disclaimer against the false notion of improving ML based outcomes. Sometimes negative results paint much truer pictures than amazing ones. Maybe if the paradigm changes for how we design generator networks, perhaps one day this presumption will be invalidated as you correctly pointed. The whole craze about my application i.e. dermatology successes in ML spurred from the 2017 Nature paper, where skin cancer was detected as good or better than dermatologists. But technically, such experiments have design problems: We had apriori knowledge of the dataset (White N American Melanoma data) & hence we could ascertain the model performance. Real world data is much more variable. Further, later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions. 3 years later my experiments could model reliably only on 10 very common diseases & of a very uniform skin type and ethnicity. Those results were nowhere close to perfect. The proposal to keep Human-in-the-loop is a much fairer alternative in ML aided medicine, than end to end machine learning. Most direction of research is headed that way. Physician assistance is much more reliable than potential replacement. The trouble with synthetic data is that it doesn't address the extended variability in real population & generalization will always suffer. Also, doctors take multi-path decision, choice by elimination, past cases - based on several diagnostic inputs & even gut intuition. At that scale of input multimodality, Type I & II errors are at a scale higher than correct identifications. And we don't know how to teach intuition or imagination to machine models well enough. Those fall back to rule based methods & edge cases.
- overkalix 5y ago
  > later it was revealed that ML model latched on to the little marking physicians made rather than generalizing on lesions ... excuse me?
  - srvmshr 5y ago
    The dermatologists make small dots/arrow markings on the positively identified lesions. What the Nature paper apparently didn't do was remove the small markings in their training. It is probably an innocuous flaw since gradient-based investigation/ interpretation only took off later than their publication (2017). As a result, instead of positively identifying the lesion based on disease pathology, it identified overwhelmingly based on presence or absence of medical marks. There was following up discussion in a certain paper of this experimental design (I can't remember the exact name), but they did gradient based activation mapping and those pointed to the marks as the identifying feature. It felt quite a revelation of why this worked so well. More information: 1.https://jamanetwork.com/journals/jamadermatology/fullarticle/2740808 https://jamanetwork.com/journals/jamadermatology/fullarticle... 2.Swetter (2020) .Novel Technologies to Improve Melanoma Detection and Care Focusing on Artificial Intelligence 3.ISIC Workshop 2019 at CVPR (Slides online) 4.https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8074854/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8074854/