4 ms·

Are you aware of any attempts to use machine learning in medical analysis and outcome prediction? I feel like this is one of the few applications where it could

Are you aware of any attempts to use machine learning in medical analysis and outcome prediction? I feel like this is one of the few applications where it could shine. I have no formal training in data science, but everything I've read so far seems to indicate that noisy and unreliable data is not an insurmountable problem. All of this talk about how "hard" the statistical analysis is, is strange to me. Maybe "advanced" would be a better term? If you get a patient with a contradictory medical history that somehow also contradicts what they are telling you, simply adjust your expected chance of success appropriately (to zero perhaps). In that extreme case, if you get a good outcome, congrats you got lucky. If you don't, it should have 0 impact on how you are evaluated as a doctor.

rscho 5y ago
Statistics are not the problem. The data is. 'Big healthcare data' does not exist. Building what you are thinking of would require huge data gathering capabilities, that are very clearly out of reach. Most big medical co do a lot of data science (Kaiser and others). Very efficient from a managerial pov. Totally useless, medically speaking.
asmithmd1 5y ago
> I feel like this is one of the few applications where it could shine. Read “The Alignment Problem”, a very good just above pop sci level book about machine learning. They have one example where ML determined seniors with COPD were at reduced risk from pneumonia, and obviously non-sensical result. Patients with COPD wound up in the hospital at a lower rate than average because doctors know they need careful attention right away.
sxg 5y ago
Before you can even do the statistical analysis you suggest, you need large amounts of high quality data—which we don't have. One place where the US (and the world?) gets data privacy right is in healthcare, but unfortunately that also means it's nearly impossible to create the data sets we need to do the statistical analysis you want. Institutions face severe penalties for wrongfully sharing patient data, so most opt to just not share any data. Any research that is performed is done internally on local populations with de-identified data sets. A few brave institutions go well out of their way to create and share de-identified data sets publically, but these data sets still undersample the general population. This is a critical problem because certain diseases are highly prevalent in certain regions (e.g., Lyme disease in New England) but unheard of in other regions (e.g., Lyme disease in Colorado). If your ML model is trained on data largely from New England, it's going to diagnose a patient with the classic "target-shaped" rash with Lyme disease even if the patient is from Colorado (high false positive rate). If the model is trained on data from Colorado, it will underdiagnose Lyme disease in patients from New England (high false negative rate). The only way I know to overcome this problem is to create even larger data sets, but this just isn't possible with data privacy laws.
- grog454 5y ago
  > If the model is trained on data from Colorado, it will underdiagnose Lyme disease in patients from New England (high false negative rate). The only way I know to overcome this problem is to create even larger data sets, but this just isn't possible with data privacy laws. My understanding is that high dimensionality in the domain is exactly where ML excels (so add location as an input), and that is exactly what a medical diagnosis involves. Perhaps the legislation will get there one day.
- clort 5y ago
  Re data privacy, as of last summer in the UK we had to opt out of data sharing of our NHS records with third parties. I am conflicted about this because as you suggested, large amounts of data could be beneficial for things like statistical analysis and machine learning to help medical research. On the other hand, what we know about human nature seems to indicate that there will already be sociopaths slavering over this data in order to enrich themselves at the expense of everybody else. https://inews.co.uk/opinion/nhs-data-shared-third-parties-weeks-opt-out-gps-worried-1028042 https://inews.co.uk/opinion/nhs-data-shared-third-parties-we... Anyway, its out there now.. so watch this space, I guess!
chefandy 5y ago
Yeah: https://www.nature.com/articles/d41586-019-03228-6 https://www.nature.com/articles/d41586-019-03228-6 I’m reluctantly pessimistic about humanity’s near-term capability to appropriately weight input from technology as imperfect but inscrutable as current ml.
Fomite 5y ago
I am. In that I have several CDC funded projects using machine learning in medical analysis and outcome prediction. Even on extremely well curated data sets this is a fairly hard problem.
the_d3f4ult 5y ago
Everyone and their mother is working on ML applications in medicine.

rscho 5y ago

Statistics are not the problem. The data is. 'Big healthcare data' does not exist. Building what you are thinking of would require huge data gathering capabilities, that are very clearly out of reach. Most big medical co do a lot of data science (Kaiser and others). Very efficient from a managerial pov. Totally useless, medically speaking.

asmithmd1 5y ago

> I feel like this is one of the few applications where it could shine. Read “The Alignment Problem”, a very good just above pop sci level book about machine learning. They have one example where ML determined seniors with COPD were at reduced risk from pneumonia, and obviously non-sensical result. Patients with COPD wound up in the hospital at a lower rate than average because doctors know they need careful attention right away.

sxg 5y ago

Before you can even do the statistical analysis you suggest, you need large amounts of high quality data—which we don't have. One place where the US (and the world?) gets data privacy right is in healthcare, but unfortunately that also means it's nearly impossible to create the data sets we need to do the statistical analysis you want. Institutions face severe penalties for wrongfully sharing patient data, so most opt to just not share any data. Any research that is performed is done internally on local populations with de-identified data sets. A few brave institutions go well out of their way to create and share de-identified data sets publically, but these data sets still undersample the general population. This is a critical problem because certain diseases are highly prevalent in certain regions (e.g., Lyme disease in New England) but unheard of in other regions (e.g., Lyme disease in Colorado). If your ML model is trained on data largely from New England, it's going to diagnose a patient with the classic "target-shaped" rash with Lyme disease even if the patient is from Colorado (high false positive rate). If the model is trained on data from Colorado, it will underdiagnose Lyme disease in patients from New England (high false negative rate). The only way I know to overcome this problem is to create even larger data sets, but this just isn't possible with data privacy laws.

grog454 5y ago
> If the model is trained on data from Colorado, it will underdiagnose Lyme disease in patients from New England (high false negative rate). The only way I know to overcome this problem is to create even larger data sets, but this just isn't possible with data privacy laws. My understanding is that high dimensionality in the domain is exactly where ML excels (so add location as an input), and that is exactly what a medical diagnosis involves. Perhaps the legislation will get there one day.
clort 5y ago
Re data privacy, as of last summer in the UK we had to opt out of data sharing of our NHS records with third parties. I am conflicted about this because as you suggested, large amounts of data could be beneficial for things like statistical analysis and machine learning to help medical research. On the other hand, what we know about human nature seems to indicate that there will already be sociopaths slavering over this data in order to enrich themselves at the expense of everybody else. https://inews.co.uk/opinion/nhs-data-shared-third-parties-weeks-opt-out-gps-worried-1028042 https://inews.co.uk/opinion/nhs-data-shared-third-parties-we... Anyway, its out there now.. so watch this space, I guess!

chefandy 5y ago

Yeah: https://www.nature.com/articles/d41586-019-03228-6 https://www.nature.com/articles/d41586-019-03228-6 I’m reluctantly pessimistic about humanity’s near-term capability to appropriately weight input from technology as imperfect but inscrutable as current ml.

Fomite 5y ago

I am. In that I have several CDC funded projects using machine learning in medical analysis and outcome prediction. Even on extremely well curated data sets this is a fairly hard problem.

the_d3f4ult 5y ago

Everyone and their mother is working on ML applications in medicine.