3 ms·

One of the main tasks that doctors do is take patient's vague and non-specific problems, build up a rapport with them, understand what is normal and what is not

One of the main tasks that doctors do is take patient's vague and non-specific problems, build up a rapport with them, understand what is normal and what is not, dealing with the "irrelevant" information present at the time, and focus the results into a finite tree of possibilities. In principle, this would be a great task for an ML algorithm. It's all conditional probability. But every such system has failed to do that well -- because the "gut feeling" the doctor develops is funded by a whole host of prior information that an ML algorithm won't be trained on: what is "normal" for an IV-drug addict, or a patient with psychosis; how significant is the "I'm a bit confused" in the middle-aged man who was cycling and came off his bike and hit his head? Do the burns on the soles of this child's feet sit consistently with the story of someone who ran over a fire-pit barbecue "for fun", or is it a non-accidental injury? It's a world of shades of grey where, if you call it the wrong way, ultimately someone could die. Doctors do Bayesian maths, and they do it with priors coming from both their own personal experience as a member of society, and professional training. That is, in my ignorant opinion, the main distinction between what I do -- oft-called "academic medicine" or "academic radiology" -- and clinical medicine. The former looks at populations. The latter looks at individuals. In other words, I don't think it's even possible to codify what the data the ML algorithms should be trained on -- they're culturally specific down to the level of an individual town in some sense; and require looking at huge populations at others.

version_five 5y ago
> In other words, I don't think it's even possible to codify what the data the ML algorithms should be trained on -- they're culturally specific down to the level of an individual town in some sense; and require looking at huge populations at others. This is inciteful. ML is by definition generalizing, and should only be used where it's ok to generalize. There is an implicit assumption in use cases like medical diagnosis that there is a latent representation of the condition that had a much lower dimensionality than the data and that the model is trained in such a way that there are no shortcuts to generalization that miss out on any information that may be important. The second condition is the hardest to meet I believe, because even if a model could take in lots of outside factors, it probably doesn't need to to do really well in training and validation, so it doesn't. The result is models that generalize, as you say, to the population instead of the individual, and end up throwing away vital context to the personal case. I also believe this is an important consideration for many other ML applications. For example those models that predict recidivism rates. I'm sure its possible to build an accurate one, but almost certainly these models stereotypes in the way I mention above, and do not actually take the individual case into account, making them unfair to use on actual people.
tomrod 5y ago
On codification: I actually disagree with this, but only slightly. Imagine if instead of face to face, doctor transactions were via text. The questioning of the doctor can be monitored and patterns in decision trees observed could be codified, and weighed against the healthcare outcome (however defined). What is missing, however, is the Counterfactual Reasoning. The "why" matters. The machine cannot reduce the doctor's choice of decision trees from all possible combinations, only that which it observes the doctor perform. Tail-cases like rare genetic disorders would often be missed.
- medvezhenok 5y ago
  Tail-case like rare genetic disorders are often missed by doctors too. I have several friends who had Lyme disease with fairly serious complications (in the Northeast) (Not that Lyme disease is that rare - it's actually much more common than is expected). Each of them got misdiagnosed for multiple years by multiple different doctors until finally getting the correct diagnosis/treatment. So every system is fallible.
  - deleted 5y ago
    [deleted]
watwut 5y ago
Personally, my exlerience with doctors is neither rapport nor nuanced understanding of my specific situation. That is really not what they do, are trained to do or have time to do.

version_five 5y ago

> In other words, I don't think it's even possible to codify what the data the ML algorithms should be trained on -- they're culturally specific down to the level of an individual town in some sense; and require looking at huge populations at others. This is inciteful. ML is by definition generalizing, and should only be used where it's ok to generalize. There is an implicit assumption in use cases like medical diagnosis that there is a latent representation of the condition that had a much lower dimensionality than the data and that the model is trained in such a way that there are no shortcuts to generalization that miss out on any information that may be important. The second condition is the hardest to meet I believe, because even if a model could take in lots of outside factors, it probably doesn't need to to do really well in training and validation, so it doesn't. The result is models that generalize, as you say, to the population instead of the individual, and end up throwing away vital context to the personal case. I also believe this is an important consideration for many other ML applications. For example those models that predict recidivism rates. I'm sure its possible to build an accurate one, but almost certainly these models stereotypes in the way I mention above, and do not actually take the individual case into account, making them unfair to use on actual people.

tomrod 5y ago

On codification: I actually disagree with this, but only slightly. Imagine if instead of face to face, doctor transactions were via text. The questioning of the doctor can be monitored and patterns in decision trees observed could be codified, and weighed against the healthcare outcome (however defined). What is missing, however, is the Counterfactual Reasoning. The "why" matters. The machine cannot reduce the doctor's choice of decision trees from all possible combinations, only that which it observes the doctor perform. Tail-cases like rare genetic disorders would often be missed.

medvezhenok 5y ago
Tail-case like rare genetic disorders are often missed by doctors too. I have several friends who had Lyme disease with fairly serious complications (in the Northeast) (Not that Lyme disease is that rare - it's actually much more common than is expected). Each of them got misdiagnosed for multiple years by multiple different doctors until finally getting the correct diagnosis/treatment. So every system is fallible.
- deleted 5y ago
  [deleted]

watwut 5y ago

Personally, my exlerience with doctors is neither rapport nor nuanced understanding of my specific situation. That is really not what they do, are trained to do or have time to do.