5 ms·

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and trea

That's okay, though, right? We're not talking about fully automating diagnosis and treatment. We're talking about using a tool to help doctors diagnose and treat more effectively. i.e. it's more like predictive text input than anything. If it makes it faster for you to diagnose, makes harder to diagnose stuff easier, and recommends the right treatment most of the time then it saves the doctor some time and energy. The only question is whether it does that.

jobigoud 8y ago
The other question is how does it compare to human doctors, they also sometimes recommend unsafe or wrong treatment.
- OscarTheGrinch 8y ago
  The some other other questions: how much does it cost, compared to training a human? Does it actually save any time / resources if it's outputs must always be scrutinized?
  - throwaway5752 8y ago
    That may be a question someone outside of an FDA approval process might asked. But if you have been, you understand the importance of "safety and effectiveness", and you will note there is nothing there about "cost effective" or "marketable".
- setr 8y ago
  One way I tend to view ML is that, when its wrong, its catastrophically wrong. Because it doesn't actually understand anything its doing, and simply looking at a probability model and picking based off it, you end up with the issue that a few carefully selected pixel changes in a picture of a cat takes the model from a cat to an ostrich. The model does not see a significant difference between the cat/ostrich, or cancer/cold, whereas we do; this implies that, when the model is wrong, it is likely to not just provide an incorrect treatment, but a catastrophically incorrect treatment. Where the human sees a cat-like creature, and if not guess a cat, then something similar to a cat (4 legged, furry, etc), the ML model is willing to jump anywhere, in the worst case. So its not just rate of misdiagnosis, but by how much as well.
wolf550e 8y ago
If you genuinely don't know how to spell something (children, ESL) then auto-correct might cause you to accept a correction to the wrong homophone and fool you into thinking it helped you. If the doctor doesn't know how to quickly and cheaply verify the machine's guess, they'll just rubber stamp its recommendations.
- UnhelpfulYoda 8y ago
  Even worse, depending on how well the IBM sales team pawns this hokum off as a legitimate diagnostic tool, the next logical step will be for malpractice insurance companies forcing physicians to consult with Watson before making any decisions.
darpa_escapee 8y ago
It depends on how doctors view IBM's magic black box. If they grow to trust and depend on it, that could become a real problem. People have a tendency to defer, and have a bias towards deferring to machines that seemingly behave as accurately as a calculator would with basic arithmetic. A true crisis can arise if doctors can shift liability by claiming to have just followed Watson's results.
throwaway5752 8y ago
No it's not okay! It increases the probability of (harmful) medical error. The "unsafe" is a big, big gotcha for regulatory approval. As someone correctly pointed out, this is relative to human error rates (can't view article because of registration).
gaius 8y ago
We're talking about using a tool to help doctors diagnose and treat more effectively. A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here.
- ApolloFortyNine 8y ago
  >A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here. Do you realize they would have never developed Watson if there weren't profits involved?
  - gaius 8y ago
    I have no objection to them selling a working technology but they knew Watson was pure snake oil and did it anyway. Same as Theranos.
    - ApolloFortyNine 8y ago
      If people require 100% accuracy out of the gate from any software then we might as well give up now. It's not as if doctors get it right all the time, if you know anyone with medical issues they've likely heard different opinions from different doctors themselves. Anything more complicated then a broken bone seems to get mixed responses about what should be done, or even what the cause of the complication is in the first place.
      - gaius 8y ago
        A technology in this stage of immaturity is fine to trial but IBM should be paying the hospitals not the other way round. They are doing experiments with peoples lives - again just like Theranos.
- jonhendry18 8y ago
  "healthcare systems are strapped for cash" I don't know about that. Most of the new construction in Connecticut seems to be building medical offices for Hartford Healthcare or other medical groups.
wlesieutre 8y ago
Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and will end very badly. The person whose Tesla drove into a concrete barrier at 65 mph earlier this year was perfectly capable of driving a car, but they mistakenly believed that the computer had it under control. 99% Invisible has a pair of podcast episodes on the subject (2015): https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/ https://99percentinvisible.org/episode/children-of-the-magen... https://99percentinvisible.org/episode/johnnycab-automation-paradox-pt-2/ https://99percentinvisible.org/episode/johnnycab-automation-...
- ballenf 8y ago
  Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.
  - FussyZeus 8y ago
    I would argue against this one, simply because of the Airbus incident from some time ago where to avoid pushing feedback into the controls, the plane was designed to average out the inputs from the two pilots. The older, more experienced pilot was attempting to maneuver the plane in such a way to correct their problem (I believe it was a stall) and the younger pilot was panicking and attempting the opposite maneuver, the plane averaged it to roughly no course adjustment, and they hit a mountainside. I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself. (And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)
ajmurmann 8y ago
To me the key is how you use the system. I like the idea of using this not to help but second guess the doctor. The doctor has to make their own diagnosis. If Watson differs a arbiter has to look at it. This obviously optimizes for quality and not cost.
skywhopper 8y ago
If it is poorly trained and recommends incorrect treatment, then no it is a poor tool. If it recommends only in clear cases and then gives the doctor tools to narrow down treatment in more obscure cases that would be good. But current machine learning tech and more importantly marketing does not actually provide that capability. Because the ML software does not work like human decision making. So it cannot really know when it’s making a poor guess and any insight into its decision making process would be useless to the doctors involved. This tech is way too immature for the uses it’s being put to. But the tech companies have started to believe their own hype.
UnhelpfulYoda 8y ago
Not really... I'm mean it's great for IBM financially, but not all that great for Hospitals and Medical Centers duped into this scam: "University of Texas says the project cost MD Anderson more than $62 million" source: MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine https://www.forbes.com/sites/matthewherper/2017/02/19/md-anderson-benches-ibm-watson-in-setback-for-artificial-intelligence-in-medicine/ https://www.forbes.com/sites/matthewherper/2017/02/19/md-and... As for your analogy, you got still tragically flawed predictive text technology for free. Nobody bought iPhones for their crappy predictive text capabilities. Imagine the horror if you actually had to pay for predictive text. As it is, it's a nice to have freebie that came along with your phone, which works.... sometimes.

jobigoud 8y ago

The other question is how does it compare to human doctors, they also sometimes recommend unsafe or wrong treatment.

OscarTheGrinch 8y ago
The some other other questions: how much does it cost, compared to training a human? Does it actually save any time / resources if it's outputs must always be scrutinized?
- throwaway5752 8y ago
  That may be a question someone outside of an FDA approval process might asked. But if you have been, you understand the importance of "safety and effectiveness", and you will note there is nothing there about "cost effective" or "marketable".
setr 8y ago
One way I tend to view ML is that, when its wrong, its catastrophically wrong. Because it doesn't actually understand anything its doing, and simply looking at a probability model and picking based off it, you end up with the issue that a few carefully selected pixel changes in a picture of a cat takes the model from a cat to an ostrich. The model does not see a significant difference between the cat/ostrich, or cancer/cold, whereas we do; this implies that, when the model is wrong, it is likely to not just provide an incorrect treatment, but a catastrophically incorrect treatment. Where the human sees a cat-like creature, and if not guess a cat, then something similar to a cat (4 legged, furry, etc), the ML model is willing to jump anywhere, in the worst case. So its not just rate of misdiagnosis, but by how much as well.

wolf550e 8y ago

If you genuinely don't know how to spell something (children, ESL) then auto-correct might cause you to accept a correction to the wrong homophone and fool you into thinking it helped you. If the doctor doesn't know how to quickly and cheaply verify the machine's guess, they'll just rubber stamp its recommendations.

UnhelpfulYoda 8y ago
Even worse, depending on how well the IBM sales team pawns this hokum off as a legitimate diagnostic tool, the next logical step will be for malpractice insurance companies forcing physicians to consult with Watson before making any decisions.

darpa_escapee 8y ago

It depends on how doctors view IBM's magic black box. If they grow to trust and depend on it, that could become a real problem. People have a tendency to defer, and have a bias towards deferring to machines that seemingly behave as accurately as a calculator would with basic arithmetic. A true crisis can arise if doctors can shift liability by claiming to have just followed Watson's results.

throwaway5752 8y ago

No it's not okay! It increases the probability of (harmful) medical error. The "unsafe" is a big, big gotcha for regulatory approval. As someone correctly pointed out, this is relative to human error rates (can't view article because of registration).

gaius 8y ago

We're talking about using a tool to help doctors diagnose and treat more effectively. A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here.

ApolloFortyNine 8y ago
>A very expensive tool when healthcare systems are strapped for cash. Super unethical of IBM to try and extract profits here. Do you realize they would have never developed Watson if there weren't profits involved?
- gaius 8y ago
  I have no objection to them selling a working technology but they knew Watson was pure snake oil and did it anyway. Same as Theranos.
  - ApolloFortyNine 8y ago
    If people require 100% accuracy out of the gate from any software then we might as well give up now. It's not as if doctors get it right all the time, if you know anyone with medical issues they've likely heard different opinions from different doctors themselves. Anything more complicated then a broken bone seems to get mixed responses about what should be done, or even what the cause of the complication is in the first place.
    - gaius 8y ago
      A technology in this stage of immaturity is fine to trial but IBM should be paying the hospitals not the other way round. They are doing experiments with peoples lives - again just like Theranos.
jonhendry18 8y ago
"healthcare systems are strapped for cash" I don't know about that. Most of the new construction in Connecticut seems to be building medical offices for Hartford Healthcare or other medical groups.

wlesieutre 8y ago

Having systems like autopilot in planes can make pilots worse at flying because they don't spend as much time practicing at the controls and monitoring what's happening. Then when something goes wrong, they risk not paying close enough attention to catch it, or not being practiced well enough to correct it. If you get complacent and just assume the computer knows what it's doing (because it usually does) this can and will end very badly. The person whose Tesla drove into a concrete barrier at 65 mph earlier this year was perfectly capable of driving a car, but they mistakenly believed that the computer had it under control. 99% Invisible has a pair of podcast episodes on the subject (2015): https://99percentinvisible.org/episode/children-of-the-magenta-automation-paradox-pt-1/ https://99percentinvisible.org/episode/children-of-the-magen... https://99percentinvisible.org/episode/johnnycab-automation-paradox-pt-2/ https://99percentinvisible.org/episode/johnnycab-automation-...

ballenf 8y ago
Instead of autopilot, think of this as (or should be) more like fly-by-wire systems in every modern fighter jet. The computer reads the pilot's inputs and translates them to the most effective possible control adjustments, making many more changes than a pilot would be able to make even given the controls. Or maybe the tool should be limited to retrospective evaluation of doctors' decisions. Basically an automated peer review.
- FussyZeus 8y ago
  I would argue against this one, simply because of the Airbus incident from some time ago where to avoid pushing feedback into the controls, the plane was designed to average out the inputs from the two pilots. The older, more experienced pilot was attempting to maneuver the plane in such a way to correct their problem (I believe it was a stall) and the younger pilot was panicking and attempting the opposite maneuver, the plane averaged it to roughly no course adjustment, and they hit a mountainside. I'm all for automating driving and other such tasks once the computers are ready, but until we know they're at least most likely ready to do it, I want the ability to turn it off and do it myself. (And frankly, I want that ability anyway because I genuinely love driving and it makes me sad to think someday I won't be able to do it.)

ajmurmann 8y ago

To me the key is how you use the system. I like the idea of using this not to help but second guess the doctor. The doctor has to make their own diagnosis. If Watson differs a arbiter has to look at it. This obviously optimizes for quality and not cost.

skywhopper 8y ago

If it is poorly trained and recommends incorrect treatment, then no it is a poor tool. If it recommends only in clear cases and then gives the doctor tools to narrow down treatment in more obscure cases that would be good. But current machine learning tech and more importantly marketing does not actually provide that capability. Because the ML software does not work like human decision making. So it cannot really know when it’s making a poor guess and any insight into its decision making process would be useless to the doctors involved. This tech is way too immature for the uses it’s being put to. But the tech companies have started to believe their own hype.

UnhelpfulYoda 8y ago

Not really... I'm mean it's great for IBM financially, but not all that great for Hospitals and Medical Centers duped into this scam: "University of Texas says the project cost MD Anderson more than $62 million" source: MD Anderson Benches IBM Watson In Setback For Artificial Intelligence In Medicine https://www.forbes.com/sites/matthewherper/2017/02/19/md-anderson-benches-ibm-watson-in-setback-for-artificial-intelligence-in-medicine/ https://www.forbes.com/sites/matthewherper/2017/02/19/md-and... As for your analogy, you got still tragically flawed predictive text technology for free. Nobody bought iPhones for their crappy predictive text capabilities. Imagine the horror if you actually had to pay for predictive text. As it is, it's a nice to have freebie that came along with your phone, which works.... sometimes.