5 ms·

I looked at a few of these (FDA approved AI products) recently because I was curious as to what the instrument was allowed to do and sure enough it was more of

I looked at a few of these (FDA approved AI products) recently because I was curious as to what the instrument was allowed to do and sure enough it was more of a smart instrument that told you, for instance, how large a volume was. These products don't, at the moment, actually diagnose a condition or recommend a treatment which is interesting. One of the constraints for AI treatment has always been authority and responsibility. To put it another way, if the AI completely messes up and recommends something terrible, whom do you sue? I am hopeful that we (as a society) will sort out these issues with self-driving cars and apply some of those rules to medicine.

superbcarrot 6y ago
It's tough to assign blame and responsibility to systems that are inherently probabilistic. ML algorithms output probabilities so even as the algorithms get incredibly good, unlikely scenarios will inevitably occur in large enough samples. This isn't a problem when the task is to tag your Facebook photos but is a problem when driving a car in the real world.
- OnlyOneCannolo 6y ago
  >It's tough to assign blame and responsibility to systems that are inherently probabilistic. I hear this all the time nowadays, but everything has always been probabilistic. Most of the time, people just don't realize that all problems are fundamentally the same from the perspective of how to approach risk.
  - superbcarrot 6y ago
    I'm not following. How is everything probabilistic (in a meaningful way)? Most other systems don't have built in uncertainty in the same way ML does.
    - OnlyOneCannolo 6y ago
      Physical systems are probabilistic. Environmental conditions are probabilistic. And we've been trusting our safety to physical products operating in the real world for a long time. Whether physical or ML or whatever ... There's always some process that you can't characterize to your liking. So you explore mitigation options until you can make a decision about whether to accept the risk. Standards and regulations enforce that you went through those steps to get a reasonably trustworthy result. Then liability law applies in the usual way. Actuaries and accountants tell you what it's going to cost you, and market research tells you how much it might sell for. Inability to control or understand what you built doesn't absolve you of any fault or responsibility. All it does is change your risk acceptance criteria.
    - TeMPOraL 6y ago
      All our models are inherently probabilistic. The way we perceive and reason about the world is probabilistic, too.
Gatsky 6y ago
I think it is a little wrong-headed to emphasise the litigation aspect. Something that I think people don't appreciate is that as a patient, you want a real person to take responsibility for your problem because that is the basis for a favourable outcome and a satisfying interaction. This applies to every service in the end, whether you are ordering food at a restaurant, trying to get a plumbing problem fixed or being treated for cancer. Any corporation, for-profit or not, generally seeks to obfuscate their responsibilities where possible (one could argue that this is the explicit purpose of a corporation - to be a virtual entity which cannot really be held accountable). Tobacco companies are the best and ongoing examples of this. Creating legal protection for AI such that accountability is somehow removed or diminished is not desirable in my opinion. It will lead inevitably to a situation where healthcare becomes analogous to trying to get technical support for your printer. Contrast this with the what happens currently, where you walk into a clinic or a hospital with a problem and meet a bunch of real people who listen to you. Of course there are massive problems with healthcare, and many people have unpleasant or dangerous experiences, but it can definitely be made worse if we aren't careful.
curiousllama 6y ago
I've done a fair amount of work in the area, and one of the biggest lessons is "even if you can diagnose, you shouldn't" We likely won't have to deal with the legal dilemmas for a while for a simple reason: doctors _hate_ when non-doctors diagnose, and you can't build a diagnostic tool without doctors. The tools simply won't get built to any real quality standard. There's an easier path, interestingly. It's much more effective to just give docs the info they need. A system that could do the following would be the holy grail: "Patient entered ER with [chest pain, shortness of breath], and has a history of [hypertension, diabetes]. They are on [X, Y medication], but have not filled their prescription in [Z weeks]." No diagnosis, just the right info at the right time.
- omarhaneef 6y ago
  The expert system guys in the 80s -- and this is before my time -- came to a similar conclusion, but for the reasons I am laying out (responsibility). They also decided it was better to have a computer make a suggestion to a doctor than to give it unvarnished to a patience, sort of assisting the doctor. I won't disagree with experts (yourself, or them), but there is the issue of scaling it. Consider a poor person in a poor country for whom the alternative is not a delayed appointment with a real person but no advice at all.

superbcarrot 6y ago

It's tough to assign blame and responsibility to systems that are inherently probabilistic. ML algorithms output probabilities so even as the algorithms get incredibly good, unlikely scenarios will inevitably occur in large enough samples. This isn't a problem when the task is to tag your Facebook photos but is a problem when driving a car in the real world.

OnlyOneCannolo 6y ago
>It's tough to assign blame and responsibility to systems that are inherently probabilistic. I hear this all the time nowadays, but everything has always been probabilistic. Most of the time, people just don't realize that all problems are fundamentally the same from the perspective of how to approach risk.
- superbcarrot 6y ago
  I'm not following. How is everything probabilistic (in a meaningful way)? Most other systems don't have built in uncertainty in the same way ML does.
  - OnlyOneCannolo 6y ago
    Physical systems are probabilistic. Environmental conditions are probabilistic. And we've been trusting our safety to physical products operating in the real world for a long time. Whether physical or ML or whatever ... There's always some process that you can't characterize to your liking. So you explore mitigation options until you can make a decision about whether to accept the risk. Standards and regulations enforce that you went through those steps to get a reasonably trustworthy result. Then liability law applies in the usual way. Actuaries and accountants tell you what it's going to cost you, and market research tells you how much it might sell for. Inability to control or understand what you built doesn't absolve you of any fault or responsibility. All it does is change your risk acceptance criteria.
  - TeMPOraL 6y ago
    All our models are inherently probabilistic. The way we perceive and reason about the world is probabilistic, too.

Gatsky 6y ago

I think it is a little wrong-headed to emphasise the litigation aspect. Something that I think people don't appreciate is that as a patient, you want a real person to take responsibility for your problem because that is the basis for a favourable outcome and a satisfying interaction. This applies to every service in the end, whether you are ordering food at a restaurant, trying to get a plumbing problem fixed or being treated for cancer. Any corporation, for-profit or not, generally seeks to obfuscate their responsibilities where possible (one could argue that this is the explicit purpose of a corporation - to be a virtual entity which cannot really be held accountable). Tobacco companies are the best and ongoing examples of this. Creating legal protection for AI such that accountability is somehow removed or diminished is not desirable in my opinion. It will lead inevitably to a situation where healthcare becomes analogous to trying to get technical support for your printer. Contrast this with the what happens currently, where you walk into a clinic or a hospital with a problem and meet a bunch of real people who listen to you. Of course there are massive problems with healthcare, and many people have unpleasant or dangerous experiences, but it can definitely be made worse if we aren't careful.

curiousllama 6y ago

I've done a fair amount of work in the area, and one of the biggest lessons is "even if you can diagnose, you shouldn't" We likely won't have to deal with the legal dilemmas for a while for a simple reason: doctors _hate_ when non-doctors diagnose, and you can't build a diagnostic tool without doctors. The tools simply won't get built to any real quality standard. There's an easier path, interestingly. It's much more effective to just give docs the info they need. A system that could do the following would be the holy grail: "Patient entered ER with [chest pain, shortness of breath], and has a history of [hypertension, diabetes]. They are on [X, Y medication], but have not filled their prescription in [Z weeks]." No diagnosis, just the right info at the right time.

omarhaneef 6y ago
The expert system guys in the 80s -- and this is before my time -- came to a similar conclusion, but for the reasons I am laying out (responsibility). They also decided it was better to have a computer make a suggestion to a doctor than to give it unvarnished to a patience, sort of assisting the doctor. I won't disagree with experts (yourself, or them), but there is the issue of scaling it. Consider a poor person in a poor country for whom the alternative is not a delayed appointment with a real person but no advice at all.