4 ms·

Fantastic. An additional way by which someone looking for help or support from a human can be denied that help and have their needs offloaded to a technology th

by lumb63 3y ago

Fantastic. An additional way by which someone looking for help or support from a human can be denied that help and have their needs offloaded to a technology that probably can’t help them. Here’s how I see this playing out: practices start to use LLMs to answer patient queries to save money. People realize their queries aren’t actually being answered fully (or at all?) and stop contacting their doctors at all. Practices save money and patients get worse outcomes. Exactly as we’ve seen in customer service over the years - talking to a human is nearly impossible. An alternate possibility is the LLM gives such bad advice that it causes irreparable harm to a patient and nobody can be held liable because the doctor didn’t issue the advice and neither did the company which produced the LLM. The universe where an LLM used in this situation does anything positive for the patient is vanishingly small. But I’m sure providers and practices will love it, since it effectively will allow them to not do part of their job and increase margins respectively.

sandspar 3y ago
Why is there always this kneejerk "bad!" reaction to AI? You don't think there's any conceivable way that this may actually help people? You don't think that Stanford Medicine may know a bit more about the problem space than you do?
- SoftTalker 3y ago
  I think it's conceivable but if the goal was to help people it would already have been done: address the systemic roadblocks that prevent more people from studying medicine, increase the supply of doctors and clinicians so that people don't have to wait months for a routine appointment.
  - jdlshore 3y ago
    That's a false dilemma. We can do both. Some people are addressing roadblocks (or attempting to; politics are hard), and some people are working on AI-assisted support. I'm as skeptical of GenAI as they come, but let's not pretend that the people working on GenAI customer support are the same people who should be working with the AMA to address the artificial constraints in medical education.
    - bluefirebrand 3y ago
      > That's a false dilemma. We can do both. The thing I notice about every argument where I see "we can do both", is that in reality we never actually do both That's why people get so adamant about choosing their way instead of the other way. There's always two options, we could do both, but we never, ever seem to
      - jdlshore 3y ago
        Who’s this “we,” kemo sabe? Let’s summarize the situation here. A group of people have chosen to work on GenAI customer support. You’re criticizing them for not working on a difficult political problem instead. Meanwhile, what are you doing? Which of the political activist groups working on the problem you care about so much have you joined? How have you offered your support? If this issue is so important to you that you think “we” should be devoting our full attention to it, surely you’re out front leading the charge, right?
- constantcrying 3y ago
  There is by far enough enthusiasm for AI already. >You don't think there's any conceivable way that this may actually help people? I don't. I believe that exactly as OP described people will notice they are talking to some AI and just give up, as they correctly understand that the AI does not have the abilities and understanding that might actually help them.
- gpm 3y ago
  Because the vast majority of commercial deployments of "AI" (llms) are extremely unhelpful chat bots that just waste customers time to try and avoid having to have a human talk to them.
  - choilive 3y ago
    And the vast majority of commercial deployments of real humans to solve the same problem are extremely unhelpful that just waste customers time. Whats your point?
  - wruza 3y ago
    Decent LLMs are < 1 year old. How much commercial deployment they’ve seen?
    - unleaded 3y ago
      Too much
      - wruza 3y ago
        Too much where? Let's ignore AI "startups" with no business idea beyond reselling system prompts for GPT-4 to end users. Does your bank have AI like last summer? A clinic? Any utility service? It is unlikely for most established businesses, cause they cannot move that fast. I just did a quick search on "{bank,hospital,...} integrated gpt-4 for client support" and similar queries and there's various "will be", "may use", but nothing specific. I believe ggp is mixing regular chat bots with integration of SotA in their statement.
        gpm 3y ago
        Off the top of my head: Air Canada, shopify, ikea, car dealerships, ... It's common enough I'm sure I've seen more than that, those are just the ones I remember because they've either personally annoyed me or been in the news. Health things around here have legal restrictions that definitely prevent them from using GPT, thankfully.
- lostlogin 3y ago
  > Why is there always this kneejerk "bad!" reaction to AI? I don’t interpret the OP as slamming AI. It’s criticism of the way in which companies find ways to save a few dollars.
- olliej 3y ago
  Because “AI” or LLMs have no understanding of what they’re producing, and every single attempt to use them in an environment where basic factual correctness is involved has confirmed this? Or the fact that the entire sales pitch is “this will be cheaper than actual people who know what they’re doing”, maybe a side claim of “outperforms people on our training set” coupled with “we accept no liability for inaccurate advice or poor outcomes”. The sole purpose of programs like these is to save money. It has nothing to do with helping anyone. The goal is to sound like your system is providing the service people paid for, without actually spending the money to provide that service. I will believe a company will is doing the right thing when the developers personally accept the criminal and legal liability that an actual doctor or nurse providing the service is subject to. But then to add insult to injury this just further increases the wait time for people by forcing them to deal with the “AI” before they get through to the further reduced pool of actual specialists and correspondingly increased wait times there as well. This is not rocket science to understand, and literally nothing I’ve said here is new - not the clear prioritization of cost cutting, the overselling of “AI”, and the generally unsafe use of randomized statistical models to provide factual information. My question is why you think that this time is different vs literally every other example?
- tbalsam 3y ago
  This is because this is what the tendency has been like in practice for many years. While the possibilities are good, living in the world of what-if for ML is very similar to the world of what-if for blockchain. It is very promising but unfortunately the past trend with reality at both a surface and deeper dynamics level does not seem to agree with it being a solid motif long-term. This is why people are pessimistic about ML/AI in people-facing positions. It has happened with automation in the past, it is _currently happening_ with ML/AI systems in customer-facing applications, and the momentum of the field is headed in that direction. To me, it would be very silly to assume that the field would make a sudden 180 from decades of momentum in a rather established direction, with no established reason other than the potential for it to be good. This is why people tend to generally be upset about it, is my understanding. It's a good tool, and it is oftentimes not used well.
- rsynnott 3y ago
  Given the long, unfortunate history of computerisation of medicine, I don’t think there should be a presumption that they necessarily know what they’re doing. If this is a disaster, it won’t be the first, and some previous medical-technical disasters have involved human factors; the operator either becoming too trusting of the machine, or too casual with its use.
jncfhnb 3y ago
Is it just me or has customer service never been easier, with real humans easily accessible for most major corporations
- pbourke 3y ago
  Real humans who are unable to resolve your issue are quite accessible indeed.
  - jncfhnb 3y ago
    Like… who? I find that large banks, telecom, and what not have largely shifted their call centers to either America or the Philippines and you now get much more proficient customer service than, say, 10 years ago when everything was going to India. Chat based humans are also much more efficient imo.
justrealist 3y ago
I don't think you understand how stupid and googleable most customer support questions are. Doctors will have more time for real questions if they aren't answering idiotic questions. You are living in a bubble.
- SoftTalker 3y ago
  Doctors would also have more time for real questions if they told fat people to exercise and quit eating so many pop-tarts instead of giving them shots with a host of risky side-effects that then have to be managed.
  - justrealist 3y ago
    Are you really trying to blame obesity on doctors? Impressive.
    - deleted 3y ago
      [deleted]
    - lupusreal 3y ago
      [flagged]
      - ceejayoz 3y ago
        No, they’re ignoring the medical fact that “tell people to lose weight” has a basically zero percent success rate. The only intervention known to work reliably was bariatric surgery until stuff like Ozempic showed up recently.
        lupusreal 3y ago
        Cutting losses to spend time with other patients isn't ignoring that little medical fact. It's an acknowledgement of it; all the effort spent treating the symptoms of obesity is basically wasted, squandering the resources (time particularly) of the medical system.
userbinator 3y ago
An alternate possibility is the LLM gives such bad advice that it causes irreparable harm to a patient and nobody can be held liable because the doctor didn’t issue the advice and neither did the company which produced the LLM. I think someone will be liable; this has already been tested in at least one court with something less life-critical: https://news.ycombinator.com/item?id=39378235 https://news.ycombinator.com/item?id=39378235
exe34 3y ago
The doctor who signs off will definitely be liable. Unfortunately we live in a society where somebody both has to die and have litigious surviving relatives to make sure the problem becomes actionable.

by lumb63 3y ago

sandspar 3y ago

Why is there always this kneejerk "bad!" reaction to AI? You don't think there's any conceivable way that this may actually help people? You don't think that Stanford Medicine may know a bit more about the problem space than you do?

SoftTalker 3y ago
I think it's conceivable but if the goal was to help people it would already have been done: address the systemic roadblocks that prevent more people from studying medicine, increase the supply of doctors and clinicians so that people don't have to wait months for a routine appointment.
- jdlshore 3y ago
  That's a false dilemma. We can do both. Some people are addressing roadblocks (or attempting to; politics are hard), and some people are working on AI-assisted support. I'm as skeptical of GenAI as they come, but let's not pretend that the people working on GenAI customer support are the same people who should be working with the AMA to address the artificial constraints in medical education.
  - bluefirebrand 3y ago
    > That's a false dilemma. We can do both. The thing I notice about every argument where I see "we can do both", is that in reality we never actually do both That's why people get so adamant about choosing their way instead of the other way. There's always two options, we could do both, but we never, ever seem to
    - jdlshore 3y ago
      Who’s this “we,” kemo sabe? Let’s summarize the situation here. A group of people have chosen to work on GenAI customer support. You’re criticizing them for not working on a difficult political problem instead. Meanwhile, what are you doing? Which of the political activist groups working on the problem you care about so much have you joined? How have you offered your support? If this issue is so important to you that you think “we” should be devoting our full attention to it, surely you’re out front leading the charge, right?
constantcrying 3y ago
There is by far enough enthusiasm for AI already. >You don't think there's any conceivable way that this may actually help people? I don't. I believe that exactly as OP described people will notice they are talking to some AI and just give up, as they correctly understand that the AI does not have the abilities and understanding that might actually help them.
gpm 3y ago
Because the vast majority of commercial deployments of "AI" (llms) are extremely unhelpful chat bots that just waste customers time to try and avoid having to have a human talk to them.
- choilive 3y ago
  And the vast majority of commercial deployments of real humans to solve the same problem are extremely unhelpful that just waste customers time. Whats your point?
- wruza 3y ago
  Decent LLMs are < 1 year old. How much commercial deployment they’ve seen?
  - unleaded 3y ago
    Too much
    - wruza 3y ago
      Too much where? Let's ignore AI "startups" with no business idea beyond reselling system prompts for GPT-4 to end users. Does your bank have AI like last summer? A clinic? Any utility service? It is unlikely for most established businesses, cause they cannot move that fast. I just did a quick search on "{bank,hospital,...} integrated gpt-4 for client support" and similar queries and there's various "will be", "may use", but nothing specific. I believe ggp is mixing regular chat bots with integration of SotA in their statement.
      - gpm 3y ago
        Off the top of my head: Air Canada, shopify, ikea, car dealerships, ... It's common enough I'm sure I've seen more than that, those are just the ones I remember because they've either personally annoyed me or been in the news. Health things around here have legal restrictions that definitely prevent them from using GPT, thankfully.
lostlogin 3y ago
> Why is there always this kneejerk "bad!" reaction to AI? I don’t interpret the OP as slamming AI. It’s criticism of the way in which companies find ways to save a few dollars.
olliej 3y ago
Because “AI” or LLMs have no understanding of what they’re producing, and every single attempt to use them in an environment where basic factual correctness is involved has confirmed this? Or the fact that the entire sales pitch is “this will be cheaper than actual people who know what they’re doing”, maybe a side claim of “outperforms people on our training set” coupled with “we accept no liability for inaccurate advice or poor outcomes”. The sole purpose of programs like these is to save money. It has nothing to do with helping anyone. The goal is to sound like your system is providing the service people paid for, without actually spending the money to provide that service. I will believe a company will is doing the right thing when the developers personally accept the criminal and legal liability that an actual doctor or nurse providing the service is subject to. But then to add insult to injury this just further increases the wait time for people by forcing them to deal with the “AI” before they get through to the further reduced pool of actual specialists and correspondingly increased wait times there as well. This is not rocket science to understand, and literally nothing I’ve said here is new - not the clear prioritization of cost cutting, the overselling of “AI”, and the generally unsafe use of randomized statistical models to provide factual information. My question is why you think that this time is different vs literally every other example?
tbalsam 3y ago
This is because this is what the tendency has been like in practice for many years. While the possibilities are good, living in the world of what-if for ML is very similar to the world of what-if for blockchain. It is very promising but unfortunately the past trend with reality at both a surface and deeper dynamics level does not seem to agree with it being a solid motif long-term. This is why people are pessimistic about ML/AI in people-facing positions. It has happened with automation in the past, it is _currently happening_ with ML/AI systems in customer-facing applications, and the momentum of the field is headed in that direction. To me, it would be very silly to assume that the field would make a sudden 180 from decades of momentum in a rather established direction, with no established reason other than the potential for it to be good. This is why people tend to generally be upset about it, is my understanding. It's a good tool, and it is oftentimes not used well.
rsynnott 3y ago
Given the long, unfortunate history of computerisation of medicine, I don’t think there should be a presumption that they necessarily know what they’re doing. If this is a disaster, it won’t be the first, and some previous medical-technical disasters have involved human factors; the operator either becoming too trusting of the machine, or too casual with its use.

jncfhnb 3y ago

Is it just me or has customer service never been easier, with real humans easily accessible for most major corporations

pbourke 3y ago
Real humans who are unable to resolve your issue are quite accessible indeed.
- jncfhnb 3y ago
  Like… who? I find that large banks, telecom, and what not have largely shifted their call centers to either America or the Philippines and you now get much more proficient customer service than, say, 10 years ago when everything was going to India. Chat based humans are also much more efficient imo.

justrealist 3y ago

I don't think you understand how stupid and googleable most customer support questions are. Doctors will have more time for real questions if they aren't answering idiotic questions. You are living in a bubble.

SoftTalker 3y ago
Doctors would also have more time for real questions if they told fat people to exercise and quit eating so many pop-tarts instead of giving them shots with a host of risky side-effects that then have to be managed.
- justrealist 3y ago
  Are you really trying to blame obesity on doctors? Impressive.
  - deleted 3y ago
    [deleted]
  - lupusreal 3y ago
    [flagged]
    - ceejayoz 3y ago
      No, they’re ignoring the medical fact that “tell people to lose weight” has a basically zero percent success rate. The only intervention known to work reliably was bariatric surgery until stuff like Ozempic showed up recently.
      - lupusreal 3y ago
        Cutting losses to spend time with other patients isn't ignoring that little medical fact. It's an acknowledgement of it; all the effort spent treating the symptoms of obesity is basically wasted, squandering the resources (time particularly) of the medical system.

userbinator 3y ago

An alternate possibility is the LLM gives such bad advice that it causes irreparable harm to a patient and nobody can be held liable because the doctor didn’t issue the advice and neither did the company which produced the LLM. I think someone will be liable; this has already been tested in at least one court with something less life-critical: https://news.ycombinator.com/item?id=39378235 https://news.ycombinator.com/item?id=39378235

exe34 3y ago

The doctor who signs off will definitely be liable. Unfortunately we live in a society where somebody both has to die and have litigious surviving relatives to make sure the problem becomes actionable.