7 ms·

I think you're being far to mild: this absolutely is letting an LLM give people not just medical advice, but medical treatment This is absurdly dangerous, and

I think you're being far to mild: this absolutely is letting an LLM give people not just medical advice, but medical treatment This is absurdly dangerous, and I"m continually astounded by the lack of push-back, especially in this community. This community understands what LLMs are much better than the general public. Same thing with the post a while back of a ycombinator startup that used AI to generate survey data. It's plain what it is, but there's shockingly little push-back.

syntaxing 3y ago
At least it’s something. The alternative today in the US is nothing. It’s easy for us to discuss the morality of this when we have a cushy wage and cushy benefits. A lot of Americans cannot afford the deductible or even have insurance. While the obvious “least dangerous” and most moral solution is universal healthcare but that’s not happening anytime soon for anyone younger than 65. There’s little push back probably because most people here understand this. Something while dangerous is better than nothing.
- lemper 3y ago
  I can't wait to read a dude who put a noose on himself because of advice from this model on the news.
  - 1024core 3y ago
    If you can convince such a model to suggest such a course of action, please blog about it.
  - samueldyson10 3y ago
    Oh, please don't be so pessimistic. Just like old generation used to say Google and Facebook will kill a lot of people. Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
  - robotnikman 3y ago
    This has already happened unfortunately https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
    - lemper 3y ago
      good lord in heaven. my expectations were low but this one takes the cake.
- ddingus 3y ago
  That argument isn't good enough. And I agree with you it really sucks in the usa. And maybe we all need to be talking to everyone else about fixing that. I'm down for that conversation anytime any day. It is profoundly irresponsible to basically encourage people to self-diagnose. It's going to kill a number of them too. That's no joke.
  - bigyikes 3y ago
    How about the opportunity cost of people dying because they don’t have access to any treatment? It can be very challenging to find a qualified professional, even if you have the necessary funds, insurance, and determination. LLM mental health treatment can’t come soon enough.
    - ddingus 3y ago
      Yes, it is ugly! The thing is bad actors are essentially forcing arguably unethical tools onto people in need. Our current Healthcare struggle is completely unnecessary. A good percentage of our economic struggles are equally unnecessary. The only reason we are even having these discussions is too many people with means are not employing those means in ways that make it worth allowing said means at all. Greed, psychopaths, ignorance and more are the problem. People are saying: Basic health care can't come soon enough. Drug price reform can't... Mental health You get the idea. I get it. Lots of us are desperate. I lost a home over this health care shit. Basically traded more time with my wife for our home. Brutal as fuck. I currently pay close to $2k in premiums and have difficulty finding options because I fall into what Franken called the "doughnut hole." Have considered some sort of crime, or leaving the country many times too. Like I said, ugly.
    - bugglebeetle 3y ago
      > LLM mental health treatment can’t come soon enough. If you think LLMs perform at a level that they can help resolve the mental health crisis in the US, you are very much a part of that problem. They can’t even always produce functional code without heavy steering, let alone negotiate the fraught domain of psychological and psychiatric health. Foisting people in crisis off on unpredictable text generation engines is IMO a textbook example of the fundamentally anti-social character of American society that produces so many of the mental health problems that this country faces.
- jasonhansel 3y ago
  This could well be much worse than nothing.
  - panagathon 3y ago
    If your leg gets blown off, and the only help you're getting is from Gary, who claims a strong stomach and has used a needle kit before, for sewing closed your femoral artery, I reckon you're taking a swing. Are you sure the desperate don't deserve this attempt to help them? I can't imagine what it must be like to have such issues and have zero professional options to turn to, and you know what the solution for many is.
    - jasonhansel 3y ago
      A bad therapist could be substantially worse than no treatment for mental health issues, particularly if people see it as more authoritative than it is.
      - kgfiohgon 3y ago
        100% It took me far too long to realize that the first psychiatrist I saw was pushing me in the opposite direction that I wanted my life to go. I had no context and blindly trusted her as an expert. Other therapists have since told me that what I was told my first psychiatrist was unhelpful, unethical, and even malpractice. No person (other than myself) has harmed my life as much as that psychiatrist. And I got to pay a small fortune for the terrible experience.
    - squeaky-clean 3y ago
      What if your leg is just sore, and Gary swears it's actually been blown off and he needs to operate immediately?
    - haldujai 3y ago
      Except if your leg is blown off every hospital in the US is legally required to treat you whether or not you have insurance or even if you’re homeless, so Gary is not your only option.
2devnull 3y ago
If you’re shocked by lack of push-back, you are in store for a lot more shocks going forward. People that build things and people that gatekeep, regulate, etc… are mostly disjoint sets. They are the two sides to upcoming war. Progress vs safety. The world as a whole is tilting towards progress right now, we’ve seen the limitations to safety.
- Gerardo1 3y ago
  I meant more in this community specifically, and, after the AI tool meant to fabricate survey results, YCombinator generally.
slashdev 3y ago
I see your “first do no harm” but I’ll counter that it is probably better than the alternative, which is to do nothing. And it may lead to people seeking professional help. It’s something for the unfilled gap in between, it’s not a replacement for professional help.
ancorevard 3y ago
Problem is that practitioners of mental health treatment themselves cause much damage today. This will not change that. Perhaps we're not too far from an LLM causing less damage than the _average_ mental health practitioner – with far more availability not to mention affordability.
- toomuchtodo 3y ago
  That's arguably for regulators to decide though, not technologists who have no exposure to the harm they cause. I can't even say "engineers" because engineers are held to a standard of care [1] that people slinging code are not. The fact that medical practitioners are sometimes wrong is not a legitimate means to wave away AI safety. In this instance, the solution isn't immature tech (that may [may!] one day be good tech), it's policy that enables healthcare access. You aren't going to fix that with LLMs. But, you know, that's what tech people keep doing. Poor tech solutions to people problems, and sometimes getting wealthy due to luck along the way. [1] https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd575265 https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd5752...
srackey 3y ago
This is in fact not “absurdly dangerous”. It may actually be safer than using a biological intelligence for the same purpose. Given that, and the shortage of biological mental health workers, it seems reasonable that many many more would suffer negative outcomes were they not allowed to use this tech. As such, I’d argue that your comment is absurdly dangerous rhetoric, and your license to post on this web forum should be revoked.
WanderPanda 3y ago
A good LLMs answers should on average still be better than what an average person would come up with People give bad advice to other people all the time
- Gerardo1 3y ago
  I'm not asking your average person for therapy or to help diagnose or work through my mental health issues.

syntaxing 3y ago

At least it’s something. The alternative today in the US is nothing. It’s easy for us to discuss the morality of this when we have a cushy wage and cushy benefits. A lot of Americans cannot afford the deductible or even have insurance. While the obvious “least dangerous” and most moral solution is universal healthcare but that’s not happening anytime soon for anyone younger than 65. There’s little push back probably because most people here understand this. Something while dangerous is better than nothing.

lemper 3y ago
I can't wait to read a dude who put a noose on himself because of advice from this model on the news.
- 1024core 3y ago
  If you can convince such a model to suggest such a course of action, please blog about it.
- samueldyson10 3y ago
  Oh, please don't be so pessimistic. Just like old generation used to say Google and Facebook will kill a lot of people. Do you think Google Search suggestion for any medical-related are all true? But, people still search for their problems and self-diagnose. How many of them suffered for that?
- robotnikman 3y ago
  This has already happened unfortunately https://www.euronews.com/next/2023/03/31/man-ends-his-life-after-an-ai-chatbot-encouraged-him-to-sacrifice-himself-to-stop-climate- https://www.euronews.com/next/2023/03/31/man-ends-his-life-a...
  - lemper 3y ago
    good lord in heaven. my expectations were low but this one takes the cake.
ddingus 3y ago
That argument isn't good enough. And I agree with you it really sucks in the usa. And maybe we all need to be talking to everyone else about fixing that. I'm down for that conversation anytime any day. It is profoundly irresponsible to basically encourage people to self-diagnose. It's going to kill a number of them too. That's no joke.
- bigyikes 3y ago
  How about the opportunity cost of people dying because they don’t have access to any treatment? It can be very challenging to find a qualified professional, even if you have the necessary funds, insurance, and determination. LLM mental health treatment can’t come soon enough.
  - ddingus 3y ago
    Yes, it is ugly! The thing is bad actors are essentially forcing arguably unethical tools onto people in need. Our current Healthcare struggle is completely unnecessary. A good percentage of our economic struggles are equally unnecessary. The only reason we are even having these discussions is too many people with means are not employing those means in ways that make it worth allowing said means at all. Greed, psychopaths, ignorance and more are the problem. People are saying: Basic health care can't come soon enough. Drug price reform can't... Mental health You get the idea. I get it. Lots of us are desperate. I lost a home over this health care shit. Basically traded more time with my wife for our home. Brutal as fuck. I currently pay close to $2k in premiums and have difficulty finding options because I fall into what Franken called the "doughnut hole." Have considered some sort of crime, or leaving the country many times too. Like I said, ugly.
  - bugglebeetle 3y ago
    > LLM mental health treatment can’t come soon enough. If you think LLMs perform at a level that they can help resolve the mental health crisis in the US, you are very much a part of that problem. They can’t even always produce functional code without heavy steering, let alone negotiate the fraught domain of psychological and psychiatric health. Foisting people in crisis off on unpredictable text generation engines is IMO a textbook example of the fundamentally anti-social character of American society that produces so many of the mental health problems that this country faces.
jasonhansel 3y ago
This could well be much worse than nothing.
- panagathon 3y ago
  If your leg gets blown off, and the only help you're getting is from Gary, who claims a strong stomach and has used a needle kit before, for sewing closed your femoral artery, I reckon you're taking a swing. Are you sure the desperate don't deserve this attempt to help them? I can't imagine what it must be like to have such issues and have zero professional options to turn to, and you know what the solution for many is.
  - jasonhansel 3y ago
    A bad therapist could be substantially worse than no treatment for mental health issues, particularly if people see it as more authoritative than it is.
    - kgfiohgon 3y ago
      100% It took me far too long to realize that the first psychiatrist I saw was pushing me in the opposite direction that I wanted my life to go. I had no context and blindly trusted her as an expert. Other therapists have since told me that what I was told my first psychiatrist was unhelpful, unethical, and even malpractice. No person (other than myself) has harmed my life as much as that psychiatrist. And I got to pay a small fortune for the terrible experience.
  - squeaky-clean 3y ago
    What if your leg is just sore, and Gary swears it's actually been blown off and he needs to operate immediately?
  - haldujai 3y ago
    Except if your leg is blown off every hospital in the US is legally required to treat you whether or not you have insurance or even if you’re homeless, so Gary is not your only option.

2devnull 3y ago

If you’re shocked by lack of push-back, you are in store for a lot more shocks going forward. People that build things and people that gatekeep, regulate, etc… are mostly disjoint sets. They are the two sides to upcoming war. Progress vs safety. The world as a whole is tilting towards progress right now, we’ve seen the limitations to safety.

Gerardo1 3y ago
I meant more in this community specifically, and, after the AI tool meant to fabricate survey results, YCombinator generally.

slashdev 3y ago

I see your “first do no harm” but I’ll counter that it is probably better than the alternative, which is to do nothing. And it may lead to people seeking professional help. It’s something for the unfilled gap in between, it’s not a replacement for professional help.

ancorevard 3y ago

Problem is that practitioners of mental health treatment themselves cause much damage today. This will not change that. Perhaps we're not too far from an LLM causing less damage than the _average_ mental health practitioner – with far more availability not to mention affordability.

toomuchtodo 3y ago
That's arguably for regulators to decide though, not technologists who have no exposure to the harm they cause. I can't even say "engineers" because engineers are held to a standard of care [1] that people slinging code are not. The fact that medical practitioners are sometimes wrong is not a legitimate means to wave away AI safety. In this instance, the solution isn't immature tech (that may [may!] one day be good tech), it's policy that enables healthcare access. You aren't going to fix that with LLMs. But, you know, that's what tech people keep doing. Poor tech solutions to people problems, and sometimes getting wealthy due to luck along the way. [1] https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd575265 https://docs.acec.org/pub/18803059-a2fd-2d06-cc39-a6d1dd5752...

srackey 3y ago

This is in fact not “absurdly dangerous”. It may actually be safer than using a biological intelligence for the same purpose. Given that, and the shortage of biological mental health workers, it seems reasonable that many many more would suffer negative outcomes were they not allowed to use this tech. As such, I’d argue that your comment is absurdly dangerous rhetoric, and your license to post on this web forum should be revoked.

WanderPanda 3y ago

A good LLMs answers should on average still be better than what an average person would come up with People give bad advice to other people all the time

Gerardo1 3y ago
I'm not asking your average person for therapy or to help diagnose or work through my mental health issues.