4 ms·

>Damn, we hardly know who our users are No, they know who you are. 1 in 3 already use AI for mental health advice among many other things.

abirch 15d ago
I would say misuse AI for mental health advice. It's already difficult to find good mental health professionals.
- fn-mote 15d ago
  Those two sentences seem to be a non-sequitor. Unavailability is one reason people would turn to AI. Ease of access is another.
  - simonwsimonw 15d ago
    [dead]
- Cthulhu_ 15d ago
  Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.
  - newswasboring 15d ago
    > In theory the LLM will escalate or recommend professional help if they're out of their depth. LLMs can barely identify gaps in their own software designs, and they are provably good at making software.
    - duskwuff 15d ago
      LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
  - officeplant 15d ago
    >In theory the LLM will escalate or recommend professional help if they're out of their depth. Given there have been multiple suicides driven by LLMs. I really doubt that.
    - tt24 15d ago
      Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
      - officeplant 15d ago
        I'm sure the Torment Nexus can be improved.
        tt24 13d ago
        Thought terminating cliche.
      - pocksuppet 13d ago
        Really hot take apparently: any technology that causes people to commit suicide during therapy isn't a good therapist.
        tt24 13d ago
        Is your claim that human therapists have caused zero suicides? I hope not for your sake - that would be a really hard one to defend
        pocksuppet 12d ago
        Why do you think that a human therapist who causes patients to commit suicide is a good therapist? I don't understand that.
        tt24 11d ago
  - micromacrofoot 15d ago
    In practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
    - voakbasda 15d ago
      And how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.
- deleted 15d ago
  [deleted]
- nekusar 15d ago
  I've yet to find anybody in mental health profession who is "good", aside psychiatrists and their drug prescriptions. And that's only cause of gatekeeping drugs, not because they're good either. The worst problem with mental health care is that there is no "you're better" metric. There is if your bone breaks and heal. There is for surgery of various things. But mental health seems to follow Parkinson's Law, and expands to the number of sessions available, whether it helps or not. And at X00$/session, yeah gets costly quick. So if something trained on all psychiatric literature costs $20/month subscription, that's a substitute therapist. Whether it's clinically trained or not.
  - cheesecakegood 14d ago
    I’ve seen about 8 therapists and 3 coaches at various times in the last 15 years. 5-6/8 and 2/3 respectively were, I eventually realized, mostly in it for the money and didn’t actually care too much (if at all) about me making any positive changes permanent. It wasn’t all bad; having someone confidential to talk to is in itself valuable, but the cost/benefit ratio is not at all consistent. With that kind of benchmark comparison, it’s not all that hard for AI to present some kind of favorable value proposition, even if also inconsistent but with a harder ceiling on the upside.
- miki123211 15d ago
  Unless you have Bill Gates money, you don't have access to a mental health professional (much less your mental health professional who knows your life situation) when a girlfriend dumps you over text at 1AM. And that's when you may need one most. People want to talk to their therapist after an argument with their husband or when their boss fires them, not Friday after next.
  - newswasboring 15d ago
    The need is not in question, the solution is. This annoying glaze machine is not it for mental health tasks.
    - nish__ 15d ago
      Most people have friends and family to "glaze" them anyway. I don't see it as a major danger. Also, it's gotten better with the glazing lately anyway.
      - hatsix 15d ago
        The danger is in the replacement. Confiding in others builds a web of trust. Talking about a break-up with a friend builds a relationship, and the next time that friend needs someone, they know you're an option. It's economics, borrowing and trading emotional support. AI as confidant destroys that economy.
    - p-e-w 15d ago
      Tens of millions of people disagree with you. What makes you think you know better than them what is right for them?
      - z0r 15d ago
        This line of reasoning doesn't hold up for many harmful habits that people voluntarily partake in. It's OK for someone to believe that using an AI as an approximate therapist might be a bad idea, especially in light of the many publicly documented cases of AI feeding delusional thoughts and encouraging suicide.

abirch 15d ago

I would say misuse AI for mental health advice. It's already difficult to find good mental health professionals.

fn-mote 15d ago
Those two sentences seem to be a non-sequitor. Unavailability is one reason people would turn to AI. Ease of access is another.
- simonwsimonw 15d ago
  [dead]
Cthulhu_ 15d ago
Exactly why people turn to AI, which I think is good enough for "casual" mental health advice (the kind people can do themselves by reading self-help books or talking with empathic friends). Arguably it's safer than talking to strangers online because now that they've been doing it for a few years, they will (or should) have plenty of safeguards, unlike people on the internet who can talk out of their ass (like me!) and give bad advice. In theory the LLM will escalate or recommend professional help if they're out of their depth.
- newswasboring 15d ago
  > In theory the LLM will escalate or recommend professional help if they're out of their depth. LLMs can barely identify gaps in their own software designs, and they are provably good at making software.
  - duskwuff 15d ago
    LLMs are also incredibly poor at recognizing that they're "out of their depth", especially when their context is already full of their attempts to do something.
- officeplant 15d ago
  >In theory the LLM will escalate or recommend professional help if they're out of their depth. Given there have been multiple suicides driven by LLMs. I really doubt that.
  - tt24 15d ago
    Fascinating perspective - any technology that has caused >0 deaths is a dead end and has no chance of improvement.
    - officeplant 15d ago
      I'm sure the Torment Nexus can be improved.
      - tt24 13d ago
        Thought terminating cliche.
    - pocksuppet 13d ago
      Really hot take apparently: any technology that causes people to commit suicide during therapy isn't a good therapist.
      - tt24 13d ago
        Is your claim that human therapists have caused zero suicides? I hope not for your sake - that would be a really hard one to defend
        pocksuppet 12d ago
        Why do you think that a human therapist who causes patients to commit suicide is a good therapist? I don't understand that.
        tt24 11d ago
- micromacrofoot 15d ago
  In practice we know that LLMs have aided in suicide, so what they should do in theory doesn't matter.
  - voakbasda 15d ago
    And how many people can it be credited for saving from self harm? Probably countless more than it has harmed, if I had to bet.
deleted 15d ago
[deleted]
nekusar 15d ago
I've yet to find anybody in mental health profession who is "good", aside psychiatrists and their drug prescriptions. And that's only cause of gatekeeping drugs, not because they're good either. The worst problem with mental health care is that there is no "you're better" metric. There is if your bone breaks and heal. There is for surgery of various things. But mental health seems to follow Parkinson's Law, and expands to the number of sessions available, whether it helps or not. And at X00$/session, yeah gets costly quick. So if something trained on all psychiatric literature costs $20/month subscription, that's a substitute therapist. Whether it's clinically trained or not.
- cheesecakegood 14d ago
  I’ve seen about 8 therapists and 3 coaches at various times in the last 15 years. 5-6/8 and 2/3 respectively were, I eventually realized, mostly in it for the money and didn’t actually care too much (if at all) about me making any positive changes permanent. It wasn’t all bad; having someone confidential to talk to is in itself valuable, but the cost/benefit ratio is not at all consistent. With that kind of benchmark comparison, it’s not all that hard for AI to present some kind of favorable value proposition, even if also inconsistent but with a harder ceiling on the upside.
miki123211 15d ago
Unless you have Bill Gates money, you don't have access to a mental health professional (much less your mental health professional who knows your life situation) when a girlfriend dumps you over text at 1AM. And that's when you may need one most. People want to talk to their therapist after an argument with their husband or when their boss fires them, not Friday after next.
- newswasboring 15d ago
  The need is not in question, the solution is. This annoying glaze machine is not it for mental health tasks.
  - nish__ 15d ago
    Most people have friends and family to "glaze" them anyway. I don't see it as a major danger. Also, it's gotten better with the glazing lately anyway.
    - hatsix 15d ago
      The danger is in the replacement. Confiding in others builds a web of trust. Talking about a break-up with a friend builds a relationship, and the next time that friend needs someone, they know you're an option. It's economics, borrowing and trading emotional support. AI as confidant destroys that economy.
  - p-e-w 15d ago
    Tens of millions of people disagree with you. What makes you think you know better than them what is right for them?
    - z0r 15d ago
      This line of reasoning doesn't hold up for many harmful habits that people voluntarily partake in. It's OK for someone to believe that using an AI as an approximate therapist might be a bad idea, especially in light of the many publicly documented cases of AI feeding delusional thoughts and encouraging suicide.