5 ms·

It’s already doing a piss-poor job dealing with suicidal teenagers. Is there any reason to believe AI will be any better than social media when it comes to men

It’s already doing a piss-poor job dealing with suicidal teenagers. Is there any reason to believe AI will be any better than social media when it comes to mental health? https://www.washingtonpost.com/technology/2025/12/27/chatgpt-suicide-openai-raine/ https://www.washingtonpost.com/technology/2025/12/27/chatgpt...

eastbound 10mo ago
Yes. It pulls people towards normality, since it gives the average words for every answer. Meanwhile social media encouraged people to be different enough to surface, and therefore encouraged abnormality.
- watwut 10mo ago
  It will happily validate you toward any deep hole or extreme you want to. It is as bad or even worst then social media in that regard.
- alistairSH 10mo ago
  Try reading the article. ChatGPT did no such thing.
- mgraczyk 10mo ago
  It's not true in any sense that LLMs "give the average words for every answer"
  - embedding-shape 10mo ago
    It's a over-simplification, that's for sure, one bordering on incorrect. But for people who don't care about the internals, I don't think it's a harmful perspective to keep.
    - mgraczyk 10mo ago
      It's harmful because in this context it leads to an incorrect conclusion. There's no reason to believe that LLMs "averaging" behavior would cause a suicidal person to be "pulled toward normal"
      - embedding-shape 10mo ago
        It's a philosophical argument more than anything I think. And it does beg the question, does your mind form itself around with the humans (entities?) you converse with? So if you talk with a lot of smart people, you'll end up a bit smarter yourself, and if you talk with a lot of dull people, you'll end up dulling yourself. If you agree with that, I can see how someone would believe that LLMs would pull people closer to the material they were trained on.
        deleted 10mo ago
        [deleted]
  - buckle8017 10mo ago
    That's literally what an LLM is. They predict what the most likely next word is.
    - mgraczyk 10mo ago
      That's wrong and even if it were not wrong, it would still not fix the problem. What if the most likely response is "kill yourself"
    - MallocVoidstar 10mo ago
      'Predicts the most likely tokens' is not the same as 'pulls people towards normality'.
embedding-shape 10mo ago
"It" being ChatGPT, in that case. I guess most people know, but not all AI is the same as all other AI, the implementation in those cases matter more than what weights are behind it, or that it's actually ML rather than something else. With that said, like most technology, it seems to come with a ton of drawbacks, and some benefits, while most people focus on the benefits, we're surely about to find out all the drawbacks shortly. Better than social media or not, its being deployed on a wide-scale, so it's less about what each person believes, and more about what we're ready to deal with and how we want to deal with it.
- KaiserPro 10mo ago
  > the implementation in those cases matter more There is/are currently no realistic ways to temper or enforce public safety on these companies. They are in full regulatory capture. Any kind of call for public safety will be set aside, and if its not someone will pay the exec to give them an exception.
  - embedding-shape 10mo ago
    > There is/are currently no realistic ways to temper or enforce public safety on these companies There is, general strikes usually does the trick if the government stops listening to the people. Of course, this doesn't apply to some countries that spent decades making unions, syndicates and other movements handicapped, but for the modern countries that still pride themselves on democracy, it is possible, given enough people care to do something about it.
    - danaris 10mo ago
      I mean, what you say is not really wrong, but it's also not really relevant to the post (or thread) you're replying to. It doesn't matter what government is in control: LLMs cannot be made safe from the problems that plague them. They are fundamental to their basic makeup.
    - KaiserPro 10mo ago
      I was talking specifically about the USA. Unless something dramatic changes, there will not be a general strike. Even when unemployment rises to ~15%
jstummbillig 10mo ago
I am honestly amazed by how people are not interested in the obvious counterfactuals. Are we seriously that disinterested in how many people were helped by AI, that we don't even feel the need to ask that question when looking at potential harm?
- MallocVoidstar 10mo ago
  Why should "some people have been helped by AI" outweigh "ChatGPT has convinced multiple teenagers to commit suicide"?
  - Saline9515 10mo ago
    I'm pretty sure that many people with no instant access to doctors have also been helped by AI to diagnose illnesses and know when to consult. As with any technology, you can't evaluate it by focusing only on the worst effects.
  - derektank 10mo ago
    What if multiple teenagers were convinced to not commit suicide by AI? The story says that ChatGPT urged the teen to seek help from loved ones dozens of time. It seems plausible to me that other people out there actually listened to ChatGPT’s advice and received care as a result, when they might have attempted suicide otherwise. We simply don’t know the scale of either side of the equation at this point.
    - MallocVoidstar 10mo ago
      If a therapist tells 10 people not to kill themselves, and convinces 5 patients to kill themselves, would you say "this therapist is doing good on the whole"?
      - jstummbillig 10mo ago
        What if it's 100000 people who not kill themselves?

eastbound 10mo ago

Yes. It pulls people towards normality, since it gives the average words for every answer. Meanwhile social media encouraged people to be different enough to surface, and therefore encouraged abnormality.

watwut 10mo ago
It will happily validate you toward any deep hole or extreme you want to. It is as bad or even worst then social media in that regard.
alistairSH 10mo ago
Try reading the article. ChatGPT did no such thing.
mgraczyk 10mo ago
It's not true in any sense that LLMs "give the average words for every answer"
- embedding-shape 10mo ago
  It's a over-simplification, that's for sure, one bordering on incorrect. But for people who don't care about the internals, I don't think it's a harmful perspective to keep.
  - mgraczyk 10mo ago
    It's harmful because in this context it leads to an incorrect conclusion. There's no reason to believe that LLMs "averaging" behavior would cause a suicidal person to be "pulled toward normal"
    - embedding-shape 10mo ago
      It's a philosophical argument more than anything I think. And it does beg the question, does your mind form itself around with the humans (entities?) you converse with? So if you talk with a lot of smart people, you'll end up a bit smarter yourself, and if you talk with a lot of dull people, you'll end up dulling yourself. If you agree with that, I can see how someone would believe that LLMs would pull people closer to the material they were trained on.
      - deleted 10mo ago
        [deleted]
- buckle8017 10mo ago
  That's literally what an LLM is. They predict what the most likely next word is.
  - mgraczyk 10mo ago
    That's wrong and even if it were not wrong, it would still not fix the problem. What if the most likely response is "kill yourself"
  - MallocVoidstar 10mo ago
    'Predicts the most likely tokens' is not the same as 'pulls people towards normality'.

embedding-shape 10mo ago

"It" being ChatGPT, in that case. I guess most people know, but not all AI is the same as all other AI, the implementation in those cases matter more than what weights are behind it, or that it's actually ML rather than something else. With that said, like most technology, it seems to come with a ton of drawbacks, and some benefits, while most people focus on the benefits, we're surely about to find out all the drawbacks shortly. Better than social media or not, its being deployed on a wide-scale, so it's less about what each person believes, and more about what we're ready to deal with and how we want to deal with it.

KaiserPro 10mo ago
> the implementation in those cases matter more There is/are currently no realistic ways to temper or enforce public safety on these companies. They are in full regulatory capture. Any kind of call for public safety will be set aside, and if its not someone will pay the exec to give them an exception.
- embedding-shape 10mo ago
  > There is/are currently no realistic ways to temper or enforce public safety on these companies There is, general strikes usually does the trick if the government stops listening to the people. Of course, this doesn't apply to some countries that spent decades making unions, syndicates and other movements handicapped, but for the modern countries that still pride themselves on democracy, it is possible, given enough people care to do something about it.
  - danaris 10mo ago
    I mean, what you say is not really wrong, but it's also not really relevant to the post (or thread) you're replying to. It doesn't matter what government is in control: LLMs cannot be made safe from the problems that plague them. They are fundamental to their basic makeup.
  - KaiserPro 10mo ago
    I was talking specifically about the USA. Unless something dramatic changes, there will not be a general strike. Even when unemployment rises to ~15%

jstummbillig 10mo ago

I am honestly amazed by how people are not interested in the obvious counterfactuals. Are we seriously that disinterested in how many people were helped by AI, that we don't even feel the need to ask that question when looking at potential harm?

MallocVoidstar 10mo ago
Why should "some people have been helped by AI" outweigh "ChatGPT has convinced multiple teenagers to commit suicide"?
- Saline9515 10mo ago
  I'm pretty sure that many people with no instant access to doctors have also been helped by AI to diagnose illnesses and know when to consult. As with any technology, you can't evaluate it by focusing only on the worst effects.
- derektank 10mo ago
  What if multiple teenagers were convinced to not commit suicide by AI? The story says that ChatGPT urged the teen to seek help from loved ones dozens of time. It seems plausible to me that other people out there actually listened to ChatGPT’s advice and received care as a result, when they might have attempted suicide otherwise. We simply don’t know the scale of either side of the equation at this point.
  - MallocVoidstar 10mo ago
    If a therapist tells 10 people not to kill themselves, and convinces 5 patients to kill themselves, would you say "this therapist is doing good on the whole"?
    - jstummbillig 10mo ago
      What if it's 100000 people who not kill themselves?