5 ms·

Are you sure ChatGPT is the solution? It just sounds like another "savior complex" sell spin from tech. 1. Social media -> connection 2. AGI -> erotica 3. Suic

by mgh2 1y ago

Are you sure ChatGPT is the solution? It just sounds like another "savior complex" sell spin from tech. 1. Social media -> connection 2. AGI -> erotica 3. Suicide -> prevention All these for engagement (i.e. addiction). It seems like the tech industry is the root cause itself trying to masquerade the problem by brainwashing the population. https://news.ycombinator.com/item?id=45026886 https://news.ycombinator.com/item?id=45026886

ben_w 1y ago
I personally take no opinion about whether or not they can actually solve anything, because I am not a psychologist and have absolutely no idea how good or bad ChatGPT is at this sort of thing, but I will say I'd rather the company at least tries to do some good given that Facebook HQ is not very far from their offices and appears to have been actively evil in this specific regard.
- btilly 1y ago
  The general rule of thumb is this. When given the right prompts, LLMs can be very effective at therapy. Certainly my wife gets a lot of mileage out of having ChatGPT help her reframe things in a better way. However "the right prompts" are not the ones that most mentally ill people would choose for themselves. And it is very easy for ChatGPT to become part of a person's delusion spiral, rather then be a helpful part of trying to solve it.
  - Demiurge 1y ago
    Is it better or worse than alternatives? Where else would a suicidal person turn, a forum with other suicidal people? Dry Wikipedia stats on suicide? Perhaps friends? Knowing how ChatGPT replies to me, I’d have a lot of trouble getting negativity influenced by it, any more than by yellow pages. Yeah, it used to try more to be your friend but GPT5 seems pretty neutral and distant.
    - btilly 1y ago
      I think that you will find a lot of strong opinions, and not a lot of hard data. Certainly any approach can work out poorly. For example antidepressants come with warnings about suicide risk. The reason is that they can enable people to take action on their suicidal feelings, before their suicidal feelings are fixed by the treatment. I know that many teens turn to social media. My strong opinions against that show up in other comments...
      - pasteldream 1y ago
        > The reason is that they can enable people to take action on their suicidal feelings, before their suicidal feelings are fixed by the treatment. I see that explanation for the increased suicide risk caused by antidepressants a lot, but what’s the evidence for it? It doesn’t necessarily have to be a study, just a reason why people believe it.
        btilly 1y ago
        Case studies support this. Which is a fancy way to say, "We carefully documented anecdotal reports and saw what looks like a pattern." There is also a strong parallel to manic depression. Manic depressives have a high suicide risk, and it usually happens when they are coming out of depression. With akathisia (fancy way to say inner restlessness) being the leading indicator. The same pattern is seen with antidepressants. The patient gets treatment, develops akathisia, then attempts suicide. But, as with many things to do with mental health, we don't really know what is going on inside of people. While also knowing that their self-reports are, shall we say, creatively misleading. So it is easy to have beliefs about what is going on. And rather harder to verify them.
        pasteldream 1y ago
        Can you point me to one of these reviews of case reports? As it is, your reply is too vague to be helpful.
    - jamilton 1y ago
  - ml-anon 1y ago
    Holy shit this is so fucking wrong and dangerous. No, LLMs are not and cannot be “very effective at therapy”.
    - Dilettante_ 1y ago
      Can you give just a little bit more effort explaining why you say that?
      - ml-anon 1y ago
        You know, usually it’s positive claims which are supposed to be substantiated, such as the claim that “LLMs can be good at therapy”. Holy shit, this thread is insane.
        Dilettante_ 1y ago
        I wasn't saying your position is wrong, just that it doesn't really make a good contribution to the discussion.
        lagosfractal42 1y ago
        You added nothing to the thread. Just get out.
        ml-anon 1y ago
        lol that’s rich given we’re in a thread about using ChatGPT as a therapist.
        btilly 1y ago
        You don't seem to understand how burden of proof works. My claim that LLMS can do effective therapeutic things is a positive claim. My report of my wife's experience is evidence. My example of something it has done for her is something that other people, who have experienced LLMs, can sanity check and decide whether they think this is possible. You responded by saying that it is categorically impossible for this to be true. Statements of impossibility are *ALSO* positive claims. You have provided no evidence for your claim. You have failed to meet the burden of proof for your position. (You have also failed to clarify exactly what you consider impossible - I suspect that you are responding to something other than what I actually said.) This is doubly true given the documented effectiveness of tools like https://www.rosebud.app/ https://www.rosebud.app/. Does it have very significant limitations? Yes. But does it deliver an experience that helps a lot of people's mental health? Also, yes. In fact that app is recommended by many therapists as a complement to therapy. But is it a replacement for therapy? Absolutely not! As they themselves point out in https://www.rosebud.app/care https://www.rosebud.app/care, LLMs consistently miss important things that a human therapist should be expected to catch. With the right prompts, LLMs are good at helping people learn and internalize positive mental health skills. But that kind of use case only covers some of the things that therapists do for you. So LLMs can and do to effective therapeutic things when prompted correctly. But they are not a replacement for therapy. And, of course, an unprompted LLM is unlikely to on its own do the potentially helpful things that it could.
- ndiddy 1y ago
  > but I will say I'd rather the company at least tries to do some good given that Facebook HQ is not very far from their offices and appears to have been actively evil in this specific regard. Sure! let's take a look at OpenAI's executive staff to see how equipped they are to take a morally different approach than Meta. Fidji Simo - CEO of Applications (formerly Head of Facebook at Meta) Vijaye Raji - CTO of Applications (formerly VP of Entertainment at Meta) Srinivas Narayanan - CTO of B2B Applications (formerly VP of Engineering at Meta) Kate Rouch - Chief Marketing Officer (formerly VP of Brand and Product Marketing at Meta) Irina Kofman - Head of Strategic Initiatives (formerly Senior Director of Product Management for Generative AI at Meta) Becky Waite - Head of Strategy/Operations (formerly Strategic Response at Meta) David Sasaki - VP of Analytics and Insights (formerly VP of Data Science for Advertising at Meta) Ashley Alexander - VP of Health Products (formerly Co-Head of Instagram Product at Meta) Ryan Beiermeister - Director of Product Policy (formerly Director of Product, Social Impact at Meta)
SecretDreams 1y ago
You actually need to add a loop in there between the suicide and erotica steps.
- anonym29 1y ago
  [flagged]
  - transcriptase 1y ago
    Bruh
  - Xevion 1y ago
    yikes
- decremental 1y ago
  [dead]
btilly 1y ago
I agree that the tech industry is the root cause of a lot of mental illness. But social media is a far bigger concern than AI. Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion...
- xedrac 1y ago
  AI is going to be more impactful than social media I'm afraid. But the two together just might be catastrophic for humanity.
- theblazehen 1y ago
  > Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion... I have noticed that TikTok can detect a depressive episode within ~a day of it starting (for me), as it always starts sending me way more self harm related content
  - Dilettante_ 1y ago
    Are you quite certain the depressive episode developed organically and Tiktok reacted to it? Maybe the algorithm started subtly on that path two days before you noticed the episode and you only realize once it starts showing self-harm content?
    - theblazehen 1y ago
      Hmm, that's quite possible (and concerning to think about) It had been showing me depressive content for days / weeks beforehand, during the start of the episode, however the sh content only started (Or I only noticed it) a few hours after I had a relapse, so the timing was rather uncanny
- deleted 1y ago
  [deleted]
skeledrew 1y ago
Whether solution or not, fact is AI* is the most available entity for anyone who has sensitive issues they'd like to share. It's (relatively) cheap, doesn't judge, is always there when wanted/needed and can continue a conversation exactly where left off at any point. * LLM would of course be technically more correct, but that term doesn't appeal to people seeking some level of intelligent interaction.
golergka 1y ago
ChatGPT/Claude can be absolutely brilliant in supportive, every day therapy, in my experience. BUT there are few caveats: I'm in therapy for a long time already (500+ hours), I don't trust it with important judgements or advice that goes counter to what I or my therapists think, and I also give Claude access to my diary with MCP, which makes it much better at figuring the context of what I'm talking about. Also, please keep in mind "supportive, every day". It's talking through stuff that I already know about, not seeking some new insights and revelations. Just shooting the shit with an entity which is booted with well defined ideas from you, your real human therapist and can give you very predictable, just common sense reactions that can still help when it's 2am and you have nobody to talk to, and all of your friends have already heard this exact talk about these exact problems 10 times already.
- zaptheimpaler 1y ago
  How do you connect your diary to an LLM? I've been struggling with getting an MCP for Evernote setup.
  - golergka 1y ago
    Obsidian.
  - tstrimple 1y ago
    I don’t use it for therapy, but my notes and journal are all just Logseq markdown. I’ve got a claude code instance running on my NAS with full two way access to my notes. It can read everything and can add new entries and tasks for me.
deleted 1y ago
[deleted]
npteljes 1y ago
I don't think "doing something about it" equals to "being a solution". Tackling the problems of the homeless, people operate a lot of food banks. Those don't even begin to solve homelessness, yet it's a precious resource, so, "doing something".

by mgh2 1y ago

ben_w 1y ago

I personally take no opinion about whether or not they can actually solve anything, because I am not a psychologist and have absolutely no idea how good or bad ChatGPT is at this sort of thing, but I will say I'd rather the company at least tries to do some good given that Facebook HQ is not very far from their offices and appears to have been actively evil in this specific regard.

btilly 1y ago
The general rule of thumb is this. When given the right prompts, LLMs can be very effective at therapy. Certainly my wife gets a lot of mileage out of having ChatGPT help her reframe things in a better way. However "the right prompts" are not the ones that most mentally ill people would choose for themselves. And it is very easy for ChatGPT to become part of a person's delusion spiral, rather then be a helpful part of trying to solve it.
- Demiurge 1y ago
  Is it better or worse than alternatives? Where else would a suicidal person turn, a forum with other suicidal people? Dry Wikipedia stats on suicide? Perhaps friends? Knowing how ChatGPT replies to me, I’d have a lot of trouble getting negativity influenced by it, any more than by yellow pages. Yeah, it used to try more to be your friend but GPT5 seems pretty neutral and distant.
  - btilly 1y ago
    I think that you will find a lot of strong opinions, and not a lot of hard data. Certainly any approach can work out poorly. For example antidepressants come with warnings about suicide risk. The reason is that they can enable people to take action on their suicidal feelings, before their suicidal feelings are fixed by the treatment. I know that many teens turn to social media. My strong opinions against that show up in other comments...
    - pasteldream 1y ago
      > The reason is that they can enable people to take action on their suicidal feelings, before their suicidal feelings are fixed by the treatment. I see that explanation for the increased suicide risk caused by antidepressants a lot, but what’s the evidence for it? It doesn’t necessarily have to be a study, just a reason why people believe it.
      - btilly 1y ago
        Case studies support this. Which is a fancy way to say, "We carefully documented anecdotal reports and saw what looks like a pattern." There is also a strong parallel to manic depression. Manic depressives have a high suicide risk, and it usually happens when they are coming out of depression. With akathisia (fancy way to say inner restlessness) being the leading indicator. The same pattern is seen with antidepressants. The patient gets treatment, develops akathisia, then attempts suicide. But, as with many things to do with mental health, we don't really know what is going on inside of people. While also knowing that their self-reports are, shall we say, creatively misleading. So it is easy to have beliefs about what is going on. And rather harder to verify them.
        pasteldream 1y ago
        Can you point me to one of these reviews of case reports? As it is, your reply is too vague to be helpful.
  - jamilton 1y ago
- ml-anon 1y ago
  Holy shit this is so fucking wrong and dangerous. No, LLMs are not and cannot be “very effective at therapy”.
  - Dilettante_ 1y ago
    Can you give just a little bit more effort explaining why you say that?
    - ml-anon 1y ago
      You know, usually it’s positive claims which are supposed to be substantiated, such as the claim that “LLMs can be good at therapy”. Holy shit, this thread is insane.
      - Dilettante_ 1y ago
        I wasn't saying your position is wrong, just that it doesn't really make a good contribution to the discussion.
      - lagosfractal42 1y ago
        You added nothing to the thread. Just get out.
        ml-anon 1y ago
        lol that’s rich given we’re in a thread about using ChatGPT as a therapist.
      - btilly 1y ago
        You don't seem to understand how burden of proof works. My claim that LLMS can do effective therapeutic things is a positive claim. My report of my wife's experience is evidence. My example of something it has done for her is something that other people, who have experienced LLMs, can sanity check and decide whether they think this is possible. You responded by saying that it is categorically impossible for this to be true. Statements of impossibility are *ALSO* positive claims. You have provided no evidence for your claim. You have failed to meet the burden of proof for your position. (You have also failed to clarify exactly what you consider impossible - I suspect that you are responding to something other than what I actually said.) This is doubly true given the documented effectiveness of tools like https://www.rosebud.app/ https://www.rosebud.app/. Does it have very significant limitations? Yes. But does it deliver an experience that helps a lot of people's mental health? Also, yes. In fact that app is recommended by many therapists as a complement to therapy. But is it a replacement for therapy? Absolutely not! As they themselves point out in https://www.rosebud.app/care https://www.rosebud.app/care, LLMs consistently miss important things that a human therapist should be expected to catch. With the right prompts, LLMs are good at helping people learn and internalize positive mental health skills. But that kind of use case only covers some of the things that therapists do for you. So LLMs can and do to effective therapeutic things when prompted correctly. But they are not a replacement for therapy. And, of course, an unprompted LLM is unlikely to on its own do the potentially helpful things that it could.
ndiddy 1y ago
> but I will say I'd rather the company at least tries to do some good given that Facebook HQ is not very far from their offices and appears to have been actively evil in this specific regard. Sure! let's take a look at OpenAI's executive staff to see how equipped they are to take a morally different approach than Meta. Fidji Simo - CEO of Applications (formerly Head of Facebook at Meta) Vijaye Raji - CTO of Applications (formerly VP of Entertainment at Meta) Srinivas Narayanan - CTO of B2B Applications (formerly VP of Engineering at Meta) Kate Rouch - Chief Marketing Officer (formerly VP of Brand and Product Marketing at Meta) Irina Kofman - Head of Strategic Initiatives (formerly Senior Director of Product Management for Generative AI at Meta) Becky Waite - Head of Strategy/Operations (formerly Strategic Response at Meta) David Sasaki - VP of Analytics and Insights (formerly VP of Data Science for Advertising at Meta) Ashley Alexander - VP of Health Products (formerly Co-Head of Instagram Product at Meta) Ryan Beiermeister - Director of Product Policy (formerly Director of Product, Social Impact at Meta)

SecretDreams 1y ago

You actually need to add a loop in there between the suicide and erotica steps.

anonym29 1y ago
[flagged]
- transcriptase 1y ago
  Bruh
- Xevion 1y ago
  yikes
decremental 1y ago
[dead]

btilly 1y ago

I agree that the tech industry is the root cause of a lot of mental illness. But social media is a far bigger concern than AI. Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion...

xedrac 1y ago
AI is going to be more impactful than social media I'm afraid. But the two together just might be catastrophic for humanity.
theblazehen 1y ago
> Unless, of course, you count the AI algorithms that TikTok uses to drive engagement, which in turn can cause social contagion... I have noticed that TikTok can detect a depressive episode within ~a day of it starting (for me), as it always starts sending me way more self harm related content
- Dilettante_ 1y ago
  Are you quite certain the depressive episode developed organically and Tiktok reacted to it? Maybe the algorithm started subtly on that path two days before you noticed the episode and you only realize once it starts showing self-harm content?
  - theblazehen 1y ago
    Hmm, that's quite possible (and concerning to think about) It had been showing me depressive content for days / weeks beforehand, during the start of the episode, however the sh content only started (Or I only noticed it) a few hours after I had a relapse, so the timing was rather uncanny
deleted 1y ago
[deleted]

skeledrew 1y ago

Whether solution or not, fact is AI* is the most available entity for anyone who has sensitive issues they'd like to share. It's (relatively) cheap, doesn't judge, is always there when wanted/needed and can continue a conversation exactly where left off at any point. * LLM would of course be technically more correct, but that term doesn't appeal to people seeking some level of intelligent interaction.

golergka 1y ago

ChatGPT/Claude can be absolutely brilliant in supportive, every day therapy, in my experience. BUT there are few caveats: I'm in therapy for a long time already (500+ hours), I don't trust it with important judgements or advice that goes counter to what I or my therapists think, and I also give Claude access to my diary with MCP, which makes it much better at figuring the context of what I'm talking about. Also, please keep in mind "supportive, every day". It's talking through stuff that I already know about, not seeking some new insights and revelations. Just shooting the shit with an entity which is booted with well defined ideas from you, your real human therapist and can give you very predictable, just common sense reactions that can still help when it's 2am and you have nobody to talk to, and all of your friends have already heard this exact talk about these exact problems 10 times already.

zaptheimpaler 1y ago
How do you connect your diary to an LLM? I've been struggling with getting an MCP for Evernote setup.
- golergka 1y ago
  Obsidian.
- tstrimple 1y ago
  I don’t use it for therapy, but my notes and journal are all just Logseq markdown. I’ve got a claude code instance running on my NAS with full two way access to my notes. It can read everything and can add new entries and tasks for me.

deleted 1y ago

[deleted]

npteljes 1y ago

I don't think "doing something about it" equals to "being a solution". Tackling the problems of the homeless, people operate a lot of food banks. Those don't even begin to solve homelessness, yet it's a precious resource, so, "doing something".