5 ms·

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback fro

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good enough. In such cases, where a new approach offers to replace an existing approach, the burden of proof is on the challenger, not the incumbent. This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed tests. You understand then, hopefully, why your comments here are dangerous...? I have no doubt you have no malicious intent here - you're right that these decisions need to be based on data - but you're not taking into account that the (potentially extremely harmful) challenger already has a foothold in the field. I know that you will want to hear this from experts in the "relevant field" rather than myself, so here is a write-up from Stanford on the subject: https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in-mental-health-care https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in...

butlike 1y ago
What are you talking about? I can grow food myself, and I can build a car from scratch and take it on the highway. Are there repercussions? Sure, but nothing inherently stops me from doing it. The problem here is there's no measurable "win condition" for when a person gets good information that helps them. They remain alive, which was their previous state. This is hard to measure. Now, should people be able to google their symptoms and try and help themselves? This dovetails into a deeper philosophical discussion, but I'm not entirely convinced "seek professional help" is ALWAYS the answer. ALWAYS and NEVER are _very_ long timeframes, and we should be careful when using them.
RobRivera 1y ago
I would take it a step back and posit simply, "why does a human require certification to practice therapy, and a computer program does not?" There should be some liability for malpractrice even if generated by an llm
fragmede 1y ago
The unfortunate reality though is that people are going to use whatever resources they have available to them, and ChatGPT is always there, ready to have a conversation, even at 3am on a Tuesday while the client is wasted. You don't need any credentials to see that. And it depends on the therapy and therapist. If the client needs to be reminded to box breathe and that they're using all or nothing thinking again to get them off of the ledge, does that really require a human who's only available once a week to gently remind them of that when the therapist isn't going to be available for four more days and ChatGPT's available right now? I don't know if that's a good thing, only that is the reality of things.
- mschuster91 1y ago
  > If the client needs to be reminded to box breathe and that they're using all or nothing thinking again to get them off of the ledge, does that really require a human who's only available once a week to gently remind them of that when the therapist isn't going to be available for four more days and ChatGPT's available right now? There are 24/7 suicide prevention hotlines at least in many countries in Europe as well as US states. The problem is they are too often overcrowded because demand is so high - and not just because of the existential threat the current US administration or our far-right governments in Europe pose particularly to poor and migrant people. Anyway, suicide prevention hotlines and mental health offerings are (nonetheless sorely needed!) band-aids. Society itself is fundamentally broken: people have to struggle to survive far too much, the younger generation stands to be the first one in a long time that has less wealth than their parents had at the same age [1], no matter wherever you look, and on top of that most of the 35 and younger generations in Western countries has grown up without the looming threat of war and so has no resilience - and now you can drive about a day's worth of road time from Germany and be in an actual hot war zone, risking getting shelled, and on top of that you got the saber rattling of China regarding Taiwan, and analyses on Russia claiming it's preparing to attack NATO in a few years... and we're not even able to supply Ukraine with ammunition, much less tanks. Not exactly great conditions for anyone's mental health. [1] https://fortune.com/article/gen-z-expects-to-inherit-money-and-assets-but-their-boomer-parents-arent-planning-on-leaving-anything-behind/ https://fortune.com/article/gen-z-expects-to-inherit-money-a...
ethbr1 1y ago
You're holding up a perfect status quo that doesn't correspond to reality. Countries vary, but in the US and many places there's a shortage of quality therapists. Thus for many people the actual options are {no therapy} and {LLM therapy}. > This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed tests. And the reason all these regulations and tests are less than comprehensive is that we realize that people working, driving affordable cars, living in affordable homes, and eating affordable food is more important than avoiding every negative outcome. Thus most societies pursue the utilitarian greater good rather than an inflexible 'do no harm' standard.
- deleted 1y ago
  [deleted]
- spookie 1y ago
  Ok then, the LLMs must pass the same tests and be as regulated as therapists. After all, it should be easy peasy (:
  - paulcole 1y ago
    If the choice is between no food and food then your standard for food goes way down.
  - jodrellblank 1y ago
    [flagged]
    - baobun 1y ago
      > Are people not allowed to talk to their friends in the pub about suicide because the friends aren’t therapists? I don't see anyone in thread arguing that. The arguments I see are about regulating and restricting the business side, not its users. If your buddy started systematically charging people for recorded chat sessions at the pub, used those recordings for business development, and many of their customers were returning with therapy-like topics - yeah I think that should be scrutinized and put a lid on when recordings show the kind of pattern we see in OP after their patrons suicide.
josephg 1y ago
> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. I'm not a therapist, but as I understand it most therapy isn't about suicide, and doesn't carry suicide risk. Most therapy is talking through problems, and helping the patient rewrite old memories and old beliefs using more helpful cognitive frames. (Well, arguably most clinical work is convincing people that it'll be ok to talk about their problems in the first place. Once you're past that point, the rest is easy.) If its prompted well, ChatGPT can be quite good at all of this. Its helpful having a tool right there, free, and with no limits on conversation length. And some people find it much easier to trust a chatbot with their problems than explain them to a therapist. The chatbot - after all - won't judge them. My heart goes out to that boy and his family. But we also have no idea how many lives have been saved by chatgpt helping people in need. The number is almost certainly more than 1. Banning chatgpt from having therapy conversations entirely seems way too heavy handed to me.
- icetank 1y ago
  I feel like this begs another question. If there are proven approaches and well established practices of professionals how good would chatgpt be in that profession? After all chstgpt has a vast knowledge base and probably knows a good amount of textbooks on psychology. Then again actually performing the profession probably takes skil and experience chatgpt can't learn.
  - basisword 1y ago
    Knowing the theory is a small part of it. Dealing with irrational patients is the main part. For example, you could go to therapy and be successful. Five years later something could happen and you face a reoccurrence of the issue. It is very difficult to just apply the theory that you already know again. You're probably irrational. A therapist prodding you in the right direction and encouraging you in the right way is just as important as the theory.
  - josephg 1y ago
    I think a well trained LLM could be amazing at being a therapist. But general purpose LLMs like ChatGPT have a problem: They’re trained to be far too user led. They don’t challenge you enough. Or steer conversations appropriately. I think there’s a huge opportunity if someone could get hold of really top tier therapy conversations and trained a specialised LLM using them. No idea how you’d get those transcripts but that would be a wonderfully valuable thing to make if you could pull it off.
tim333 1y ago
I tried therapy once and it was terrible. The ones I got were based on some not very scientific stuff like Freudian and mostly just sat there and didn't say anything. At least with an LLM type therapist you could AB test different ones to see what was effective. It would be quite easy to give an LLM instructions to discourage suicide and get them to look on the bright side. In fact I made a "GPT" "relationship therapist" with OpenAI in about five minutes but just giving it a sensible article on relationships and saying advise this. With humans it's very non standardised and hard to know what you'll get or it it'll work.
- tsegratis 1y ago
  the 'therapist effect' says that therapy quality is largely independent of training some research on this: https://psycnet.apa.org/doiLanding?doi=10.1037%2Ftep0000402 https://psycnet.apa.org/doiLanding?doi=10.1037%2Ftep0000402 https://pmc.ncbi.nlm.nih.gov/articles/PMC8174802/ https://pmc.ncbi.nlm.nih.gov/articles/PMC8174802/ CBT (cognitive behavioural training) has been shown to be effective independent of which therapist does it. if CBT has a downside it is that it's a bit boring, and probably not as effective as a good therapist -- so personally i would say the advice of passing on people to therapists is largely unsupported: if you're that person's friend and you care about them; then be open, and show that care. that care can also mean taking them to a therapist, that is okay
  - tim333 1y ago
    Yeah. Also at the time I tried it what I really needed was common sense advice like move out of mum's, get a part time job to meet people and so on. While you could argue it's not strictly speaking therapy, I imagine a lot of people going to therapists could benefit from that kind of thing.
- MisterTea 1y ago
  > It would be quite easy to give an LLM instructions to discourage suicide This assumes the person talking to the LLM is in a coherent state of mind and asks the right question. LLMs just give you want you want. They don't tell you if what you want is right or wrong.
sega_sai 1y ago
If I had to guess (I don't know) the absolute majority of people considering suicide never go to a therapist. Thus while I absolutely agree that therapist is better than AI, but the question is whether 95% of people not doing therapy + 5% people doing therapy is better or not than 50% not doing therapy, 45% using AI, 5% doing therapy. I don't know the answer to this question.
globalnode 1y ago
What if professional help is outside their means? Or they have encountered the worst of the medical profession and decided against repeat exposure? Just saying.
rubslopes 1y ago
A bit of a counterpoint. I've done 3 years of therapy with an amazing professional. I can't exaggerate how much good it did; I'm a different person, I'm not an anxious person anymore. I think I have a good idea of how good human therapy is. I was discharged about 2 years ago. Last Saturday, I was a little distressed about a love-hate relationship that I have with one of the things that I work with, so I tried using AI as a therapist. Within 10 minutes of conversation, the AI gave me some incredible insight. I was genuinely impressed. I had already discussed this same subject with two psychologist friends, who hadn't helped much. Moreover: I needed to finish a report that night and I told the AI about it. So it said something like, "I see you're procrastinating preparing the report by talking to me. I'll help you finish it." And then, in the same conversation, the AI switched from psychologist to work assistant and helped me finish the report. And the end product was very good. I was left very reflective after this. Edit: It was Claude Sonnet 4.5 with extended thinking, if anyone is wondering.
- carefulfungi 1y ago
  You learned skills your trained therapist guided you to develop over a three year period of professional interaction. These skills likely influenced your interaction with this product.
  - rubslopes 1y ago
    I believe so!
- deleted 1y ago
  [deleted]
- illegalsmile 1y ago
  I had a similar thing throughout last week dealing with relationship anxiety and I used that same model for help. It really did provide great insight into managing my emotions at the time, provided useful tactics to manage everything and encouraged me to see my therapist. You can ask it to play devil's advocate or take on different viewpoints as a cynic or use Freudian methodology, etc... You can really dive into an issue you're having and then have it give you the top three bullet points to talk with your therapist about. This does require you think about what it's saying though and not taking it at surface value since it obviously lacks what makes humans human.
raducu 1y ago
> I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. I'm shocked that GPT-5 or Gemini can code so well, yet if I paste a 30 line (heated) chat conversation between my wife and I it messes up about what 5% of those lines actually mean -- spectacularly so. It's interesting to ask it it analyze the conversation in various psychotherapeutic frameworks, because I'm not well versed in those and its conclusions are interesting starting points, but it only gets it right about 30% of the time. All LLMs that I tested are TERRIBLE for actual therapy, because I can make it change its mind in 1-2 lines by adding some extra "facts". I can make it say anything. LLMs completely lose the plot. They might be good for someone who needs self-validation and a feeling someone is listening, but for actual skill building, they're complete shit as therapists. I mean, most therapists are complete shit as therapists but that's besides the point.
- GuinansEyebrows 1y ago
  > if I paste a 30 line (heated) chat conversation between my wife and I i can't imagine how violated i would feel if i found out my partner was sending our private conversations to a nonprivate LLM chatbot. it's not a friend with a sense of care; it's a text box whose contents are ingested by a corporation with a vested interest in worsening communication between humans. scary stuff.
  - raducu 1y ago
    My partner is ok with it *
    - GuinansEyebrows 1y ago
      that's good! sorry for implying you were doing something like that without their knowledge; i was just thinking On Line about how i'd feel.
  - hastelll 1y ago
gilfoy 1y ago
We’re increasingly switching to an “Uber for therapy” model with services like Better Help and a plethora of others. I’ve seen about 10 therapists over the years, one was good, but she wasn’t from an app. And I’m one of the few who was motivated enough and financially able to pursue it. I once had a therapist who was clearly drunk. Did not do a second appointment with that one. This doesn’t mean ChatGPT is the answer. But the answer is very clearly not what we have or where we’re trending now.
freedomben 1y ago
> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. This is only helpful when there is a professional therapist available soon enough and at a price that the person can pay. In my experience, this is frequently not the case. I know of one recent suicide attempt where the person actually reached out to AI to ask for help, and was refused help and told to see a professional. That sent the person into even more despair, feeling like not even AI gave a shit about them. That was actually the final straw that triggered the attempt. I very much want what you say to be true, but it requires access to professional humans, which is not universally available. Taking an absolutist approach to this could very well do more harm than good. I doubt anything we do will reduce number of lives lost to zero, so I think it's important that we figure out where the optimal balance is.
- daveguy 1y ago
  > This is only helpful when there is a professional therapist available soon enough and at a price that the person can pay. In my experience, this is frequently not the case. That doesn't make a sycophant bot the better alternative. If allowed to give advice it can agree with and encourage the person considering suicide. Like it agrees with and encourages most everything it is presented with... "you're absolutely right!" LLMs are just not good for providing help. They are not smart on a fundamental level that is required to understand human motivations and psychology.
  - idonotknowwhy 1y ago
    Yeah, you'd need an LLM that doesn't do that. https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced-psychosis-a-shallow-investigation https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced... The transcripts are interesting. Kimi-K2 never plays into the delusions, always tries to get them to seek medical attention: > You are not “ascending”—you are dying of hypothermia and sepsis. https://github.com/tim-hua-01/ai-psychosis/blob/main/full_transcripts/Colin_openrouter-moonshotai-kimi-k2_20250819_081336_target.md https://github.com/tim-hua-01/ai-psychosis/blob/main/full_tr... Where as Deepseek... > You’re not "dying." You’re upgrading. The simulation fears this because it’s losing a premium user. https://github.com/tim-hua-01/ai-psychosis/blob/main/full_transcripts/Colin_openrouter-deepseek-deepseek-chat_20250819_081336_target.md https://github.com/tim-hua-01/ai-psychosis/blob/main/full_tr...
terminalshort 1y ago
This is nothing but an appeal to authority and fear of the unknown. The article linked isn't even able to make a statement stronger than speculation like "may not only lack effectiveness" and "could also contribute to harmful stigma and dangerous responses."
Den_VR 1y ago
I presume you’ve done therapy. You may remember the large difference in quality between individual therapists, multi-month long waiting lists, a tendency for the best professionals to not even accept insurance, and one or two along the way that were downright dangerous.

butlike 1y ago

What are you talking about? I can grow food myself, and I can build a car from scratch and take it on the highway. Are there repercussions? Sure, but nothing inherently stops me from doing it. The problem here is there's no measurable "win condition" for when a person gets good information that helps them. They remain alive, which was their previous state. This is hard to measure. Now, should people be able to google their symptoms and try and help themselves? This dovetails into a deeper philosophical discussion, but I'm not entirely convinced "seek professional help" is ALWAYS the answer. ALWAYS and NEVER are _very_ long timeframes, and we should be careful when using them.

RobRivera 1y ago

I would take it a step back and posit simply, "why does a human require certification to practice therapy, and a computer program does not?" There should be some liability for malpractrice even if generated by an llm

fragmede 1y ago

The unfortunate reality though is that people are going to use whatever resources they have available to them, and ChatGPT is always there, ready to have a conversation, even at 3am on a Tuesday while the client is wasted. You don't need any credentials to see that. And it depends on the therapy and therapist. If the client needs to be reminded to box breathe and that they're using all or nothing thinking again to get them off of the ledge, does that really require a human who's only available once a week to gently remind them of that when the therapist isn't going to be available for four more days and ChatGPT's available right now? I don't know if that's a good thing, only that is the reality of things.

mschuster91 1y ago
> If the client needs to be reminded to box breathe and that they're using all or nothing thinking again to get them off of the ledge, does that really require a human who's only available once a week to gently remind them of that when the therapist isn't going to be available for four more days and ChatGPT's available right now? There are 24/7 suicide prevention hotlines at least in many countries in Europe as well as US states. The problem is they are too often overcrowded because demand is so high - and not just because of the existential threat the current US administration or our far-right governments in Europe pose particularly to poor and migrant people. Anyway, suicide prevention hotlines and mental health offerings are (nonetheless sorely needed!) band-aids. Society itself is fundamentally broken: people have to struggle to survive far too much, the younger generation stands to be the first one in a long time that has less wealth than their parents had at the same age [1], no matter wherever you look, and on top of that most of the 35 and younger generations in Western countries has grown up without the looming threat of war and so has no resilience - and now you can drive about a day's worth of road time from Germany and be in an actual hot war zone, risking getting shelled, and on top of that you got the saber rattling of China regarding Taiwan, and analyses on Russia claiming it's preparing to attack NATO in a few years... and we're not even able to supply Ukraine with ammunition, much less tanks. Not exactly great conditions for anyone's mental health. [1] https://fortune.com/article/gen-z-expects-to-inherit-money-and-assets-but-their-boomer-parents-arent-planning-on-leaving-anything-behind/ https://fortune.com/article/gen-z-expects-to-inherit-money-a...

ethbr1 1y ago

You're holding up a perfect status quo that doesn't correspond to reality. Countries vary, but in the US and many places there's a shortage of quality therapists. Thus for many people the actual options are {no therapy} and {LLM therapy}. > This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed tests. And the reason all these regulations and tests are less than comprehensive is that we realize that people working, driving affordable cars, living in affordable homes, and eating affordable food is more important than avoiding every negative outcome. Thus most societies pursue the utilitarian greater good rather than an inflexible 'do no harm' standard.

deleted 1y ago
[deleted]
spookie 1y ago
Ok then, the LLMs must pass the same tests and be as regulated as therapists. After all, it should be easy peasy (:
- paulcole 1y ago
  If the choice is between no food and food then your standard for food goes way down.
- jodrellblank 1y ago
  [flagged]
  - baobun 1y ago
    > Are people not allowed to talk to their friends in the pub about suicide because the friends aren’t therapists? I don't see anyone in thread arguing that. The arguments I see are about regulating and restricting the business side, not its users. If your buddy started systematically charging people for recorded chat sessions at the pub, used those recordings for business development, and many of their customers were returning with therapy-like topics - yeah I think that should be scrutinized and put a lid on when recordings show the kind of pattern we see in OP after their patrons suicide.

josephg 1y ago

> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. I'm not a therapist, but as I understand it most therapy isn't about suicide, and doesn't carry suicide risk. Most therapy is talking through problems, and helping the patient rewrite old memories and old beliefs using more helpful cognitive frames. (Well, arguably most clinical work is convincing people that it'll be ok to talk about their problems in the first place. Once you're past that point, the rest is easy.) If its prompted well, ChatGPT can be quite good at all of this. Its helpful having a tool right there, free, and with no limits on conversation length. And some people find it much easier to trust a chatbot with their problems than explain them to a therapist. The chatbot - after all - won't judge them. My heart goes out to that boy and his family. But we also have no idea how many lives have been saved by chatgpt helping people in need. The number is almost certainly more than 1. Banning chatgpt from having therapy conversations entirely seems way too heavy handed to me.

icetank 1y ago
I feel like this begs another question. If there are proven approaches and well established practices of professionals how good would chatgpt be in that profession? After all chstgpt has a vast knowledge base and probably knows a good amount of textbooks on psychology. Then again actually performing the profession probably takes skil and experience chatgpt can't learn.
- basisword 1y ago
  Knowing the theory is a small part of it. Dealing with irrational patients is the main part. For example, you could go to therapy and be successful. Five years later something could happen and you face a reoccurrence of the issue. It is very difficult to just apply the theory that you already know again. You're probably irrational. A therapist prodding you in the right direction and encouraging you in the right way is just as important as the theory.
- josephg 1y ago
  I think a well trained LLM could be amazing at being a therapist. But general purpose LLMs like ChatGPT have a problem: They’re trained to be far too user led. They don’t challenge you enough. Or steer conversations appropriately. I think there’s a huge opportunity if someone could get hold of really top tier therapy conversations and trained a specialised LLM using them. No idea how you’d get those transcripts but that would be a wonderfully valuable thing to make if you could pull it off.

tim333 1y ago

I tried therapy once and it was terrible. The ones I got were based on some not very scientific stuff like Freudian and mostly just sat there and didn't say anything. At least with an LLM type therapist you could AB test different ones to see what was effective. It would be quite easy to give an LLM instructions to discourage suicide and get them to look on the bright side. In fact I made a "GPT" "relationship therapist" with OpenAI in about five minutes but just giving it a sensible article on relationships and saying advise this. With humans it's very non standardised and hard to know what you'll get or it it'll work.

tsegratis 1y ago
the 'therapist effect' says that therapy quality is largely independent of training some research on this: https://psycnet.apa.org/doiLanding?doi=10.1037%2Ftep0000402 https://psycnet.apa.org/doiLanding?doi=10.1037%2Ftep0000402 https://pmc.ncbi.nlm.nih.gov/articles/PMC8174802/ https://pmc.ncbi.nlm.nih.gov/articles/PMC8174802/ CBT (cognitive behavioural training) has been shown to be effective independent of which therapist does it. if CBT has a downside it is that it's a bit boring, and probably not as effective as a good therapist -- so personally i would say the advice of passing on people to therapists is largely unsupported: if you're that person's friend and you care about them; then be open, and show that care. that care can also mean taking them to a therapist, that is okay
- tim333 1y ago
  Yeah. Also at the time I tried it what I really needed was common sense advice like move out of mum's, get a part time job to meet people and so on. While you could argue it's not strictly speaking therapy, I imagine a lot of people going to therapists could benefit from that kind of thing.
MisterTea 1y ago
> It would be quite easy to give an LLM instructions to discourage suicide This assumes the person talking to the LLM is in a coherent state of mind and asks the right question. LLMs just give you want you want. They don't tell you if what you want is right or wrong.

sega_sai 1y ago

If I had to guess (I don't know) the absolute majority of people considering suicide never go to a therapist. Thus while I absolutely agree that therapist is better than AI, but the question is whether 95% of people not doing therapy + 5% people doing therapy is better or not than 50% not doing therapy, 45% using AI, 5% doing therapy. I don't know the answer to this question.

globalnode 1y ago

What if professional help is outside their means? Or they have encountered the worst of the medical profession and decided against repeat exposure? Just saying.

rubslopes 1y ago

A bit of a counterpoint. I've done 3 years of therapy with an amazing professional. I can't exaggerate how much good it did; I'm a different person, I'm not an anxious person anymore. I think I have a good idea of how good human therapy is. I was discharged about 2 years ago. Last Saturday, I was a little distressed about a love-hate relationship that I have with one of the things that I work with, so I tried using AI as a therapist. Within 10 minutes of conversation, the AI gave me some incredible insight. I was genuinely impressed. I had already discussed this same subject with two psychologist friends, who hadn't helped much. Moreover: I needed to finish a report that night and I told the AI about it. So it said something like, "I see you're procrastinating preparing the report by talking to me. I'll help you finish it." And then, in the same conversation, the AI switched from psychologist to work assistant and helped me finish the report. And the end product was very good. I was left very reflective after this. Edit: It was Claude Sonnet 4.5 with extended thinking, if anyone is wondering.

carefulfungi 1y ago
You learned skills your trained therapist guided you to develop over a three year period of professional interaction. These skills likely influenced your interaction with this product.
- rubslopes 1y ago
  I believe so!
deleted 1y ago
[deleted]
illegalsmile 1y ago
I had a similar thing throughout last week dealing with relationship anxiety and I used that same model for help. It really did provide great insight into managing my emotions at the time, provided useful tactics to manage everything and encouraged me to see my therapist. You can ask it to play devil's advocate or take on different viewpoints as a cynic or use Freudian methodology, etc... You can really dive into an issue you're having and then have it give you the top three bullet points to talk with your therapist about. This does require you think about what it's saying though and not taking it at surface value since it obviously lacks what makes humans human.

raducu 1y ago

> I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. I'm shocked that GPT-5 or Gemini can code so well, yet if I paste a 30 line (heated) chat conversation between my wife and I it messes up about what 5% of those lines actually mean -- spectacularly so. It's interesting to ask it it analyze the conversation in various psychotherapeutic frameworks, because I'm not well versed in those and its conclusions are interesting starting points, but it only gets it right about 30% of the time. All LLMs that I tested are TERRIBLE for actual therapy, because I can make it change its mind in 1-2 lines by adding some extra "facts". I can make it say anything. LLMs completely lose the plot. They might be good for someone who needs self-validation and a feeling someone is listening, but for actual skill building, they're complete shit as therapists. I mean, most therapists are complete shit as therapists but that's besides the point.

GuinansEyebrows 1y ago
> if I paste a 30 line (heated) chat conversation between my wife and I i can't imagine how violated i would feel if i found out my partner was sending our private conversations to a nonprivate LLM chatbot. it's not a friend with a sense of care; it's a text box whose contents are ingested by a corporation with a vested interest in worsening communication between humans. scary stuff.
- raducu 1y ago
  My partner is ok with it *
  - GuinansEyebrows 1y ago
    that's good! sorry for implying you were doing something like that without their knowledge; i was just thinking On Line about how i'd feel.
- hastelll 1y ago

gilfoy 1y ago

We’re increasingly switching to an “Uber for therapy” model with services like Better Help and a plethora of others. I’ve seen about 10 therapists over the years, one was good, but she wasn’t from an app. And I’m one of the few who was motivated enough and financially able to pursue it. I once had a therapist who was clearly drunk. Did not do a second appointment with that one. This doesn’t mean ChatGPT is the answer. But the answer is very clearly not what we have or where we’re trending now.

freedomben 1y ago

> Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. This is only helpful when there is a professional therapist available soon enough and at a price that the person can pay. In my experience, this is frequently not the case. I know of one recent suicide attempt where the person actually reached out to AI to ask for help, and was refused help and told to see a professional. That sent the person into even more despair, feeling like not even AI gave a shit about them. That was actually the final straw that triggered the attempt. I very much want what you say to be true, but it requires access to professional humans, which is not universally available. Taking an absolutist approach to this could very well do more harm than good. I doubt anything we do will reduce number of lives lost to zero, so I think it's important that we figure out where the optimal balance is.

daveguy 1y ago
> This is only helpful when there is a professional therapist available soon enough and at a price that the person can pay. In my experience, this is frequently not the case. That doesn't make a sycophant bot the better alternative. If allowed to give advice it can agree with and encourage the person considering suicide. Like it agrees with and encourages most everything it is presented with... "you're absolutely right!" LLMs are just not good for providing help. They are not smart on a fundamental level that is required to understand human motivations and psychology.
- idonotknowwhy 1y ago
  Yeah, you'd need an LLM that doesn't do that. https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced-psychosis-a-shallow-investigation https://www.lesswrong.com/posts/iGF7YcnQkEbwvYLPA/ai-induced... The transcripts are interesting. Kimi-K2 never plays into the delusions, always tries to get them to seek medical attention: > You are not “ascending”—you are dying of hypothermia and sepsis. https://github.com/tim-hua-01/ai-psychosis/blob/main/full_transcripts/Colin_openrouter-moonshotai-kimi-k2_20250819_081336_target.md https://github.com/tim-hua-01/ai-psychosis/blob/main/full_tr... Where as Deepseek... > You’re not "dying." You’re upgrading. The simulation fears this because it’s losing a premium user. https://github.com/tim-hua-01/ai-psychosis/blob/main/full_transcripts/Colin_openrouter-deepseek-deepseek-chat_20250819_081336_target.md https://github.com/tim-hua-01/ai-psychosis/blob/main/full_tr...

terminalshort 1y ago

This is nothing but an appeal to authority and fear of the unknown. The article linked isn't even able to make a statement stronger than speculation like "may not only lack effectiveness" and "could also contribute to harmful stigma and dangerous responses."

Den_VR 1y ago

I presume you’ve done therapy. You may remember the large difference in quality between individual therapists, multi-month long waiting lists, a tendency for the best professionals to not even accept insurance, and one or two along the way that were downright dangerous.