3 ms·

How is anyone versed in LLM technical details surprised by this? They are very useful algorithms which solve for document generation. That's it. LLM's do not

How is anyone versed in LLM technical details surprised by this? They are very useful algorithms which solve for document generation. That's it. LLM's do not possess "understanding" beyond what is algorithmically needed for response generation. LLM's do not possess shared experiences people have in order to potentially relate to others in therapy sessions as LLM's are not people. LLM's do not possess professional experience needed for successful therapy, such as knowing when to not say something as LLM's are not people. In short, LLM's are not people.

never_inline 1y ago
1. Ars Technica's (OP website) audience includes tech enthusiast people who don't necessarily have a mental model of LLMs, instruction tuning or RLHF. 2. why would this "study" exist? - for the reason computer science academics conduct study on whether LLMs are empirically helpful in software engineering. (The therapy industrial complex would also have some reasons to sponsor this kind of a research, unlike SWE productivity studies where the incentive is usually the opposite.)
- AdieuToLogic 1y ago
  Both great points. For the record, my initial question was more rhetorical in nature, but I am glad you took the time to share your thoughts as it gave me (and hopefully others) perspectives to think about.
coliveira 1y ago
Computer scientists are, in part, responsible for the public confusion about what LLMs are and can do. Tech investors and founders, however, are the biggest liars and BS peddlers when they go around saying irresponsible things like LLMs are on the verge of becoming "conscious" and other unfounded and impossible things (for LLMs). It's not a surprise that many people believe that you can have a personal "conversation" with a tool that generates text based on statistical analysis of previous data.
wongarsu 1y ago
Self-help books help people (at least sometimes). In an ideal world an LLM could be like the ultimate self-help book, dispensing the advice and anecdotes you need in your current situation. It doesn't need to be human to be beneficial. And at least from first-order principles it's not at all obvious that they are more harmful than helpful. To me it appears that most of the harm is in the overly affirming sycophant personally most of them are trained into, which is not a necessary or even natural feature of LLMs at all Not that the study wouldn't be valuable even if it was obvious
- deleted 1y ago
  [deleted]
- Retric 1y ago
  Self-help books are designed to sell, they’re not particularly useful on their own. LLM’s are plagued by poor accuracy so they preform terribly in any situation where inaccuracies have serious downsides and there is no process validating the output. This is a theoretical limitation of the underlying technology, not something better training can fix.
  - Dylan16807 1y ago
    I don't think that argument is solid enough. "serious downsides" doesn't always mean "perform terribly". Most unfixable flaws can be worked around with enough effort and skill.
    - Retric 1y ago
      At scale it does when “serious downsides” are both common and actually serious like death. Suppose every time you got into your car an LLM was going to recreate the all safety critical software from an identical prompt but using slightly randomized output. Would you feel comfortable with such an arrangement? > Most unfixable flaws can be worked around with enough effort and skill. Not when the underlying idea is flawed enough. You can’t get from the earth to the moon by training yourself to jump that distance, I don’t care who you’re asking to design your exercise routine.
BeetleB 1y ago
> In short, LLM's are not people. Not really sure that is relevant in the context of therapy. > LLM's do not possess shared experiences people have in order to potentially relate to others in therapy sessions as LLM's are not people. Licensed therapists need not possess a lot of shared experiences to effectively help people. > LLM's do not possess professional experience needed for successful therapy, such as knowing when to not say something as LLM's are not people. Most people do not either. That an LLM is not a person doesn't seem particularly notable or relevant here. Your comment is really saying: "You need to be a person to have the skills/ability to do therapy" That's a bold statement.
- ClumsyPilot 1y ago
  > You need to be a person to have the skills Generally a non-person doesn’t have skills, it’s a pretty likely to be true statement even if made on a random subject.
  - BeetleB 1y ago
    Once again: The argument appears to be "LLMs cannot be therapists because they are LLMs." Circular logic. > Generally a non-person doesn’t have skills, A semantic argument isn't helpful. A chess grandmaster has a lot of skill. A computer doesn't (according to you). Yet, the computer can beat the grandmaster pretty much every time. Does it matter that the computer had no skill, and the grandmaster did? That they don't have "skill" does not seem particularly notable in this context. It doesn't help answer "Is it possible to get better therapy from an LLM than from a licensed therapist?"
    - crashabr 1y ago
      > Does it matter that the computer had no skill, and the grandmaster did? Yes it does when pondering about the transferability of the skills mobilized to achieve the result (grandmaster status) to other domains.
- padolsey 1y ago
  >> LLM's do not possess professional experience needed for successful therapy, such as knowing when to not say something as LLM's are not people. > Most people do not either. That an LLM is not a person doesn't seem particularly notable or relevant here. Of relevance I think: LLMs by their nature will often keep talking. They are functions that cannot return null. They have a hard time not using up tokens. Humans however can sit and listen and partake in reflection without using so many words. To use the words of the parent comment: trained humans have the pronounced ability to _not_ say something.
mise_en_place 1y ago
The average person in 2025 has been so thoroughly stripped of their humanity, that even a simulacrum of a human is enough.
gonzobonzo 1y ago
It feels like 95% of the people are responding to the headline instead of reading the article. From the article: > The Stanford research tested controlled scenarios rather than real-world therapy conversations, and the study did not examine potential benefits of AI-assisted therapy or cases where people have reported positive experiences with chatbots for mental health support. In an earlier study, researchers from King's College and Harvard Medical School interviewed 19 participants who used generative AI chatbots for mental health and found reports of high engagement and positive impacts, including improved relationships and healing from trauma. ** > "This isn't simply 'LLMs for therapy is bad,' but it's asking us to think critically about the role of LLMs in therapy," Haber told the Stanford Report, which publicizes the university's research. "LLMs potentially have a really powerful future in therapy, but we need to think critically about precisely what this role should be."

never_inline 1y ago

1. Ars Technica's (OP website) audience includes tech enthusiast people who don't necessarily have a mental model of LLMs, instruction tuning or RLHF. 2. why would this "study" exist? - for the reason computer science academics conduct study on whether LLMs are empirically helpful in software engineering. (The therapy industrial complex would also have some reasons to sponsor this kind of a research, unlike SWE productivity studies where the incentive is usually the opposite.)

AdieuToLogic 1y ago
Both great points. For the record, my initial question was more rhetorical in nature, but I am glad you took the time to share your thoughts as it gave me (and hopefully others) perspectives to think about.

coliveira 1y ago

Computer scientists are, in part, responsible for the public confusion about what LLMs are and can do. Tech investors and founders, however, are the biggest liars and BS peddlers when they go around saying irresponsible things like LLMs are on the verge of becoming "conscious" and other unfounded and impossible things (for LLMs). It's not a surprise that many people believe that you can have a personal "conversation" with a tool that generates text based on statistical analysis of previous data.

wongarsu 1y ago

Self-help books help people (at least sometimes). In an ideal world an LLM could be like the ultimate self-help book, dispensing the advice and anecdotes you need in your current situation. It doesn't need to be human to be beneficial. And at least from first-order principles it's not at all obvious that they are more harmful than helpful. To me it appears that most of the harm is in the overly affirming sycophant personally most of them are trained into, which is not a necessary or even natural feature of LLMs at all Not that the study wouldn't be valuable even if it was obvious

deleted 1y ago
[deleted]
Retric 1y ago
Self-help books are designed to sell, they’re not particularly useful on their own. LLM’s are plagued by poor accuracy so they preform terribly in any situation where inaccuracies have serious downsides and there is no process validating the output. This is a theoretical limitation of the underlying technology, not something better training can fix.
- Dylan16807 1y ago
  I don't think that argument is solid enough. "serious downsides" doesn't always mean "perform terribly". Most unfixable flaws can be worked around with enough effort and skill.
  - Retric 1y ago
    At scale it does when “serious downsides” are both common and actually serious like death. Suppose every time you got into your car an LLM was going to recreate the all safety critical software from an identical prompt but using slightly randomized output. Would you feel comfortable with such an arrangement? > Most unfixable flaws can be worked around with enough effort and skill. Not when the underlying idea is flawed enough. You can’t get from the earth to the moon by training yourself to jump that distance, I don’t care who you’re asking to design your exercise routine.

BeetleB 1y ago

> In short, LLM's are not people. Not really sure that is relevant in the context of therapy. > LLM's do not possess shared experiences people have in order to potentially relate to others in therapy sessions as LLM's are not people. Licensed therapists need not possess a lot of shared experiences to effectively help people. > LLM's do not possess professional experience needed for successful therapy, such as knowing when to not say something as LLM's are not people. Most people do not either. That an LLM is not a person doesn't seem particularly notable or relevant here. Your comment is really saying: "You need to be a person to have the skills/ability to do therapy" That's a bold statement.

ClumsyPilot 1y ago
> You need to be a person to have the skills Generally a non-person doesn’t have skills, it’s a pretty likely to be true statement even if made on a random subject.
- BeetleB 1y ago
  Once again: The argument appears to be "LLMs cannot be therapists because they are LLMs." Circular logic. > Generally a non-person doesn’t have skills, A semantic argument isn't helpful. A chess grandmaster has a lot of skill. A computer doesn't (according to you). Yet, the computer can beat the grandmaster pretty much every time. Does it matter that the computer had no skill, and the grandmaster did? That they don't have "skill" does not seem particularly notable in this context. It doesn't help answer "Is it possible to get better therapy from an LLM than from a licensed therapist?"
  - crashabr 1y ago
    > Does it matter that the computer had no skill, and the grandmaster did? Yes it does when pondering about the transferability of the skills mobilized to achieve the result (grandmaster status) to other domains.
padolsey 1y ago
>> LLM's do not possess professional experience needed for successful therapy, such as knowing when to not say something as LLM's are not people. > Most people do not either. That an LLM is not a person doesn't seem particularly notable or relevant here. Of relevance I think: LLMs by their nature will often keep talking. They are functions that cannot return null. They have a hard time not using up tokens. Humans however can sit and listen and partake in reflection without using so many words. To use the words of the parent comment: trained humans have the pronounced ability to _not_ say something.

mise_en_place 1y ago

The average person in 2025 has been so thoroughly stripped of their humanity, that even a simulacrum of a human is enough.

gonzobonzo 1y ago

It feels like 95% of the people are responding to the headline instead of reading the article. From the article: > The Stanford research tested controlled scenarios rather than real-world therapy conversations, and the study did not examine potential benefits of AI-assisted therapy or cases where people have reported positive experiences with chatbots for mental health support. In an earlier study, researchers from King's College and Harvard Medical School interviewed 19 participants who used generative AI chatbots for mental health and found reports of high engagement and positive impacts, including improved relationships and healing from trauma. ** > "This isn't simply 'LLMs for therapy is bad,' but it's asking us to think critically about the role of LLMs in therapy," Haber told the Stanford Report, which publicizes the university's research. "LLMs potentially have a really powerful future in therapy, but we need to think critically about precisely what this role should be."