3 ms·

The study coauthor actually seems positive on their potential: 'LLMs potentially have a really powerful future in therapy, but we need to think critically abou

The study coauthor actually seems positive on their potential: 'LLMs potentially have a really powerful future in therapy, but we need to think critically about precisely what this role should be.' And they also mention a previous paper that found high levels of engagement from patients. So, they have potential but currently are giving dangerous advice. It sounds like they are saying a fine tuned therapist model is needed because 'you are a great therapist' prompt, just gives you something that vaguely sounds like a therapist to an outsider. Sounds like an opportunity honestly. Would people value a properly trained therapist enough to pay for it over an existing chatgpt subscription?

deleted 1y ago
[deleted]
AstralStorm 1y ago
What it we gave therapists the same interface as ChatGPT? Mechanical Turk anyone?
Paracompact 1y ago
I expect any LLM, even a fine-tuned one, is going to run into the problem of user-selected conversations that drift ever further away from whatever discourse the original LLM deployers consider appropriate. Actual therapy requires more unsafe topics than regular talk. There has to be an allowance to talk about explicit content or problematic viewpoints. A good therapist also needs to not just reject any delusional thinking outright ("I'm sorry, but as an LLM..."), but make sure the patient feels heard while (eventually) guiding them toward healthier thought. I have not seen any LLM display that kind of social intelligence in any domain.
apical_dendrite 1y ago
High levels of engagement aren't necessarily a good thing. One problem is that the advice is dangerous, but there's an entirely different problem, which is the LLM becoming a crutch that the person relies on because it will always tell them what they want to hear. Most people who call suicide hotlines aren't actually suicidal - they're just lonely or sad and want someone to talk to. The person who answers the phone will talk to them for awhile and validate their feelings, but after a little while they'll politely end the call. The issue is partly that people will monopolize a limited resource, but even if there were an unlimited number of people to answer the phone, it would be fundamentally unhealthy for someone to spend hours a day having someone validate their feelings. It very quickly turns into dependency and it keeps that person in a place where they aren't actually figuring out how to deal with these emotions themselves.

deleted 1y ago

[deleted]

AstralStorm 1y ago

What it we gave therapists the same interface as ChatGPT? Mechanical Turk anyone?

Paracompact 1y ago

I expect any LLM, even a fine-tuned one, is going to run into the problem of user-selected conversations that drift ever further away from whatever discourse the original LLM deployers consider appropriate. Actual therapy requires more unsafe topics than regular talk. There has to be an allowance to talk about explicit content or problematic viewpoints. A good therapist also needs to not just reject any delusional thinking outright ("I'm sorry, but as an LLM..."), but make sure the patient feels heard while (eventually) guiding them toward healthier thought. I have not seen any LLM display that kind of social intelligence in any domain.

apical_dendrite 1y ago

High levels of engagement aren't necessarily a good thing. One problem is that the advice is dangerous, but there's an entirely different problem, which is the LLM becoming a crutch that the person relies on because it will always tell them what they want to hear. Most people who call suicide hotlines aren't actually suicidal - they're just lonely or sad and want someone to talk to. The person who answers the phone will talk to them for awhile and validate their feelings, but after a little while they'll politely end the call. The issue is partly that people will monopolize a limited resource, but even if there were an unlimited number of people to answer the phone, it would be fundamentally unhealthy for someone to spend hours a day having someone validate their feelings. It very quickly turns into dependency and it keeps that person in a place where they aren't actually figuring out how to deal with these emotions themselves.