3 ms·
Therapy is one of the most dangerous applications you could imagine for an LLM. Exposing people who already have mental health issues, who are extremely vulnera
by lowsong 1y ago
Therapy is one of the most dangerous applications you could imagine for an LLM. Exposing people who already have mental health issues, who are extremely vulnerable to manipulation or delusions to a machine that's designed to to produce human-like text is so obviously risky it boggles the mind that anyone would even consider it.
- ltbarcly3 1y agoEveryone is already using LLMs for therapy. The should argument is moot.
- lowsong 1y agoThen it needs to be banned and prosecuted as medical malpractice. The "should" argument is very much not moot.
- ltbarcly3 1y agoProsecute the person using the LLM or the LLM?
- lowsong 1y agoI'd hope it was obvious I wasn't referring to the user. You shouldn't prosecute a person using a tool that wasn't intended for that function on themselves, in the same way you can't prosecute a person for performing 'surgery' on themselves without medical knowledge or the right tools. I mean we should prosecute companies or individuals positioning LLMs as therapy tools, suggesting that they can be replacements for therapy, or are appropriate for medical use at all. It's medical malpractice and misrepresentation, pure and simple.
- ltbarcly3 1y agoLiterally nobody is doing what you are saying. People are using LLMs for therapy because that's an immediate and obvious thing to do.
- lowsong 1y ago> Literally nobody is doing what you are saying. The paper we're discussing is literally about this. It's the second sentence in the abstract: "In this paper, we investigate the use of LLMs to replace mental health providers, a use case promoted in the tech startup and research space."