3 ms·
I don't think you should be getting mental health advice from LLMs, it has been shown that their sycophantic nature reinforces your own diagnoses not that of a
by ehnto 1y ago
I don't think you should be getting mental health advice from LLMs, it has been shown that their sycophantic nature reinforces your own diagnoses not that of a professionals. Using an LLM in a field you're knowledgeable in would make you think twice about their accuracy in fields you are unable to know the correctness of personally. It will all sound very convincing, but may be a fundamentally flawed diagnoses.
Another way of thinking about, a non-trivial amount of training data is internet comments and blogs, which have an alarming amount of self diagnoses, non-professional diagnoses, and totally fabricated facts about mental illness.
- hyperadvanced 1y agoAgree, pointing to TikTok and LLMs as a “disrupter” to the prevailing psychotherapeutic industry is crazy town. Those media reify / amplify the exact same crazy of their training set into absurdity.
- bjornsing 1y agoTo be clear I have nothing against the prevailing psychotherapeutic industry. I’m very thankful for that part of the mental health community. I think it’s rather well-aligned with TikTok too, btw.
- hyperadvanced 1y agoI’m fairly opposed to both, but respect the difference of opinion
- true_religion 1y agoI agree. I was using an LLM to test some assumptions I was making about a chemical combination, and although it's reasoning was logical it made an error about how pH works when you combine two acids of similar strength then it held onto that incorrect conclusion for the rest of the chat. Since I wasn't asking anything related to pH, I skimmed past that section and didn't notice the error until much later in the chat when the LLM decided to build upon erroneous reasoning. I think someone who hadn't studied chemistry would've relied on its' answer since all the rest of the logic would've been correct if the solution really did become more acidic.
- bjornsing 1y agoI’m well aware of the risks. But I think you have an unrealistically rosy picture of professionals’ diagnoses.
- dxdm 1y agoReplacing expert opinion with engagement-baiting content from feed machines and hallucinating matrices seems to me is a part of the problem, and not the solution. Especially using TikTok to try and improve mental health issues seems a bit like trying to fight fire with a (edit: spelling) hose of jet fuel.
- bjornsing 1y agoSometimes it’s exactly the right medicine. :)
- queenkjuul 1y agoI very much think you're wrong
- ehnto 1y agoThere's certainly no shortage of issues with the mental health professional field, that much is true. I hope you are doing well regardless. I suspect we're going to see this story play out a lot, where the limitations of AI should be a major limiting factor, but people will get results anyway. I imagine it will rely a lot on the pilot, and how well they understand those limitations. Perhaps the bigger risks are those without good understanding of LLMs who just treat it like an all knowing expert human.
- bjornsing 1y agoIf you are only capable of blind trust or distrust then there’s a huge difference between human experts and LLMs (and it’s of course wise to put your blind trust in a human, not an LLM). But if you have more of a ”trust no one” or ”trust but verify” mentality in general then it’s not so clear cut. The LLMs have their advantages. For example every chat with an LLM is an independent sample, whereas once a doctor has diagnosed you it’s very hard to get them to consider evidence contradictory to that diagnosis.
- icehawk 1y agoI am really beginning to think that the Gell-Mann amnesia effect has a shining example in LLM usage. Its obvious there's no intelligence behind it when you make it try to do something with data that gets parsed by a computer.
- lelanthran 1y ago> I don't think you should be getting mental health advice from LLMs, it has been shown that their sycophantic nature reinforces your own diagnoses not that of a professionals. And therapists don't? All the data used in diagnosis is self-reported feelings. All the progress made from seeing a therapist is also self-reported feelings. Random chance might make more accurate diagnoses than any based on self-reported feelings.
- ehnto 1y agoTo an extent, but the words you give an LLM are the entirety of what it has to go on, and again they are sycophantic. If you tell it you have depression, and you insist, it will do its best to agree. In the exact same conversation, you could later convince it you have schizophrenia instead. A human wouldn't buy it. A trained psychologist is going to use their procedural training to diagnose you. Not text input, they are asking you questions with subtext, and you may not even realise what they learned about you from your answer. With an LLM, you are loading it's context with your world view and it will go off that.