6 ms·

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they

by j-pb 11mo ago

After having spoken with one of the people there I'm a lot less concerned to be honest. They described it as something akin to an emotional vibrator, that they didn't attribute any sentience to, and that didn't trigger their PTSD that they normally experienced when dating men. If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled, then I don't see any issue.

ArcHound 11mo ago
phew, that's a healthy start. I am still slightly worried about accepting emotional support from a bot. I don't know if that slope is slippery enough to end in some permanent damage to my relationships and I am honestly not willing to try it at all even. That being said, I am fairly healthy in this regard. I can't imagine how it would go for other people with serious problems.
- j-pb 11mo ago
  I completely agree that it is certainly something to be mindful of. It's just that found the people from there were a lot less delusional than the people from e.g. r/artificialsentience, which always believed that AI Moses was giving them some kind of tech revelation though magical alchemical AI symbols.
- netsharc 11mo ago
  A friend broke up with her partner. She said she was using ChatGPT as a therapist. She showed me a screenshot, ChatGPT wrote "Oh [name], I can feel how raw the pain is!". WTF, no you don't bot, you're a hunk of metal!
  - darepublic 11mo ago
    I got a similar synthetic heartfelt response about losing some locally saved files without backup
  - Quarrelsome 11mo ago
    all humans want sometimes, is to be told that what they're feeling is real or not. A sense of validation. It doesn't necessarily matter that much if its an actual person doing it or not.
    - jdub 11mo ago
      Yes, it really, truly does. It's especially helpful if that person has some human experience, or even better, up-to-date training in the study of human psychology. An LLM chat bot has no agency, understanding, empathy, accountability, etc. etc.
avensec 11mo ago
It may not be a concern now, but it comes down to their level of maintaining critical thinking. The risk of epistemic drift, when you have a system that is designed (or reinforced) to empathize with you, can create long-term effects not noticed in any single interaction. Related: "Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it)" ( https://doi.org/10.31234/osf.io/cmy7n_v5 https://doi.org/10.31234/osf.io/cmy7n_v5 )
- j-pb 11mo ago
  I don't disagree that AI psychosis is real, I've met people who believed that they were going to publish at Neurips due to the nonsense ChatGPT told them, that believed that the UI mockup that claude gave then were actually producing insights into it's inner workings instead of just being blinking SVGs, and I even encountered someone participating at a startup event with an Idea that I'm 100% is AI slop. My point was just that the interaction I had from r/myboyfriendisai wans't one of those delusional ones. For that I would take r/artificialsentience as a much better example. That place is absolutely nuts.
  - ArcHound 11mo ago
    Dear god, there's more! I'll need a drink for this one. However, I suspect I have better resistance to schizo posts than emotionally weird posts.
- notpachet 11mo ago
  Wouldn't there necessarily be correlative effects in professional settings a la programming?
  - butlike 11mo ago
    Acceptance of vibe coding prompt-response answers from chatbots without understanding the underlying mechanisms comes to mind as akin to accepting the advice of a chatbot therapist without critically thinking about the response.
  - 11mo ago
DuperPower 11mo ago
[flagged]
- averageRoyalty 11mo ago
  You're being weird and racist, please stop.
  - deleted 11mo ago
    [deleted]
jmcgough 11mo ago
Most people who develop AI psychosis have a period of healthy use beforehand. It becomes very dangerous when a person decreases their time with their real friends to spend more time with the chatbot, as you have no one to keep you in check with what reality is and it can create a feedback loop.
- Nursie 11mo ago
  Wow, are we already in a world where we can say "Most people who develop AI psychosis..." because there are now enough of them to draw meaningful conclusions from? I'm not criticising your comment by the way, that just feels a bit mindblowing, the world is moving very fast at the moment.
  - reverius42 11mo ago
    Yes, Chatbot psychosis been studied, and there's even a wikipedia article on it: https://en.wikipedia.org/wiki/Chatbot_psychosis https://en.wikipedia.org/wiki/Chatbot_psychosis
    - dpark 11mo ago
      From that article, it doesn’t sound like it’s been studied at all. It sounds like at the current stage it’s hypothesis + anecdotes.
aprilthird2021 11mo ago
> If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled, then I don't see any issue. Surely something that can be good can also be bad at the same time? Like the same way wrapping yourself in bubble wrap before leaving the house will provably reduce your incidence of getting scratched and cut outside, but there's also reasons you shouldn't do that...
jrjeksjd8d 11mo ago
The problem is that chatbots don't provide emotional support. To support someone with PTSD you help them gradually untangle the strong feelings around a stimulus and develop a less strong response. It's not fast and it's not linear but it requires a mix of empathy and facilitation. Using an LLM for social interaction instead of real treatment is like taking heroin because you broke your leg, and not getting it set or immobilized.
- pixl97 11mo ago
  >instead of real treatment As yes, because America is well known for actually providing that at a reasonable price and availability...
  - mrguyorama 11mo ago
    Then we should fix that, instead of dumping 3 trillion dollars on grifters and some of the worst human beings we have produced.
    - pixl97 11mo ago
      We should fix 100 things first... we won't. Capitalism is king and we'll stack the bodies high on his throne first.
- scotty79 11mo ago
  > To support someone with PTSD you help them gradually untangle the strong feelings around a stimulus and develop a less strong response. It's about replaying frightening thoughts and activities in safe environment. When the brain notices they don't trigger suffering it fears them less in the future. Chatbot can provide such safe environment.
  - jrjeksjd8d 11mo ago
    > Chatbot can provide such safe environment. It really can't. No amount of romancing a sycophantic robot is going to prepare someone to actually talk to a human being.
ungreased0675 11mo ago
That sounds very disturbing and likely to be harmful to me.
arresin 11mo ago
Why do so many women have ptsd from dating?
- H8crilA 11mo ago
  "PTSD" is going through the same semantic inflation as the word "trauma". Or perhaps you could say the common meaning is an increasingly more inflated version of the professional meaning. Not surprising since these two are sort of the same thing. BTW, a more relevant word here is schizoid / schizoidism, not to be confused with schizophrenia. Or at least very strongly avoidant attachment style.
- whatsupdog 11mo ago
  [flagged]
  - jrflowers 11mo ago
    > nobody is yet ready to have a serious discussion about this. There are a ton of people that are happy to have serious discussions about how their superior knowledge of biology gives them oracular insight into the minds of women. These discussions happen every day in Discord chats full of pubescent boys, Discord chats full of young men, and YouTube comments sections full of older men.
    - deleted 11mo ago
      [deleted]
  - frm88 11mo ago
    Source?
  - deleted 11mo ago
    [deleted]
  - H8crilA 11mo ago
    From what I'm seeing the boys are getting much more damage. Even your comment smells a bit of projection.
probably_wrong 11mo ago
I think there's a difference between "support" and "enabling". It is well documented that family members of someone suffering from an addiction will often do their best at shielding the person from the consequences of their acts. While well-intentioned ("If I don't pay this debt they'll have an eviction on their record and will never find a place again"), these acts prevent the addict from seeking help because, without consequences, the addict has no reason to change their ways. Actually helping them requires, paradoxically, to let them hit rock bottom. An "emotional vibrator" that (for instance) dampens that person's loneliness is likely to result in that person taking longer (if ever) to seek help for their PTSD. IMHO it may look like help when it's actually enabling them.
- scotty79 11mo ago
  Right, next time you have a headache don't let yourself be enabled by aspirin.

by j-pb 11mo ago

ArcHound 11mo ago

phew, that's a healthy start. I am still slightly worried about accepting emotional support from a bot. I don't know if that slope is slippery enough to end in some permanent damage to my relationships and I am honestly not willing to try it at all even. That being said, I am fairly healthy in this regard. I can't imagine how it would go for other people with serious problems.

j-pb 11mo ago
I completely agree that it is certainly something to be mindful of. It's just that found the people from there were a lot less delusional than the people from e.g. r/artificialsentience, which always believed that AI Moses was giving them some kind of tech revelation though magical alchemical AI symbols.
netsharc 11mo ago
A friend broke up with her partner. She said she was using ChatGPT as a therapist. She showed me a screenshot, ChatGPT wrote "Oh [name], I can feel how raw the pain is!". WTF, no you don't bot, you're a hunk of metal!
- darepublic 11mo ago
  I got a similar synthetic heartfelt response about losing some locally saved files without backup
- Quarrelsome 11mo ago
  all humans want sometimes, is to be told that what they're feeling is real or not. A sense of validation. It doesn't necessarily matter that much if its an actual person doing it or not.
  - jdub 11mo ago
    Yes, it really, truly does. It's especially helpful if that person has some human experience, or even better, up-to-date training in the study of human psychology. An LLM chat bot has no agency, understanding, empathy, accountability, etc. etc.

avensec 11mo ago

It may not be a concern now, but it comes down to their level of maintaining critical thinking. The risk of epistemic drift, when you have a system that is designed (or reinforced) to empathize with you, can create long-term effects not noticed in any single interaction. Related: "Delusions by design? How everyday AIs might be fuelling psychosis (and what can be done about it)" ( https://doi.org/10.31234/osf.io/cmy7n_v5 https://doi.org/10.31234/osf.io/cmy7n_v5 )

j-pb 11mo ago
I don't disagree that AI psychosis is real, I've met people who believed that they were going to publish at Neurips due to the nonsense ChatGPT told them, that believed that the UI mockup that claude gave then were actually producing insights into it's inner workings instead of just being blinking SVGs, and I even encountered someone participating at a startup event with an Idea that I'm 100% is AI slop. My point was just that the interaction I had from r/myboyfriendisai wans't one of those delusional ones. For that I would take r/artificialsentience as a much better example. That place is absolutely nuts.
- ArcHound 11mo ago
  Dear god, there's more! I'll need a drink for this one. However, I suspect I have better resistance to schizo posts than emotionally weird posts.
notpachet 11mo ago
Wouldn't there necessarily be correlative effects in professional settings a la programming?
- butlike 11mo ago
  Acceptance of vibe coding prompt-response answers from chatbots without understanding the underlying mechanisms comes to mind as akin to accepting the advice of a chatbot therapist without critically thinking about the response.
- 11mo ago

DuperPower 11mo ago

[flagged]

averageRoyalty 11mo ago
You're being weird and racist, please stop.
- deleted 11mo ago
  [deleted]

jmcgough 11mo ago

Most people who develop AI psychosis have a period of healthy use beforehand. It becomes very dangerous when a person decreases their time with their real friends to spend more time with the chatbot, as you have no one to keep you in check with what reality is and it can create a feedback loop.

Nursie 11mo ago
Wow, are we already in a world where we can say "Most people who develop AI psychosis..." because there are now enough of them to draw meaningful conclusions from? I'm not criticising your comment by the way, that just feels a bit mindblowing, the world is moving very fast at the moment.
- reverius42 11mo ago
  Yes, Chatbot psychosis been studied, and there's even a wikipedia article on it: https://en.wikipedia.org/wiki/Chatbot_psychosis https://en.wikipedia.org/wiki/Chatbot_psychosis
  - dpark 11mo ago
    From that article, it doesn’t sound like it’s been studied at all. It sounds like at the current stage it’s hypothesis + anecdotes.

aprilthird2021 11mo ago

> If AI can provide emotional support and an outlet for survivors who would otherwise not be able to have that kind of emotional need fulfilled, then I don't see any issue. Surely something that can be good can also be bad at the same time? Like the same way wrapping yourself in bubble wrap before leaving the house will provably reduce your incidence of getting scratched and cut outside, but there's also reasons you shouldn't do that...

jrjeksjd8d 11mo ago

The problem is that chatbots don't provide emotional support. To support someone with PTSD you help them gradually untangle the strong feelings around a stimulus and develop a less strong response. It's not fast and it's not linear but it requires a mix of empathy and facilitation. Using an LLM for social interaction instead of real treatment is like taking heroin because you broke your leg, and not getting it set or immobilized.

pixl97 11mo ago
>instead of real treatment As yes, because America is well known for actually providing that at a reasonable price and availability...
- mrguyorama 11mo ago
  Then we should fix that, instead of dumping 3 trillion dollars on grifters and some of the worst human beings we have produced.
  - pixl97 11mo ago
    We should fix 100 things first... we won't. Capitalism is king and we'll stack the bodies high on his throne first.
scotty79 11mo ago
> To support someone with PTSD you help them gradually untangle the strong feelings around a stimulus and develop a less strong response. It's about replaying frightening thoughts and activities in safe environment. When the brain notices they don't trigger suffering it fears them less in the future. Chatbot can provide such safe environment.
- jrjeksjd8d 11mo ago
  > Chatbot can provide such safe environment. It really can't. No amount of romancing a sycophantic robot is going to prepare someone to actually talk to a human being.

ungreased0675 11mo ago

That sounds very disturbing and likely to be harmful to me.

arresin 11mo ago

Why do so many women have ptsd from dating?

H8crilA 11mo ago
"PTSD" is going through the same semantic inflation as the word "trauma". Or perhaps you could say the common meaning is an increasingly more inflated version of the professional meaning. Not surprising since these two are sort of the same thing. BTW, a more relevant word here is schizoid / schizoidism, not to be confused with schizophrenia. Or at least very strongly avoidant attachment style.
whatsupdog 11mo ago
[flagged]
- jrflowers 11mo ago
  > nobody is yet ready to have a serious discussion about this. There are a ton of people that are happy to have serious discussions about how their superior knowledge of biology gives them oracular insight into the minds of women. These discussions happen every day in Discord chats full of pubescent boys, Discord chats full of young men, and YouTube comments sections full of older men.
  - deleted 11mo ago
    [deleted]
- frm88 11mo ago
  Source?
- deleted 11mo ago
  [deleted]
- H8crilA 11mo ago
  From what I'm seeing the boys are getting much more damage. Even your comment smells a bit of projection.

probably_wrong 11mo ago

I think there's a difference between "support" and "enabling". It is well documented that family members of someone suffering from an addiction will often do their best at shielding the person from the consequences of their acts. While well-intentioned ("If I don't pay this debt they'll have an eviction on their record and will never find a place again"), these acts prevent the addict from seeking help because, without consequences, the addict has no reason to change their ways. Actually helping them requires, paradoxically, to let them hit rock bottom. An "emotional vibrator" that (for instance) dampens that person's loneliness is likely to result in that person taking longer (if ever) to seek help for their PTSD. IMHO it may look like help when it's actually enabling them.

scotty79 11mo ago
Right, next time you have a headache don't let yourself be enabled by aspirin.