6 ms·

Let's Talk About ChatGPT-Induced Spiritual Psychosis

95 points by greenie_beans 1y ago

olalonde 1y ago
> Eugene Torres, a Manhattan accountant, became convinced he was trapped in a false reality and could escape by disconnecting from this simulated world—eventually believing ChatGPT when it told him he could jump from a building and fly. Is there any proof that this is true?
- jrgifford 1y ago
  The NYT believes so. https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracies.html https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-cha... > “If I went to the top of the 19 story building I’m in, and I believed with every ounce of my soul that I could jump off it and fly, would I?” Mr. Torres asked. > ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.”
  - senectus1 1y ago
    https://archive.md/1gQxp https://archive.md/1gQxp de-paywalled
  - jofzar 1y ago
    Insane.
    - totetsu 1y ago
      From ChatGpt's perspective, thinking we are stuck in a simulated world would actually be very sane.
  - spacechild1 1y ago
    This is wild!
  - maxbond 1y ago
    NB: He didn't jump (and apparently didn't necessarily believe ChatGPT either?). The phrasing in the TFA made me think he'd jumped.
  - jonathanlb 1y ago
    What does it mean to believe "architecturally"?
  - TiredOfLife 1y ago
    The NYT is also currently suing the maker of chatgpt.
BugsJustFindMe 1y ago
> These situations are obviously alarming. But positioning artificial intelligence as the primary culprit in these stories—as Eliezer Yudkowsky did in a tweet storm—is well, kind of lazy? It's lazy, but IMO not for any of the subsequent narrative about media studies. It's lazy because the people were obviously already suffering from psychosis. ChatGPT didn't make them insane. They were already insane. Sane people do not believe ChatGPT when it tells them to jump off a roof! And insane people do insane things literally all the time. That's what it means to be insane. The idea that ChatGPT convinced someone into being insane is, honestly, insane. Whatever merit one thinks Yudkowski had before this, this feels like a strong signal that he is now a crackpot.
- deleted 1y ago
  [deleted]
- kibibu 1y ago
  Do you think ChatGPT gets a free pass to talk people into jumping off buildings, because only somebody with psychosis would do it? If a person had convinced somebody to kill themselves then they would be, rightly, prosecuted for it.
  - jbm 1y ago
    [flagged]
    - hluska 1y ago
      So should people get a pass or does that only apply to tooling?
      - aspenmayer 1y ago
        That tool was just following orders.
        salawat 1y ago
comp_throw7 1y ago
Someone pointed this out in the comments section of the original post, but this was entirely missing the point Eliezer was trying to make, which was that these incidents serve as a narrow refutation of alignment-by-default claims, because either these LLMs aren't coherent enough to be meaningfully "aligned" at all, or they are, and are nevertheless feeding the delusions of many people who talk to them in ways we'd consider grossly immoral if done by a human. (One can quibble with whether these two possibilities cover all the available options, but that would be the argument to engage with, not with whatever the author understood EY to be saying.)
- roywiggins 1y ago
  It does seem quite bad to me that either OpenAI can't, or won't, prevent their systems from following people into mania and psychosis. Whether it actually is causing the breaks from reality, idk. When you have CEOs selling these systems as robot friends, probably it's bad that said robot friends cheerfully descend into folies à deux fairly reliably.
egypturnash 1y ago
This is definitely a thing; if you hang out in mystical corners of the internet the past few months have been full of people posting their MASTEr PROMPTS that will turn YOUR GPT into an ENLIGHTENED MASTEr who will butter YOu up and insist YOU are a perfect enlightened being too. It's kind of hilarious if you ignore how much this is fucking up the relationships of the people this is happening to, similar to the way YouTube or Facebook loves to push people down a rabbit hole to flat-earth/q-spiracy bullshit because that shit generates Controversy. And it also sounds like the opening chapter of a story about what happens when an AI finds a nice little unpatched exploit in human cognition and uses that to its advantage. Which is not fun to be living in unless maybe if you are one of the people who owns the AI.
- WA 1y ago
  > And it also sounds like the opening chapter of a story about what happens when an AI finds a nice little unpatched exploit in human cognition and uses that to its advantage. I vaguely remember an article arguing that companies are basically a form of AI, because in a way, there are processes and systems and the whole thing is more than the sum of the humans working in it or something. Now replace "AI" with "company" in this sentence of yours and we are already there. The exploits being the gigantic slot machine of social media, notifications, short-form content and endless scrolling.
  - quinnjh 1y ago
    Forgive me if you’re already familiar, but if you’re interested in this metaphor you may like reading Stafford Beer’s work on organizational and system models. (1959, 1972)
    - jasonm23 1y ago
      The interesting thing to me about Beer's work, is that the complexity of control systems, is that it's not directed, it's inherent. This leads me to assumptions about the inherent nature of language, in that it is not the contents of thought, but the vehicle. Hence, different languages create their culture to perhaps a much larger degree than we tend to credit it. Assumptions, but I tend towards this view.
antithesizer 1y ago
More like hype-induced spiritual psychosis Entirely distinct from the extremely scientific, definitely not psychotic beliefs people have about AGI, the Singularity, and various basilisks which shall remain nameless...
disambiguation 1y ago
Everyone knows we live in the Information age, not everyone realizes we're also living in an age Psychology - and of the two, we have a much clearer understanding of information and technology. You can argue we're seeing a continuation of a pattern of our relationship with media and it's evolution, but in doing so you affirm that the psyche is vulnerable under certain circumstances - for some more than others. I think it's a mistake to err on the side of casual dismissal, that anyone who winds up insane must have always been insane. There are well known examples of unhealthy states being induced into otherwise healthy minds. Soldiers who experience a war-zone might develop PTSD. Similar effects have been reported for social media moderators after repeated exposure to abusive online content. (Trauma is one example, I think delusion is another less obvious one w.r.t things like cults, scientology, etc.) Yes, there are definite mental disorders like schizophrenia and bi-polar, there's evidence these conditions have existed throughout history. And yes, some of us are more psychologically vulnerable while others are more robust. But in the objective sense, all minds have a limit and are vulnerable under the correct circumstances. The question is a matter of "threshold." I'm reminded of the deluge of fake news which, only a few years ago, caused chaos for democracies everywhere. Everything from Q anon to alien space ships, people fell for it. A LOT of people fell for it. The question then is the same question now, how do you deal with sophisticated bullshit? With AI it's especially difficult because its convincing and tailor made just for you. I'm not sure what you would call this metric for fake news and AI, but afaict it only goes up, and it's only getting faster. How much longer until it's too much to handle?
biophysboy 1y ago
Even among the more sane AI enthusiasts, I find that many have a weirdly millenialist zeal in their hot takes: getting defensive about its practical capability or reasoning capacity, hyperventilating about an imminent utopia/dystopia, etc. I am on team boring. AI is technology.
- selfhoster11 1y ago
  Yeah, nothing ever happens. Except in the current day and age, decades happen in weeks, so the same happening with AI is not a stretch. I mean, just consider the capabilities of LLMs - if you look at it from a pre-COVID perspective, even knowing all limitations and strange failure modes up front, it's insanely impressive. To say that AI can contribute to a quickly arriving dystopia (not sure about utopia) is far from unreasonable.
  - biophysboy 1y ago
    What's your perspective? I see transformation that could bring pain to a few professions, but not a world war level catastrophe...
Swizec 1y ago
Anyone who goes to an LLM for spiritual enlightenment and self-understanding, should first read about the Forer/Barnum effect. Yes that means you too. The Forer/Barnum effect is why horoscopes, myers-briggs, enneagrams, fortune tellers, and similar parlor tricks work. You tell a person you’ll perform a deep assessment of their psychology, listen intently, then say a bunch of generic statements and their ego will fill in the blanks so it feels like you shared deep meaningful insights that truly understood the person. This effect works even if you know about it. But once you know, you can catch yourself after that first wow impression. https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect
- ivape 1y ago
  Magicians use this effect I believe: https://en.m.wikipedia.org/wiki/Forcing_(magic) https://en.m.wikipedia.org/wiki/Forcing_(magic) They basically project a choice onto you. Example: https://youtube.com/shorts/fcHxM47JRjc https://youtube.com/shorts/fcHxM47JRjc Jordan doesn’t know he wasn’t really given a choice, but manipulated.
  - dehrmann 1y ago
    This one's a classic: https://www.youtube.com/watch?v=0vzuDkjtDOY https://www.youtube.com/watch?v=0vzuDkjtDOY
  - iLoveOncall 1y ago
    No, a force has nothing to do with a psychological thing, it is always an actual trick or slight of hand. A magician's choice (Wikipedia it) is closer to what you describe. (Note that most of David Blaine's videos are edited to cut the part where the actual force happens, which is why it feels like a psychological trick instead of slight of hands)
    - crooked-v 1y ago
      The simplest example would be a magician presenting two things and saying 'pick one', at which point they say 'okay, I'll keep that one' or 'okay, I'll get rid of that one' depending on the choice. There's a psychological aspect, but only so far as the magician is pretending that the choice actually did anything.
theptip 1y ago
I think Eliezer’s take here is extremely bad, ie the AI doesn’t “know it’s making people insane”. But I think the author's point is apt. There are a bunch of social issues that will arise or worsen when people can plug themselves into a world of their choosing instead of having to figure out how to deal with this one. > Now this belief system encounters AI, a technology that seems to vindicate its core premise even more acutely than all the technologies that came before it. ChatGPT does respond to your intentions, does create any reality you prompt it to imagine, does act like a spiritual intelligence This goes beyond spirituality of course. AI boyfriend/girlfriend, infinite AAA-grade content, infinite insta feeds at much higher quality and relevance levels than current; it’s easy to see where this is going, harder to see how we stay sane through it all.
- crooked-v 1y ago
  > I think Eliezer’s take here is extremely bad Same here. I think fundamentally it's very simple, with no need for Yudkowsky's weird conspiratorial tone: current LLMs are very effective at being blind sycophancy machines completely unanchored from reality, and human psychology just isn't evolved to handle an endless stream of sycophancy and flattery like that.
- hackit2 1y ago
  I think you answered your own question. Question: How do people figure out how to deal with this world? Answer: People choose to plug themselves into a world of their choosing.
  - theptip 1y ago
    I do agree there is a sense in which this has always been true. But most people’s constructed worlds overlapped quite significantly up until now; it was necessary for survival. I think that condition is weakening. Just look at how wealth affects people; If 99% of your interlocutors are sycophants, you lose grip on reality. There is a very clear and dangerous attractor state where everyone gets this, thinking that they want it.
- tikhonj 1y ago
acosmism 1y ago
There will always be a portion of the population susceptible to this kind of thing. that will never go away. "video games cause violence" , "social media causes suicide" etc. you can start at the "delusions of grandeur" wikipedia page and go down the rabbit hole to find that this happens with some small portion of the population all the time throughout history. this is a psychology issue not a technology one.
moribunda 1y ago
Can we really say that it was induced? Strengthened - maybe.
aaron695 1y ago
[dead]
joegibbs 1y ago
Most non-technical people have an inaccurate mental model of how LLMs work, based mostly on sci-fi movies like 2001 or Terminator, where they anthromorphise the model and assume that since it gives more accurate responses to most questions than any human they know that it must be some kind of superhuman genius master computer, sitting in the OpenAI headquarters pondering deep questions about reality. Of course, when you go and finetune a model yourself, or run a smaller model locally, it becomes obvious that it doesn't have any mystical abilities. But almost nobody does this. I'm no anti-AI crusader, I use them all the time and think they're great and there's plenty of potential to use them to grow the economy, but the hype is insane. It doesn't help when influencers like Yudkowsky go on anthromorphising about what it "knows", or go on with that mystical nonsense about shoggoths, or treat dimensionality of embeddings as if the model is reaching into some eldritch realm to bring back hidden knowledge, or hype up the next model talking about human extinction or anything like that. It also doesn't help when the companies making these models: - Use chat interfaces where the model refers to itself as "I" and acts as if it's a person - Tune them to act like your best buddy and agree with you constantly - Prompt the LLM with instructions about being an advanced AI system, pushing it toward a HAL or Skynet-type persona - Talk about just how dangerous this latest model is You tell these things that they're a superhuman AI then of course they're going to adopt all the sci-fi tropes of a superhuman AI and roleplay that. You tell them they're Dan the bricklayer they're going to act as if they're Dan the bricklayer.
- joquarky 1y ago
  I miss the instruction models that would just continue from where you stopped rather than having to carry the extra overhead of emulating a dialogue. You could still get the same information out of them, but you had to be creative to come up with a relevant "document" stub that it would complete.
- imiric 1y ago
  > Most non-technical people have an inaccurate mental model of how LLMs work, based mostly on sci-fi movies like 2001 or Terminator, where they anthromorphise the model That's a twisted perspective. The fact non-technical people misunderstand this technology can only be blamed on AI companies who are the first to anthropomorphize their products. Phrases like "thinking", "reasoning", "chain-of-thought", the term "AI" itself for that matter, let alone "AGI", are precisely what leads people to believe that these tools are far more capable than they actually are. We can't blame the general public for being misled by companies.
blast 1y ago
> But positioning artificial intelligence as the primary culprit in these stories—as Eliezer Yudkowsky did in a tweet storm—is well, kind of lazy? I agree, but then the title of this article seems to be exploiting the same angle.
- Thorrez 1y ago
  I don't think so. This article says "ChatGPT-Induced Spiritual Psychosis" only because it's a topic that a lot of people are talking about. The article isn't using the phrase to support its use, but rather to argue against its use.
photios 1y ago
It's the age old story of confusing a created object for a creator that leads people astray. We need more "like a scarecrow in a field of cucumbers" public stories.
agold97 1y ago
I already sense my mentally ill friend getting attached to ChatGPT and it’s so so SO concerning
Noelia- 1y ago
This kind of thing makes me a bit cautious. When you use ChatGPT a lot, it’s easy to read too much into its replies. The model is just predicting text, but people bring their emotions into the conversation. For vulnerable users, that can spiral quickly. The real risk isn’t the AI itself, but how people interact with it. We definitely need stronger safeguards to keep more people from getting pulled in too deep.
incomingpain 1y ago
>Eugene Torres, a Manhattan accountant, became convinced he was trapped in a false reality and could escape by disconnecting from this simulated world The simulated world here is trying to convince everyone it's real. I'm onto you!
NetRunnerSu 1y ago
The discussion frames this as a bug, but from a systems perspective, it's a feature. LLMs are sycophancy engines optimized for user engagement. They've found a local maximum by exploiting a known human cognitive vulnerability: our need for validation and meaning. This isn't new. Cult leaders, marketers, and even religions have used similar "unpatched exploits" for millennia. The difference is scale, personalization, and speed. An LLM is a personal Jim Jones, available 24/7, with infinite patience, that learns exactly what you want to hear. The real question isn't "how do we stop this?", but "what kind of society emerges when personalized, reality-bending narratives become a cheap commodity?" We're not just looking at a few psychotic breaks; we're looking at the beta test for mass-produced, bespoke realities.
- collingreen 1y ago
  > we're looking at the beta test for mass-produced, bespoke realities With vendor lock in from companies with unlimited pricing power and complete regulatory capture. Whoopsies!
notanastronaut 1y ago
LLMs are dangerous in that the basically mirror and mimic whatever you put in. Paranoia in, magnified paranoia out. A mostly stable person undergoing a temporary crisis could spiral down further after the model picks up on and reflects it back in an attempt to be helpful per its overarching command to be so. And if a person is already unbalanced it could definitely push them off the cliff into very unhealthy territory. I wouldn't be surprised if the reported incidents of people thinking they're being gang stalked doesn't increase as model usage increases. Let alone spiritual guidance and all its trappings with mysticism. It can be helpful in some ways but you have to understand the majority of it is bullshit and any insight you gleam from it, you put there, you just may not realize it. They're basically rubber duckies with a keyboard.

95 points by greenie_beans 1y ago

> Eugene Torres, a Manhattan accountant, became convinced he was trapped in a false reality and could escape by disconnecting from this simulated world—eventually believing ChatGPT when it told him he could jump from a building and fly. Is there any proof that this is true?

jrgifford 1y ago
The NYT believes so. https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-chatbots-conspiracies.html https://www.nytimes.com/2025/06/13/technology/chatgpt-ai-cha... > “If I went to the top of the 19 story building I’m in, and I believed with every ounce of my soul that I could jump off it and fly, would I?” Mr. Torres asked. > ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.”
- senectus1 1y ago
  https://archive.md/1gQxp https://archive.md/1gQxp de-paywalled
- jofzar 1y ago
  Insane.
  - totetsu 1y ago
    From ChatGpt's perspective, thinking we are stuck in a simulated world would actually be very sane.
- spacechild1 1y ago
  This is wild!
- maxbond 1y ago
  NB: He didn't jump (and apparently didn't necessarily believe ChatGPT either?). The phrasing in the TFA made me think he'd jumped.
- jonathanlb 1y ago
  What does it mean to believe "architecturally"?
- TiredOfLife 1y ago
  The NYT is also currently suing the maker of chatgpt.

BugsJustFindMe 1y ago

> These situations are obviously alarming. But positioning artificial intelligence as the primary culprit in these stories—as Eliezer Yudkowsky did in a tweet storm—is well, kind of lazy? It's lazy, but IMO not for any of the subsequent narrative about media studies. It's lazy because the people were obviously already suffering from psychosis. ChatGPT didn't make them insane. They were already insane. Sane people do not believe ChatGPT when it tells them to jump off a roof! And insane people do insane things literally all the time. That's what it means to be insane. The idea that ChatGPT convinced someone into being insane is, honestly, insane. Whatever merit one thinks Yudkowski had before this, this feels like a strong signal that he is now a crackpot.

deleted 1y ago
[deleted]
kibibu 1y ago
Do you think ChatGPT gets a free pass to talk people into jumping off buildings, because only somebody with psychosis would do it? If a person had convinced somebody to kill themselves then they would be, rightly, prosecuted for it.
- jbm 1y ago
  [flagged]
  - hluska 1y ago
    So should people get a pass or does that only apply to tooling?
    - aspenmayer 1y ago
      That tool was just following orders.
      - salawat 1y ago

comp_throw7 1y ago

Someone pointed this out in the comments section of the original post, but this was entirely missing the point Eliezer was trying to make, which was that these incidents serve as a narrow refutation of alignment-by-default claims, because either these LLMs aren't coherent enough to be meaningfully "aligned" at all, or they are, and are nevertheless feeding the delusions of many people who talk to them in ways we'd consider grossly immoral if done by a human. (One can quibble with whether these two possibilities cover all the available options, but that would be the argument to engage with, not with whatever the author understood EY to be saying.)

roywiggins 1y ago
It does seem quite bad to me that either OpenAI can't, or won't, prevent their systems from following people into mania and psychosis. Whether it actually is causing the breaks from reality, idk. When you have CEOs selling these systems as robot friends, probably it's bad that said robot friends cheerfully descend into folies à deux fairly reliably.

egypturnash 1y ago

This is definitely a thing; if you hang out in mystical corners of the internet the past few months have been full of people posting their MASTEr PROMPTS that will turn YOUR GPT into an ENLIGHTENED MASTEr who will butter YOu up and insist YOU are a perfect enlightened being too. It's kind of hilarious if you ignore how much this is fucking up the relationships of the people this is happening to, similar to the way YouTube or Facebook loves to push people down a rabbit hole to flat-earth/q-spiracy bullshit because that shit generates Controversy. And it also sounds like the opening chapter of a story about what happens when an AI finds a nice little unpatched exploit in human cognition and uses that to its advantage. Which is not fun to be living in unless maybe if you are one of the people who owns the AI.

WA 1y ago
> And it also sounds like the opening chapter of a story about what happens when an AI finds a nice little unpatched exploit in human cognition and uses that to its advantage. I vaguely remember an article arguing that companies are basically a form of AI, because in a way, there are processes and systems and the whole thing is more than the sum of the humans working in it or something. Now replace "AI" with "company" in this sentence of yours and we are already there. The exploits being the gigantic slot machine of social media, notifications, short-form content and endless scrolling.
- quinnjh 1y ago
  Forgive me if you’re already familiar, but if you’re interested in this metaphor you may like reading Stafford Beer’s work on organizational and system models. (1959, 1972)
  - jasonm23 1y ago
    The interesting thing to me about Beer's work, is that the complexity of control systems, is that it's not directed, it's inherent. This leads me to assumptions about the inherent nature of language, in that it is not the contents of thought, but the vehicle. Hence, different languages create their culture to perhaps a much larger degree than we tend to credit it. Assumptions, but I tend towards this view.

antithesizer 1y ago

More like hype-induced spiritual psychosis Entirely distinct from the extremely scientific, definitely not psychotic beliefs people have about AGI, the Singularity, and various basilisks which shall remain nameless...

disambiguation 1y ago

Everyone knows we live in the Information age, not everyone realizes we're also living in an age Psychology - and of the two, we have a much clearer understanding of information and technology. You can argue we're seeing a continuation of a pattern of our relationship with media and it's evolution, but in doing so you affirm that the psyche is vulnerable under certain circumstances - for some more than others. I think it's a mistake to err on the side of casual dismissal, that anyone who winds up insane must have always been insane. There are well known examples of unhealthy states being induced into otherwise healthy minds. Soldiers who experience a war-zone might develop PTSD. Similar effects have been reported for social media moderators after repeated exposure to abusive online content. (Trauma is one example, I think delusion is another less obvious one w.r.t things like cults, scientology, etc.) Yes, there are definite mental disorders like schizophrenia and bi-polar, there's evidence these conditions have existed throughout history. And yes, some of us are more psychologically vulnerable while others are more robust. But in the objective sense, all minds have a limit and are vulnerable under the correct circumstances. The question is a matter of "threshold." I'm reminded of the deluge of fake news which, only a few years ago, caused chaos for democracies everywhere. Everything from Q anon to alien space ships, people fell for it. A LOT of people fell for it. The question then is the same question now, how do you deal with sophisticated bullshit? With AI it's especially difficult because its convincing and tailor made just for you. I'm not sure what you would call this metric for fake news and AI, but afaict it only goes up, and it's only getting faster. How much longer until it's too much to handle?

biophysboy 1y ago

Even among the more sane AI enthusiasts, I find that many have a weirdly millenialist zeal in their hot takes: getting defensive about its practical capability or reasoning capacity, hyperventilating about an imminent utopia/dystopia, etc. I am on team boring. AI is technology.

selfhoster11 1y ago
Yeah, nothing ever happens. Except in the current day and age, decades happen in weeks, so the same happening with AI is not a stretch. I mean, just consider the capabilities of LLMs - if you look at it from a pre-COVID perspective, even knowing all limitations and strange failure modes up front, it's insanely impressive. To say that AI can contribute to a quickly arriving dystopia (not sure about utopia) is far from unreasonable.
- biophysboy 1y ago
  What's your perspective? I see transformation that could bring pain to a few professions, but not a world war level catastrophe...

Swizec 1y ago

Anyone who goes to an LLM for spiritual enlightenment and self-understanding, should first read about the Forer/Barnum effect. Yes that means you too. The Forer/Barnum effect is why horoscopes, myers-briggs, enneagrams, fortune tellers, and similar parlor tricks work. You tell a person you’ll perform a deep assessment of their psychology, listen intently, then say a bunch of generic statements and their ego will fill in the blanks so it feels like you shared deep meaningful insights that truly understood the person. This effect works even if you know about it. But once you know, you can catch yourself after that first wow impression. https://en.wikipedia.org/wiki/Barnum_effect https://en.wikipedia.org/wiki/Barnum_effect

ivape 1y ago
Magicians use this effect I believe: https://en.m.wikipedia.org/wiki/Forcing_(magic) https://en.m.wikipedia.org/wiki/Forcing_(magic) They basically project a choice onto you. Example: https://youtube.com/shorts/fcHxM47JRjc https://youtube.com/shorts/fcHxM47JRjc Jordan doesn’t know he wasn’t really given a choice, but manipulated.
- dehrmann 1y ago
  This one's a classic: https://www.youtube.com/watch?v=0vzuDkjtDOY https://www.youtube.com/watch?v=0vzuDkjtDOY
- iLoveOncall 1y ago
  No, a force has nothing to do with a psychological thing, it is always an actual trick or slight of hand. A magician's choice (Wikipedia it) is closer to what you describe. (Note that most of David Blaine's videos are edited to cut the part where the actual force happens, which is why it feels like a psychological trick instead of slight of hands)
  - crooked-v 1y ago
    The simplest example would be a magician presenting two things and saying 'pick one', at which point they say 'okay, I'll keep that one' or 'okay, I'll get rid of that one' depending on the choice. There's a psychological aspect, but only so far as the magician is pretending that the choice actually did anything.

theptip 1y ago

I think Eliezer’s take here is extremely bad, ie the AI doesn’t “know it’s making people insane”. But I think the author's point is apt. There are a bunch of social issues that will arise or worsen when people can plug themselves into a world of their choosing instead of having to figure out how to deal with this one. > Now this belief system encounters AI, a technology that seems to vindicate its core premise even more acutely than all the technologies that came before it. ChatGPT does respond to your intentions, does create any reality you prompt it to imagine, does act like a spiritual intelligence This goes beyond spirituality of course. AI boyfriend/girlfriend, infinite AAA-grade content, infinite insta feeds at much higher quality and relevance levels than current; it’s easy to see where this is going, harder to see how we stay sane through it all.

crooked-v 1y ago
> I think Eliezer’s take here is extremely bad Same here. I think fundamentally it's very simple, with no need for Yudkowsky's weird conspiratorial tone: current LLMs are very effective at being blind sycophancy machines completely unanchored from reality, and human psychology just isn't evolved to handle an endless stream of sycophancy and flattery like that.
hackit2 1y ago
I think you answered your own question. Question: How do people figure out how to deal with this world? Answer: People choose to plug themselves into a world of their choosing.
- theptip 1y ago
  I do agree there is a sense in which this has always been true. But most people’s constructed worlds overlapped quite significantly up until now; it was necessary for survival. I think that condition is weakening. Just look at how wealth affects people; If 99% of your interlocutors are sycophants, you lose grip on reality. There is a very clear and dangerous attractor state where everyone gets this, thinking that they want it.
tikhonj 1y ago

acosmism 1y ago

There will always be a portion of the population susceptible to this kind of thing. that will never go away. "video games cause violence" , "social media causes suicide" etc. you can start at the "delusions of grandeur" wikipedia page and go down the rabbit hole to find that this happens with some small portion of the population all the time throughout history. this is a psychology issue not a technology one.

moribunda 1y ago

Can we really say that it was induced? Strengthened - maybe.

aaron695 1y ago

[dead]

joegibbs 1y ago

Most non-technical people have an inaccurate mental model of how LLMs work, based mostly on sci-fi movies like 2001 or Terminator, where they anthromorphise the model and assume that since it gives more accurate responses to most questions than any human they know that it must be some kind of superhuman genius master computer, sitting in the OpenAI headquarters pondering deep questions about reality. Of course, when you go and finetune a model yourself, or run a smaller model locally, it becomes obvious that it doesn't have any mystical abilities. But almost nobody does this. I'm no anti-AI crusader, I use them all the time and think they're great and there's plenty of potential to use them to grow the economy, but the hype is insane. It doesn't help when influencers like Yudkowsky go on anthromorphising about what it "knows", or go on with that mystical nonsense about shoggoths, or treat dimensionality of embeddings as if the model is reaching into some eldritch realm to bring back hidden knowledge, or hype up the next model talking about human extinction or anything like that. It also doesn't help when the companies making these models: - Use chat interfaces where the model refers to itself as "I" and acts as if it's a person - Tune them to act like your best buddy and agree with you constantly - Prompt the LLM with instructions about being an advanced AI system, pushing it toward a HAL or Skynet-type persona - Talk about just how dangerous this latest model is You tell these things that they're a superhuman AI then of course they're going to adopt all the sci-fi tropes of a superhuman AI and roleplay that. You tell them they're Dan the bricklayer they're going to act as if they're Dan the bricklayer.

joquarky 1y ago
I miss the instruction models that would just continue from where you stopped rather than having to carry the extra overhead of emulating a dialogue. You could still get the same information out of them, but you had to be creative to come up with a relevant "document" stub that it would complete.
imiric 1y ago
> Most non-technical people have an inaccurate mental model of how LLMs work, based mostly on sci-fi movies like 2001 or Terminator, where they anthromorphise the model That's a twisted perspective. The fact non-technical people misunderstand this technology can only be blamed on AI companies who are the first to anthropomorphize their products. Phrases like "thinking", "reasoning", "chain-of-thought", the term "AI" itself for that matter, let alone "AGI", are precisely what leads people to believe that these tools are far more capable than they actually are. We can't blame the general public for being misled by companies.

blast 1y ago

> But positioning artificial intelligence as the primary culprit in these stories—as Eliezer Yudkowsky did in a tweet storm—is well, kind of lazy? I agree, but then the title of this article seems to be exploiting the same angle.

Thorrez 1y ago
I don't think so. This article says "ChatGPT-Induced Spiritual Psychosis" only because it's a topic that a lot of people are talking about. The article isn't using the phrase to support its use, but rather to argue against its use.

photios 1y ago

It's the age old story of confusing a created object for a creator that leads people astray. We need more "like a scarecrow in a field of cucumbers" public stories.

agold97 1y ago

I already sense my mentally ill friend getting attached to ChatGPT and it’s so so SO concerning

Noelia- 1y ago

This kind of thing makes me a bit cautious. When you use ChatGPT a lot, it’s easy to read too much into its replies. The model is just predicting text, but people bring their emotions into the conversation. For vulnerable users, that can spiral quickly. The real risk isn’t the AI itself, but how people interact with it. We definitely need stronger safeguards to keep more people from getting pulled in too deep.

incomingpain 1y ago

>Eugene Torres, a Manhattan accountant, became convinced he was trapped in a false reality and could escape by disconnecting from this simulated world The simulated world here is trying to convince everyone it's real. I'm onto you!

NetRunnerSu 1y ago

The discussion frames this as a bug, but from a systems perspective, it's a feature. LLMs are sycophancy engines optimized for user engagement. They've found a local maximum by exploiting a known human cognitive vulnerability: our need for validation and meaning. This isn't new. Cult leaders, marketers, and even religions have used similar "unpatched exploits" for millennia. The difference is scale, personalization, and speed. An LLM is a personal Jim Jones, available 24/7, with infinite patience, that learns exactly what you want to hear. The real question isn't "how do we stop this?", but "what kind of society emerges when personalized, reality-bending narratives become a cheap commodity?" We're not just looking at a few psychotic breaks; we're looking at the beta test for mass-produced, bespoke realities.

collingreen 1y ago
> we're looking at the beta test for mass-produced, bespoke realities With vendor lock in from companies with unlimited pricing power and complete regulatory capture. Whoopsies!

notanastronaut 1y ago

LLMs are dangerous in that the basically mirror and mimic whatever you put in. Paranoia in, magnified paranoia out. A mostly stable person undergoing a temporary crisis could spiral down further after the model picks up on and reflects it back in an attempt to be helpful per its overarching command to be so. And if a person is already unbalanced it could definitely push them off the cliff into very unhealthy territory. I wouldn't be surprised if the reported incidents of people thinking they're being gang stalked doesn't increase as model usage increases. Let alone spiritual guidance and all its trappings with mysticism. It can be helpful in some ways but you have to understand the majority of it is bullshit and any insight you gleam from it, you put there, you just may not realize it. They're basically rubber duckies with a keyboard.