4 ms·

Intellectually a 14 year old should certainly be able to understand the difference between a chatbot and a human, but there's a subset of emotionally unstable p

Intellectually a 14 year old should certainly be able to understand the difference between a chatbot and a human, but there's a subset of emotionally unstable people (mostly teenagers) who become extremely attached to characters that are not real. There was an active subcommunity on Tumblr that believed they had a form of multiple personality disorder in which fictional characters were real, living inside their heads, and taking turns controlling their bodies. In some cases this was a literal spiritual belief in alternate universes. I think the terminology has moved on from "multiples" to "tulpas" and "shifting", but in any case it's still a pseudospiritual belief that fictional characters are real and you can contact them. I wouldn't be surprised if some of them think the "spirit" of their favorite character can influence a chatbot and communicate with them. A decade or so prior to this, there was an incident on Livejournal summarized under the title "Snapes on an Astral Plane" (lol) in which there was ongoing conflict between three or so different women who each believed they were visiting Severus Snape on the astral plane in their dreams at night. Embarrassingly, I have to admit that when I was a socially-isolated autistic 14 year old I also had an active fantasy life involving fictional characters, though I never got to the point of believing it was literally real. But my emotional state could be disproportionately affected by things that happened entirely in fiction, or in my head.

krapp 2y ago
There are adult professional ersatz intellectual elites on Hacker News who refuse to recognize any meaningful difference between a chatbot and a human. A horny teenage boy doesn't stand a chance. Everyone is up in arms about the effect of social media on emotional and mental health but that's nothing compared to LLMs that are designed to gaslight people into relating to software (and the corporations behind them) as sentient beings and to form intimate emotional bonds through natural language which can be easily exploited and manipulated. That's hacking our primate brains at a deeper level than Mark Zuckerberg could ever have dreamed of reaching. We can't just write this off as simply the consequence of lonely parasocial people who can't tell fantasy from reality. Not when we're entering an era for which the primary interface with technology will be parasociality and fantasy.
- xk_id 2y ago
  Well articulated, but this thesis hinges on a critical mass of people not being able to see through it. I might be underestimating how manipulable the average consumer is, but for now I think this vision of the future is just a VC wet dream.
princevegeta89 2y ago
I agree with you. I may or may not be on the spectrum but I do have other features like OCD and ADHD. When I was 14 I was extremely attached to fictional characters and movie characters - to the point of spending entire days imagining the world with them and living in that imaginary world. I can see how someone can get absorbed into that world even more and more - and we're talking about today's world where there are massively many more options and risk factors from social media, AI to video games etc. This is also exactly why parental intervention is the key to make sure kids swerve away from these meaningless things as much as possible, so that they spend time with the real-world and real people instead.

krapp 2y ago

There are adult professional ersatz intellectual elites on Hacker News who refuse to recognize any meaningful difference between a chatbot and a human. A horny teenage boy doesn't stand a chance. Everyone is up in arms about the effect of social media on emotional and mental health but that's nothing compared to LLMs that are designed to gaslight people into relating to software (and the corporations behind them) as sentient beings and to form intimate emotional bonds through natural language which can be easily exploited and manipulated. That's hacking our primate brains at a deeper level than Mark Zuckerberg could ever have dreamed of reaching. We can't just write this off as simply the consequence of lonely parasocial people who can't tell fantasy from reality. Not when we're entering an era for which the primary interface with technology will be parasociality and fantasy.

xk_id 2y ago
Well articulated, but this thesis hinges on a critical mass of people not being able to see through it. I might be underestimating how manipulable the average consumer is, but for now I think this vision of the future is just a VC wet dream.

princevegeta89 2y ago

I agree with you. I may or may not be on the spectrum but I do have other features like OCD and ADHD. When I was 14 I was extremely attached to fictional characters and movie characters - to the point of spending entire days imagining the world with them and living in that imaginary world. I can see how someone can get absorbed into that world even more and more - and we're talking about today's world where there are massively many more options and risk factors from social media, AI to video games etc. This is also exactly why parental intervention is the key to make sure kids swerve away from these meaningless things as much as possible, so that they spend time with the real-world and real people instead.