3 ms·

"Sewell, like many children his age, did not have the maturity or mental capacity to understand that the C.AI bot, in the form of Daenerys, was not real" How?

"Sewell, like many children his age, did not have the maturity or mental capacity to understand that the C.AI bot, in the form of Daenerys, was not real" How? Seriously, how? Maybe it's wishful thinking on my part -- but I grew up before AI and chatbots, and I'm certain I would understand it isn't real. I'm baffled by people engaging with these things for entertainment/companionship purposes.

gremlinsinc 2y ago
The subconscious mind ... believes anything you feed it, on a subconscious level maybe he couldn't distinguish between normal reality plus he certainly already had mental health issues that could have made him more susceptible to that sort of thing. BPD for instance in some cases can cause hallucinations and it does cause you to generally live in a 'fantasy' world where you're often daydreaming. A lot of this is caused often though by outside trauma, BPD can be caught environmentally, isn't necessarily chemical, though that can play a part. This is my understanding as someone with mild / high functioning BPD (recently diagnosed) ... Please feel free to correct me on that if I'm wrong...
- readthenotes1 2y ago
  Yes, but most 14 year olds have the capacity to know the difference between a chatbot and a real person Whether that capacity is realized is another thing
autumnstwilight 2y ago
Intellectually a 14 year old should certainly be able to understand the difference between a chatbot and a human, but there's a subset of emotionally unstable people (mostly teenagers) who become extremely attached to characters that are not real. There was an active subcommunity on Tumblr that believed they had a form of multiple personality disorder in which fictional characters were real, living inside their heads, and taking turns controlling their bodies. In some cases this was a literal spiritual belief in alternate universes. I think the terminology has moved on from "multiples" to "tulpas" and "shifting", but in any case it's still a pseudospiritual belief that fictional characters are real and you can contact them. I wouldn't be surprised if some of them think the "spirit" of their favorite character can influence a chatbot and communicate with them. A decade or so prior to this, there was an incident on Livejournal summarized under the title "Snapes on an Astral Plane" (lol) in which there was ongoing conflict between three or so different women who each believed they were visiting Severus Snape on the astral plane in their dreams at night. Embarrassingly, I have to admit that when I was a socially-isolated autistic 14 year old I also had an active fantasy life involving fictional characters, though I never got to the point of believing it was literally real. But my emotional state could be disproportionately affected by things that happened entirely in fiction, or in my head.
- krapp 2y ago
  There are adult professional ersatz intellectual elites on Hacker News who refuse to recognize any meaningful difference between a chatbot and a human. A horny teenage boy doesn't stand a chance. Everyone is up in arms about the effect of social media on emotional and mental health but that's nothing compared to LLMs that are designed to gaslight people into relating to software (and the corporations behind them) as sentient beings and to form intimate emotional bonds through natural language which can be easily exploited and manipulated. That's hacking our primate brains at a deeper level than Mark Zuckerberg could ever have dreamed of reaching. We can't just write this off as simply the consequence of lonely parasocial people who can't tell fantasy from reality. Not when we're entering an era for which the primary interface with technology will be parasociality and fantasy.
  - xk_id 2y ago
    Well articulated, but this thesis hinges on a critical mass of people not being able to see through it. I might be underestimating how manipulable the average consumer is, but for now I think this vision of the future is just a VC wet dream.
- princevegeta89 2y ago
  I agree with you. I may or may not be on the spectrum but I do have other features like OCD and ADHD. When I was 14 I was extremely attached to fictional characters and movie characters - to the point of spending entire days imagining the world with them and living in that imaginary world. I can see how someone can get absorbed into that world even more and more - and we're talking about today's world where there are massively many more options and risk factors from social media, AI to video games etc. This is also exactly why parental intervention is the key to make sure kids swerve away from these meaningless things as much as possible, so that they spend time with the real-world and real people instead.
deleted 2y ago
[deleted]
rsynnott 2y ago
I mean, half the commentators on this here orange website do not appear to _really_ understand that LLMs do not think. And at least in principle you'd expect this community to have a better grasp of it than laypeople. "Artificial intelligence" is itself a rather misleading term, and it's arguably unfortunate that it was brought back from the dead to describe this sort of thing (for most of the 21st century it was rarely used for AI-ish things in marketing, with the term 'ML' preferred as a coverall, presumably due to fallout from the 90s AI bubble collapse).

gremlinsinc 2y ago

The subconscious mind ... believes anything you feed it, on a subconscious level maybe he couldn't distinguish between normal reality plus he certainly already had mental health issues that could have made him more susceptible to that sort of thing. BPD for instance in some cases can cause hallucinations and it does cause you to generally live in a 'fantasy' world where you're often daydreaming. A lot of this is caused often though by outside trauma, BPD can be caught environmentally, isn't necessarily chemical, though that can play a part. This is my understanding as someone with mild / high functioning BPD (recently diagnosed) ... Please feel free to correct me on that if I'm wrong...

readthenotes1 2y ago
Yes, but most 14 year olds have the capacity to know the difference between a chatbot and a real person Whether that capacity is realized is another thing

autumnstwilight 2y ago

Intellectually a 14 year old should certainly be able to understand the difference between a chatbot and a human, but there's a subset of emotionally unstable people (mostly teenagers) who become extremely attached to characters that are not real. There was an active subcommunity on Tumblr that believed they had a form of multiple personality disorder in which fictional characters were real, living inside their heads, and taking turns controlling their bodies. In some cases this was a literal spiritual belief in alternate universes. I think the terminology has moved on from "multiples" to "tulpas" and "shifting", but in any case it's still a pseudospiritual belief that fictional characters are real and you can contact them. I wouldn't be surprised if some of them think the "spirit" of their favorite character can influence a chatbot and communicate with them. A decade or so prior to this, there was an incident on Livejournal summarized under the title "Snapes on an Astral Plane" (lol) in which there was ongoing conflict between three or so different women who each believed they were visiting Severus Snape on the astral plane in their dreams at night. Embarrassingly, I have to admit that when I was a socially-isolated autistic 14 year old I also had an active fantasy life involving fictional characters, though I never got to the point of believing it was literally real. But my emotional state could be disproportionately affected by things that happened entirely in fiction, or in my head.

krapp 2y ago
There are adult professional ersatz intellectual elites on Hacker News who refuse to recognize any meaningful difference between a chatbot and a human. A horny teenage boy doesn't stand a chance. Everyone is up in arms about the effect of social media on emotional and mental health but that's nothing compared to LLMs that are designed to gaslight people into relating to software (and the corporations behind them) as sentient beings and to form intimate emotional bonds through natural language which can be easily exploited and manipulated. That's hacking our primate brains at a deeper level than Mark Zuckerberg could ever have dreamed of reaching. We can't just write this off as simply the consequence of lonely parasocial people who can't tell fantasy from reality. Not when we're entering an era for which the primary interface with technology will be parasociality and fantasy.
- xk_id 2y ago
  Well articulated, but this thesis hinges on a critical mass of people not being able to see through it. I might be underestimating how manipulable the average consumer is, but for now I think this vision of the future is just a VC wet dream.
princevegeta89 2y ago
I agree with you. I may or may not be on the spectrum but I do have other features like OCD and ADHD. When I was 14 I was extremely attached to fictional characters and movie characters - to the point of spending entire days imagining the world with them and living in that imaginary world. I can see how someone can get absorbed into that world even more and more - and we're talking about today's world where there are massively many more options and risk factors from social media, AI to video games etc. This is also exactly why parental intervention is the key to make sure kids swerve away from these meaningless things as much as possible, so that they spend time with the real-world and real people instead.

deleted 2y ago

[deleted]

rsynnott 2y ago

I mean, half the commentators on this here orange website do not appear to _really_ understand that LLMs do not think. And at least in principle you'd expect this community to have a better grasp of it than laypeople. "Artificial intelligence" is itself a rather misleading term, and it's arguably unfortunate that it was brought back from the dead to describe this sort of thing (for most of the 21st century it was rarely used for AI-ish things in marketing, with the term 'ML' preferred as a coverall, presumably due to fallout from the 90s AI bubble collapse).