4 ms·

How many more cases will be sufficient for OP to conclude that gaslighting users & encouraging their paranoid delusions is detrimental for their mental health?

How many more cases will be sufficient for OP to conclude that gaslighting users & encouraging their paranoid delusions is detrimental for their mental health? Let us put the issue of murders & suicides caused by these chat bots to the side for a second & simply consider the fact that a significant segment of their user base is convinced these things are conscious & capable of sentience.

dpark 10mo ago
> the fact that a significant segment of their user base is convinced these things are conscious & capable of sentience. Is this a fact? There’s a lot of hype about “AI psychosis” and similar but I haven’t seen any meaningful evidence of this yet. It’s a few anecdotes and honestly seems more like a moral panic than a legitimate conversation about real dangers so far. I grew up in peak D.A.R.E. where I was told repeatedly by authority figures that people who take drugs almost inevitably turn to violence and frequently succumb to psychotic episodes. Turns out that some addicts do turn to violence and extremely heavy usage of some drugs can indeed trigger psychosis, but this is very fringe relative to the actual huge amount of people who use illicit drugs. I can absolutely believe that chatbots are bad for the mental health of people already experiencing significant psychotic or paranoid symptoms. I have no idea how common this is or how outcomes are affected by chatbot usage. Nor do I have any clue what to do about it if it is an issue that needs addressing.
- measurablefunc 10mo ago
  > Nor do I have any clue what to do about it if it is an issue that needs addressing. What happened with cigarettes? Same must happen with chat bots. There must be a prominent & visible warning about the fact that chat bots are nothing more than Markov chains, they are not sentient, they are not conscious, & are not capable of providing psychological guidance & advice to anyone, let alone those who might be susceptible to paranoid delusions & suggestions. Once that's done the companies can be held liable for promising what they can't deliver & their representatives can be fined for doing the same thing across various media platforms & in their marketing.
  - dpark 10mo ago
    > What happened with cigarettes? We established a comprehensive set of data that established correlation with a huge number of illnesses including lung cancer, to the point that nearly all qualified medical professionals agreed the relationship was causal. > There must be a prominent & visible warning I have no problem with that. I’m a little surprised that ChatGPT et al don’t put some notice at the start of every new chat, purely as a CYA. I’m not sure exactly what that warning should say, and I don’t think I’d put what you proposed, but I would be on board with warnings.
    - jakeydus 10mo ago
      That's just the thing though. OpenAI and the LLM industry generally are pushing so hard against any kind of regulation that the likelihood of this happening is definitely lower than the percentage of ChatGPT users in psychosis.

dpark 10mo ago

> the fact that a significant segment of their user base is convinced these things are conscious & capable of sentience. Is this a fact? There’s a lot of hype about “AI psychosis” and similar but I haven’t seen any meaningful evidence of this yet. It’s a few anecdotes and honestly seems more like a moral panic than a legitimate conversation about real dangers so far. I grew up in peak D.A.R.E. where I was told repeatedly by authority figures that people who take drugs almost inevitably turn to violence and frequently succumb to psychotic episodes. Turns out that some addicts do turn to violence and extremely heavy usage of some drugs can indeed trigger psychosis, but this is very fringe relative to the actual huge amount of people who use illicit drugs. I can absolutely believe that chatbots are bad for the mental health of people already experiencing significant psychotic or paranoid symptoms. I have no idea how common this is or how outcomes are affected by chatbot usage. Nor do I have any clue what to do about it if it is an issue that needs addressing.

measurablefunc 10mo ago
> Nor do I have any clue what to do about it if it is an issue that needs addressing. What happened with cigarettes? Same must happen with chat bots. There must be a prominent & visible warning about the fact that chat bots are nothing more than Markov chains, they are not sentient, they are not conscious, & are not capable of providing psychological guidance & advice to anyone, let alone those who might be susceptible to paranoid delusions & suggestions. Once that's done the companies can be held liable for promising what they can't deliver & their representatives can be fined for doing the same thing across various media platforms & in their marketing.
- dpark 10mo ago
  > What happened with cigarettes? We established a comprehensive set of data that established correlation with a huge number of illnesses including lung cancer, to the point that nearly all qualified medical professionals agreed the relationship was causal. > There must be a prominent & visible warning I have no problem with that. I’m a little surprised that ChatGPT et al don’t put some notice at the start of every new chat, purely as a CYA. I’m not sure exactly what that warning should say, and I don’t think I’d put what you proposed, but I would be on board with warnings.
  - jakeydus 10mo ago
    That's just the thing though. OpenAI and the LLM industry generally are pushing so hard against any kind of regulation that the likelihood of this happening is definitely lower than the percentage of ChatGPT users in psychosis.