3 ms·

> I think there will be always people in bad mental state who will unfortunately commit suicide after some interaction I've been getting strong flashbacks to P

> I think there will be always people in bad mental state who will unfortunately commit suicide after some interaction I've been getting strong flashbacks to Patricia Pulling and the anti-Dungeons-and-Dragons panic. [0] Back in the 1980's, Patricia's son Irving committed suicide, and it was associated (at least in her mind) with him picking up Dungeons and Dragons. This led to a number of lawsuits, and organizations and campaigns from people who were concerned about role-playing games causing its players to lose touch with the boundaries between fantasy and reality, and (they claimed) was dangerous and deadly for its players. LLMs / D&D forms an interesting parallel to me, because -- like chatbots -- an immersive roleplaying experience is largely a reflection of what you (and the other players) put into the game. Chatbots (and things like LLM-psychosis) are on an entirely different magnitude than RPGs, but I hear a lot of similar phrases regarding "detachment from reality" and "reinforcement of delusions" that I heard back in the 80's around D&D as well. Is it more "real" this time? I remain skeptical, but I certainly believe that all of the marketing spin to anthropomorphize AI isn't doing it any favors. Demystifying AI will help everyone. This is why I prefer to say I work with "Artificial AI" -- I don't work on the "real stuff". There are no personalities or consciousness here -- it just looks like it. * [0] - https://en.wikipedia.org/wiki/Patricia_Pulling https://en.wikipedia.org/wiki/Patricia_Pulling

KittenInABox 9mo ago
I think it is right to be skeptical that this is another media buzz. However I also think that there is a fundamental different magnitude going on. Being mentally ill requires careful handling that should be left up to professionals with licenses on the line and liabilities if they are found to be mispracticing.
- HanClinto 9mo ago
  > Being mentally ill requires careful handling that should be left up to professionals with licenses on the line and liabilities if they are found to be mispracticing. Part of the trouble is that "undiagnosed but mentally ill" is not a binary checkbox that most people tick in their day-to-day lives, nor is it easily discernable (even for people themselves, much less people engineers who build apps or platforms). We're all mixed together in the same general populace.
  - KittenInABox 9mo ago
    I agree that this is part of the trouble. I don't think any of this is a binary checkbox. But I also think there's likely enough evidence or public pressure that the company is being found by the public to be responsible if their service encourages a mentally ill person to commit murder/suicide. I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault?
    - HanClinto 9mo ago
      I don't know how related this is or not, but one thing that I've noticed is that a lot of the "How to awaken your LLM!" and "Secret prompt to turn on the personhood of your ChatGPT!" types of guides use role-playing games as a foundation. One prompts the LLM: "Imagine a fantasy scenario where XYZ is true, play along with me!" I think this is another part of the reason why these discussions remind me of the D&D panic, because so many of the dangers being pointed to are cases where the line is being blurred between fantasy and reality. If you are a DM in an RPG, and a player is exhibiting troubling psychological behavior (such as sociopathy, a focus on death and/or killing, etc), at what point do you decide that you think it's a problem, or else just chalk it up as regular player "murder hobo" behavior? It's very much not cut-and-dry. > I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault? Tort is not something I'm very familiar with, but adding "safeties" to tools can easily make them less powerful or capable. Your analogy of flammable furniture is a good one. The analogy of safeties on power tools is another one that comes to mind. What are reasonable safeguards to place on powerful tools? And even with safeguards in place, people have still sued (and won) lawsuits against table-saw manufacturers -- even in cases where the users intentionally mis-used the saw or disabled safety features. In this case, what can be done when someone takes a tool built and targeted for X purpose, and it's (mis)used and it leads to injury? Assuming the tool was built with reasonable safeties in place, even a 99.9999% safety rating will result in thousands of accidents. Chasing those last few decimal points in a pursuit of true 100% (with zero accidents) is a tyranny and futility all its own.
soiltype 9mo ago
You laid out the difference in your own post. The D&D backlash wasn't sparked by widespread incidents of serious delusions. But LLM delusions are actually happening, a lot, and leading directly to deaths.
- HanClinto 9mo ago
  > The D&D backlash wasn't sparked by widespread incidents of serious delusions It was sparked by real incidents which resulted in real deaths. Patricia wasn't the only concerned parent dealing with real tragedy. The questions are "how widespread" and "how directly-connected". I don't think we can assume the number is zero -- I would bet good money that -- on multiple occasions -- games exacerbated mental-illness and was a factor that resulted in quantifiable harm (even death). But at the time that this was all new and breaking, it was very difficult to separate hearsay and anecdote from the larger picture. I don't hold any enmity towards my parents for finding my gaming supplies and making me get rid of them -- it was the 80's. They were well-intentioned, and a lot of what we heard was nearly impossible to quantify or verify. > But LLM delusions are actually happening, a lot, and leading directly to deaths. I believe this is also happening. "A lot" is what I'm still trying to quantify. There are "a lot" of regular users, and laws of large numbers apply here. Even just 0.001% of 800 million is still 8000 incidents.

KittenInABox 9mo ago

I think it is right to be skeptical that this is another media buzz. However I also think that there is a fundamental different magnitude going on. Being mentally ill requires careful handling that should be left up to professionals with licenses on the line and liabilities if they are found to be mispracticing.

HanClinto 9mo ago
> Being mentally ill requires careful handling that should be left up to professionals with licenses on the line and liabilities if they are found to be mispracticing. Part of the trouble is that "undiagnosed but mentally ill" is not a binary checkbox that most people tick in their day-to-day lives, nor is it easily discernable (even for people themselves, much less people engineers who build apps or platforms). We're all mixed together in the same general populace.
- KittenInABox 9mo ago
  I agree that this is part of the trouble. I don't think any of this is a binary checkbox. But I also think there's likely enough evidence or public pressure that the company is being found by the public to be responsible if their service encourages a mentally ill person to commit murder/suicide. I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault?
  - HanClinto 9mo ago
    I don't know how related this is or not, but one thing that I've noticed is that a lot of the "How to awaken your LLM!" and "Secret prompt to turn on the personhood of your ChatGPT!" types of guides use role-playing games as a foundation. One prompts the LLM: "Imagine a fantasy scenario where XYZ is true, play along with me!" I think this is another part of the reason why these discussions remind me of the D&D panic, because so many of the dangers being pointed to are cases where the line is being blurred between fantasy and reality. If you are a DM in an RPG, and a player is exhibiting troubling psychological behavior (such as sociopathy, a focus on death and/or killing, etc), at what point do you decide that you think it's a problem, or else just chalk it up as regular player "murder hobo" behavior? It's very much not cut-and-dry. > I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault? Tort is not something I'm very familiar with, but adding "safeties" to tools can easily make them less powerful or capable. Your analogy of flammable furniture is a good one. The analogy of safeties on power tools is another one that comes to mind. What are reasonable safeguards to place on powerful tools? And even with safeguards in place, people have still sued (and won) lawsuits against table-saw manufacturers -- even in cases where the users intentionally mis-used the saw or disabled safety features. In this case, what can be done when someone takes a tool built and targeted for X purpose, and it's (mis)used and it leads to injury? Assuming the tool was built with reasonable safeties in place, even a 99.9999% safety rating will result in thousands of accidents. Chasing those last few decimal points in a pursuit of true 100% (with zero accidents) is a tyranny and futility all its own.

soiltype 9mo ago

You laid out the difference in your own post. The D&D backlash wasn't sparked by widespread incidents of serious delusions. But LLM delusions are actually happening, a lot, and leading directly to deaths.

HanClinto 9mo ago
> The D&D backlash wasn't sparked by widespread incidents of serious delusions It was sparked by real incidents which resulted in real deaths. Patricia wasn't the only concerned parent dealing with real tragedy. The questions are "how widespread" and "how directly-connected". I don't think we can assume the number is zero -- I would bet good money that -- on multiple occasions -- games exacerbated mental-illness and was a factor that resulted in quantifiable harm (even death). But at the time that this was all new and breaking, it was very difficult to separate hearsay and anecdote from the larger picture. I don't hold any enmity towards my parents for finding my gaming supplies and making me get rid of them -- it was the 80's. They were well-intentioned, and a lot of what we heard was nearly impossible to quantify or verify. > But LLM delusions are actually happening, a lot, and leading directly to deaths. I believe this is also happening. "A lot" is what I'm still trying to quantify. There are "a lot" of regular users, and laws of large numbers apply here. Even just 0.001% of 800 million is still 8000 incidents.