3 ms·

I agree that this is part of the trouble. I don't think any of this is a binary checkbox. But I also think there's likely enough evidence or public pressure tha

I agree that this is part of the trouble. I don't think any of this is a binary checkbox. But I also think there's likely enough evidence or public pressure that the company is being found by the public to be responsible if their service encourages a mentally ill person to commit murder/suicide. I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault?

HanClinto 9mo ago
I don't know how related this is or not, but one thing that I've noticed is that a lot of the "How to awaken your LLM!" and "Secret prompt to turn on the personhood of your ChatGPT!" types of guides use role-playing games as a foundation. One prompts the LLM: "Imagine a fantasy scenario where XYZ is true, play along with me!" I think this is another part of the reason why these discussions remind me of the D&D panic, because so many of the dangers being pointed to are cases where the line is being blurred between fantasy and reality. If you are a DM in an RPG, and a player is exhibiting troubling psychological behavior (such as sociopathy, a focus on death and/or killing, etc), at what point do you decide that you think it's a problem, or else just chalk it up as regular player "murder hobo" behavior? It's very much not cut-and-dry. > I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault? Tort is not something I'm very familiar with, but adding "safeties" to tools can easily make them less powerful or capable. Your analogy of flammable furniture is a good one. The analogy of safeties on power tools is another one that comes to mind. What are reasonable safeguards to place on powerful tools? And even with safeguards in place, people have still sued (and won) lawsuits against table-saw manufacturers -- even in cases where the users intentionally mis-used the saw or disabled safety features. In this case, what can be done when someone takes a tool built and targeted for X purpose, and it's (mis)used and it leads to injury? Assuming the tool was built with reasonable safeties in place, even a 99.9999% safety rating will result in thousands of accidents. Chasing those last few decimal points in a pursuit of true 100% (with zero accidents) is a tyranny and futility all its own.

HanClinto 9mo ago

I don't know how related this is or not, but one thing that I've noticed is that a lot of the "How to awaken your LLM!" and "Secret prompt to turn on the personhood of your ChatGPT!" types of guides use role-playing games as a foundation. One prompts the LLM: "Imagine a fantasy scenario where XYZ is true, play along with me!" I think this is another part of the reason why these discussions remind me of the D&D panic, because so many of the dangers being pointed to are cases where the line is being blurred between fantasy and reality. If you are a DM in an RPG, and a player is exhibiting troubling psychological behavior (such as sociopathy, a focus on death and/or killing, etc), at what point do you decide that you think it's a problem, or else just chalk it up as regular player "murder hobo" behavior? It's very much not cut-and-dry. > I guess similar to maybe how non-flammable furniture is now regulated even though setting fires is not the materials' fault? Tort is not something I'm very familiar with, but adding "safeties" to tools can easily make them less powerful or capable. Your analogy of flammable furniture is a good one. The analogy of safeties on power tools is another one that comes to mind. What are reasonable safeguards to place on powerful tools? And even with safeguards in place, people have still sued (and won) lawsuits against table-saw manufacturers -- even in cases where the users intentionally mis-used the saw or disabled safety features. In this case, what can be done when someone takes a tool built and targeted for X purpose, and it's (mis)used and it leads to injury? Assuming the tool was built with reasonable safeties in place, even a 99.9999% safety rating will result in thousands of accidents. Chasing those last few decimal points in a pursuit of true 100% (with zero accidents) is a tyranny and futility all its own.