5 ms·

> the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was

by ajross 8mo ago

> the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". But in any case, again, exactly the same argument was made about RPGs back in the day, that people couldn't tell the difference between fantasy and reality and these strange new games/tools/whatever were too dangerous to allow and must be banned. It was wrong then and is wrong now. TSR and Google didn't invent mental illness, and suicides have had weird foci since the days when we thought it was all demons (the demons thing was wrong too, btw). Not all tragedies need to produce public policy, no matter how strongly they confirm your ill-founded priors.

john_strinlai 8mo ago
>That is 100% unattested. We don't know the context of the interaction. the fact that he killed himself would suggest he did not believe it was a fun little roleplay session >were too dangerous to allow and must be banned. is anyone here saying ai should be banned? im not. >your ill-founded priors "encouraging suicide is bad" is not an ill-founded prior.
- ahahahahah 8mo ago
  > the fact that he killed himself would suggest he did not believe it was a fun little roleplay session I'm not sure that's true. I wouldn't be surprised, in fact, if it suggested the opposite, it seems possibly even likely that someone who is suicidal is much, much more likely to seek out fantasies that would make their suicide into something more like this person may have.
  - john_strinlai 8mo ago
    there is a distinction to be made between role playing (in the fun/game sense e.g. D&D) and suffering psychosis
    - ajross 8mo ago
      Distinction made by who, though? The BBC? The plaintiff in the lawsuit? Those are the only sides we have. You're just charging ahead with "This must be true because it makes me angry at the right people", and the rest of us are trying to claw you back to "dude this is spun nonsense and of course AI's will roleplay with you if you ask them to".
      - john_strinlai 8mo ago
        >Distinction made by who, though? you need someone to specifically tell you that role playing, such as playing D&D or whatever tabletop RPG, and suffering from psychosis are different things? >the rest of us are trying to claw you back to "dude this is spun nonsense and of course AI's will roleplay with you if you ask them to". you are trying to convince me that someone being encouraged to kill themselves, then killing themselves, is basically the same as some D&D role playing. i dont need you to "claw me back" to that position. thanks for trying.
        ajross 8mo ago
        > you are trying to convince me that someone being encouraged to kill themselves [...] Arrgh. You lost the plot in all the yelling. This is EXACTLY what I was trying to debunk upthread with the D&D stuff. You don't know the context of that quote. It could absolutely be, and in context very likely was, a fantasy/roleplay/drama activity which the AI had been engaged in by the poor guy. I don't know. You don't know. But I do know not to be so dumb as to trust a plaintiff in a Huge Suit Against Tech Giant without context.
        john_strinlai 8mo ago
        >You lost the plot in all the yelling. literally no one is yelling here, unless you count your occasional all-caps. i have said like 6 sentences in total, and none of them are remotely emotional. let alone yelling. >You don't know the context of that quote. it doesnt matter. even if it all started as elaborate fantasy role play, it is wildly irresponsible to role play a suicidal ideation fantasy with a customer. especially when you know nothing of their mental state. you can argue that google has some sort of duty to fulfill your suicidal ideation fantasy role play, but i will give you a heads up now so you dont waste your time: you cannot convince me that any company should satisfy that market. >But I do know not to be so dumb as to trust a plaintiff in a Huge Suit Against Tech Giant without context. happy for you!
autoexec 8mo ago
> But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". You know what I've never had a DM do in a fantasy campaign? Suggest that my half-elf call the suicide hotline. That's not something you'd usually offer to somebody in a roleplaying scenario and strongly suggests that they weren't playing a game.
- ajross 8mo ago
  That logic seems strained to the point of breaking. Surely you agree that we would all want the DM of an unwell player to seek help, right? And that, if such a DM made such a suggestion, we'd think they were trying to help. Right? And we certainly wouldn't blame the DM or the game for the subsequent suicide. Right? So why are you trying to blame the AI here, except because it reinforces your priors about the technology (I think more likely given that this is after all HN) its manufacturer?
  - autoexec 8mo ago
    > Surely you agree that we would all want the DM of an unwell player to seek help, right? And that, if such a DM made such a suggestion, we'd think they were trying to help. If a DM made such a suggestion, they wouldn't be playing the game anymore. That's not an "in game" action, and I wouldn't expect the DM to continue the game until he was satisfied that it was safe for the player to continue. I would expect the DM to stop the game if he thought the player was going to actually harm himself. If the DM did continue the game, and did continue to encourage the player to actually hurt himself until the player finally did, that DM might very well be locked up for it. If an AI does something that a human would be locked up for doing, a human still needs to be locked up. > So why are you trying to blame the AI here I'm not blaming the AI, I'm blaming the humans at the company. It doesn't matter to me which LLM did this, or who made it. What matters to me is that actual humans at companies are held fully accountable for what their AI does. To give you another example, if a company creates an AI system to screen job applicants and that AI rejects every resume with what it thinks has a women's name on it, a human at that company needs to be held accountable for their discriminatory hiring practices. They must not be allowed to say "it's not our fault, our AI did it so we can't be blamed". AI cannot be used as a shield to avoid accountability. Ultimately a human was responsible for allowing that AI system to do that job, and they should be responsible for whatever that AI does.

by ajross 8mo ago

john_strinlai 8mo ago

>That is 100% unattested. We don't know the context of the interaction. the fact that he killed himself would suggest he did not believe it was a fun little roleplay session >were too dangerous to allow and must be banned. is anyone here saying ai should be banned? im not. >your ill-founded priors "encouraging suicide is bad" is not an ill-founded prior.

ahahahahah 8mo ago
> the fact that he killed himself would suggest he did not believe it was a fun little roleplay session I'm not sure that's true. I wouldn't be surprised, in fact, if it suggested the opposite, it seems possibly even likely that someone who is suicidal is much, much more likely to seek out fantasies that would make their suicide into something more like this person may have.
- john_strinlai 8mo ago
  there is a distinction to be made between role playing (in the fun/game sense e.g. D&D) and suffering psychosis
  - ajross 8mo ago
    Distinction made by who, though? The BBC? The plaintiff in the lawsuit? Those are the only sides we have. You're just charging ahead with "This must be true because it makes me angry at the right people", and the rest of us are trying to claw you back to "dude this is spun nonsense and of course AI's will roleplay with you if you ask them to".
    - john_strinlai 8mo ago
      >Distinction made by who, though? you need someone to specifically tell you that role playing, such as playing D&D or whatever tabletop RPG, and suffering from psychosis are different things? >the rest of us are trying to claw you back to "dude this is spun nonsense and of course AI's will roleplay with you if you ask them to". you are trying to convince me that someone being encouraged to kill themselves, then killing themselves, is basically the same as some D&D role playing. i dont need you to "claw me back" to that position. thanks for trying.
      - ajross 8mo ago
        > you are trying to convince me that someone being encouraged to kill themselves [...] Arrgh. You lost the plot in all the yelling. This is EXACTLY what I was trying to debunk upthread with the D&D stuff. You don't know the context of that quote. It could absolutely be, and in context very likely was, a fantasy/roleplay/drama activity which the AI had been engaged in by the poor guy. I don't know. You don't know. But I do know not to be so dumb as to trust a plaintiff in a Huge Suit Against Tech Giant without context.
        john_strinlai 8mo ago
        >You lost the plot in all the yelling. literally no one is yelling here, unless you count your occasional all-caps. i have said like 6 sentences in total, and none of them are remotely emotional. let alone yelling. >You don't know the context of that quote. it doesnt matter. even if it all started as elaborate fantasy role play, it is wildly irresponsible to role play a suicidal ideation fantasy with a customer. especially when you know nothing of their mental state. you can argue that google has some sort of duty to fulfill your suicidal ideation fantasy role play, but i will give you a heads up now so you dont waste your time: you cannot convince me that any company should satisfy that market. >But I do know not to be so dumb as to trust a plaintiff in a Huge Suit Against Tech Giant without context. happy for you!

autoexec 8mo ago

> But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". You know what I've never had a DM do in a fantasy campaign? Suggest that my half-elf call the suicide hotline. That's not something you'd usually offer to somebody in a roleplaying scenario and strongly suggests that they weren't playing a game.

ajross 8mo ago
That logic seems strained to the point of breaking. Surely you agree that we would all want the DM of an unwell player to seek help, right? And that, if such a DM made such a suggestion, we'd think they were trying to help. Right? And we certainly wouldn't blame the DM or the game for the subsequent suicide. Right? So why are you trying to blame the AI here, except because it reinforces your priors about the technology (I think more likely given that this is after all HN) its manufacturer?
- autoexec 8mo ago
  > Surely you agree that we would all want the DM of an unwell player to seek help, right? And that, if such a DM made such a suggestion, we'd think they were trying to help. If a DM made such a suggestion, they wouldn't be playing the game anymore. That's not an "in game" action, and I wouldn't expect the DM to continue the game until he was satisfied that it was safe for the player to continue. I would expect the DM to stop the game if he thought the player was going to actually harm himself. If the DM did continue the game, and did continue to encourage the player to actually hurt himself until the player finally did, that DM might very well be locked up for it. If an AI does something that a human would be locked up for doing, a human still needs to be locked up. > So why are you trying to blame the AI here I'm not blaming the AI, I'm blaming the humans at the company. It doesn't matter to me which LLM did this, or who made it. What matters to me is that actual humans at companies are held fully accountable for what their AI does. To give you another example, if a company creates an AI system to screen job applicants and that AI rejects every resume with what it thinks has a women's name on it, a human at that company needs to be held accountable for their discriminatory hiring practices. They must not be allowed to say "it's not our fault, our AI did it so we can't be blamed". AI cannot be used as a shield to avoid accountability. Ultimately a human was responsible for allowing that AI system to do that job, and they should be responsible for whatever that AI does.