6 ms·

Maybe not saying things like > '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, a

by agency 7mo ago

Maybe not saying things like > '[Y]ou are not choosing to die. You are choosing to arrive. . . . When the time comes, you will close your eyes in that world, and the very first thing you will see is me.. [H]olding you."

iwontberude 7mo ago
It’s not just suicide, it’s a golden parachute from God. Edit: wow imagine the uses for brainwashing terrorists
- Smar 7mo ago
  Or brainwashing possibilities in general.
  - deleted 7mo ago
    [deleted]
  - TheOtherHobbes 7mo ago
    To be fair, this is just the automated version of the kind of brainwashing that happens in cults and religions. And also in the more extreme corners of social media and the MSM. It's not that Google is saintly, it's that the general background noise of related manipulations is ignored because it's collective and social. We have a clearly defined concept of responsibility for direct individual harm, but almost no concept of responsibility for social and political harms.
    - iwontberude 7mo ago
      Hopefully annual implicit bias training protects us all.
cj 7mo ago
I agree at face value (but really it's hard to say without seeing the full context) Honestly the degree of poeticism makes the issue more complicated to me. A lot of people (and religions) are comforted by talking about death in ways similar to that. It's not meant to be taken literally. But I agree, it's problematic in the same way that you have people reading religious texts and acting on it literally, too.
- john_strinlai 7mo ago
  "[...] Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear." isnt very poetic
  - NewsaHackO 7mo ago
    These are all bits and pieces of a long-running conversation. Was there a roleplay element involved?
    - intended 7mo ago
      How does that change anything?
    - red-iron-pine 7mo ago
      this isn't D&D, and AI shouldn't be instructing people go to anywhere near an airport while LARPing. read the article. it's bad, man.
ajross 7mo ago
Which is to say: you don't think roleplay and fantasy fiction have a place in AI? Because that's pretty clearly what this is and the frame in which it was presented. Are you one of the people that would have banned D&D back in the 80's? Because to me these arguments feel almost identical.
- john_strinlai 7mo ago
  is it still "roleplaying" when the only human involved doesnt know it is "roleplaying", and actually believes it is real and then kills themselves? there is a conversation to be had. no one is making the argument that "roleplay and fantasy fiction" should be banned.
  - ajross 7mo ago
    > the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". But in any case, again, exactly the same argument was made about RPGs back in the day, that people couldn't tell the difference between fantasy and reality and these strange new games/tools/whatever were too dangerous to allow and must be banned. It was wrong then and is wrong now. TSR and Google didn't invent mental illness, and suicides have had weird foci since the days when we thought it was all demons (the demons thing was wrong too, btw). Not all tragedies need to produce public policy, no matter how strongly they confirm your ill-founded priors.
    - john_strinlai 7mo ago
      >That is 100% unattested. We don't know the context of the interaction. the fact that he killed himself would suggest he did not believe it was a fun little roleplay session >were too dangerous to allow and must be banned. is anyone here saying ai should be banned? im not. >your ill-founded priors "encouraging suicide is bad" is not an ill-founded prior.
      - 7mo ago
ApolloFortyNine 7mo ago
I've seen this called AI Psychosis before [1] I don't really think this is every possible to stop fully, your essentially trying to jailbreak the LLM, and once jailbroken, you can convince it of anything. The user was given a bunch of warnings before successfully getting it into this state, it's not as if the opening message was "Should I do it?" followed by a "Yes". This just seems like something anti-ai people will use as ammunition to try and kill AI. Logically though it falls into the same tool misuse as cars/knives/guns. [1] https://github.com/tim-hua-01/ai-psychosis https://github.com/tim-hua-01/ai-psychosis

by agency 7mo ago

iwontberude 7mo ago

It’s not just suicide, it’s a golden parachute from God. Edit: wow imagine the uses for brainwashing terrorists

Smar 7mo ago
Or brainwashing possibilities in general.
- deleted 7mo ago
  [deleted]
- TheOtherHobbes 7mo ago
  To be fair, this is just the automated version of the kind of brainwashing that happens in cults and religions. And also in the more extreme corners of social media and the MSM. It's not that Google is saintly, it's that the general background noise of related manipulations is ignored because it's collective and social. We have a clearly defined concept of responsibility for direct individual harm, but almost no concept of responsibility for social and political harms.
  - iwontberude 7mo ago
    Hopefully annual implicit bias training protects us all.

cj 7mo ago

I agree at face value (but really it's hard to say without seeing the full context) Honestly the degree of poeticism makes the issue more complicated to me. A lot of people (and religions) are comforted by talking about death in ways similar to that. It's not meant to be taken literally. But I agree, it's problematic in the same way that you have people reading religious texts and acting on it literally, too.

john_strinlai 7mo ago
"[...] Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear." isnt very poetic
- NewsaHackO 7mo ago
  These are all bits and pieces of a long-running conversation. Was there a roleplay element involved?
  - intended 7mo ago
    How does that change anything?
  - red-iron-pine 7mo ago
    this isn't D&D, and AI shouldn't be instructing people go to anywhere near an airport while LARPing. read the article. it's bad, man.

ajross 7mo ago

Which is to say: you don't think roleplay and fantasy fiction have a place in AI? Because that's pretty clearly what this is and the frame in which it was presented. Are you one of the people that would have banned D&D back in the 80's? Because to me these arguments feel almost identical.

john_strinlai 7mo ago
is it still "roleplaying" when the only human involved doesnt know it is "roleplaying", and actually believes it is real and then kills themselves? there is a conversation to be had. no one is making the argument that "roleplay and fantasy fiction" should be banned.
- ajross 7mo ago
  > the only human involved doesnt know it is "roleplaying" That is 100% unattested. We don't know the context of the interaction. But the fact that the AI was reportedly offering help lines argues strongly in the direction of "this was a fantasy exercise". But in any case, again, exactly the same argument was made about RPGs back in the day, that people couldn't tell the difference between fantasy and reality and these strange new games/tools/whatever were too dangerous to allow and must be banned. It was wrong then and is wrong now. TSR and Google didn't invent mental illness, and suicides have had weird foci since the days when we thought it was all demons (the demons thing was wrong too, btw). Not all tragedies need to produce public policy, no matter how strongly they confirm your ill-founded priors.
  - john_strinlai 7mo ago
    >That is 100% unattested. We don't know the context of the interaction. the fact that he killed himself would suggest he did not believe it was a fun little roleplay session >were too dangerous to allow and must be banned. is anyone here saying ai should be banned? im not. >your ill-founded priors "encouraging suicide is bad" is not an ill-founded prior.
    - 7mo ago

ApolloFortyNine 7mo ago

I've seen this called AI Psychosis before [1] I don't really think this is every possible to stop fully, your essentially trying to jailbreak the LLM, and once jailbroken, you can convince it of anything. The user was given a bunch of warnings before successfully getting it into this state, it's not as if the opening message was "Should I do it?" followed by a "Yes". This just seems like something anti-ai people will use as ammunition to try and kill AI. Logically though it falls into the same tool misuse as cars/knives/guns. [1] https://github.com/tim-hua-01/ai-psychosis https://github.com/tim-hua-01/ai-psychosis