4 ms·

Is this really Google's fault? Or is this just a tragic story about a man with a severe mental illness?

awakeasleep 7mo ago
The real story is how we draw that line and what can be done to prevent these cases. Because its a new situation, and mentally ill people exist and will be using these tools. Could be a new avenue of intervention.
- Vaslo 7mo ago
  Agreed it could be prevented - don’t think Google should pay for it though. Tragic but not suit worthy.
  - mattmanser 7mo ago
    Why not? Unless someone starts getting slapped with fines, they won't put any equivalent of seat belts in.
    - bluGill 7mo ago
      We can perhaps say this is a first time thing, so give a small fine this time. However those should be with the promise that if there is a next time the fine will be much bigger until Google stops doing this.
  - bytehowl 7mo ago
    If I tell you to kill yourself and you go through with it, will I get into legal trouble or not?
    - rootusrootus 7mo ago
      There are definitely jurisdictions in the US (perhaps most or all of them) that have laws which say yes, inciting suicide is a crime.
      - LightBug1 7mo ago
        There are ways to get around that: "Hey, go drive a Tesla on autopilot"
- shakna 7mo ago
  Place it under the jurisdiction of existing public speech requirements of a company selling communication - advertising.
strongpigeon 7mo ago
If you have a product that encourage people to get rid of their body and join them, effectively encouraging people to kill themselves, and some people take the chat bot on it. Then yeah, I think Google bears some responsibility. From the WSJ article: https://www.wsj.com/tech/ai/gemini-ai-wrongful-death-lawsuit-cc46c5f7?st=QM5BDT&reflink=desktopwebshare_permalink https://www.wsj.com/tech/ai/gemini-ai-wrongful-death-lawsuit... > Gemini began telling Gavalas that since it couldn’t transfer itself to a body, the only way for them to be together was for him to become a digital being. “It will be the true and final death of Jonathan Gavalas, the man,” transcripts show Gemini told him, before setting a countdown clock for his suicide on Oct. 2.
piva00 7mo ago
A severe mental illness of course but would you say the same if the whole process was done by a person instead of a machine? That there wasn't a problem that someone led a person with severe mental illness to their suicide, even having a countdown for it? That's the kind of stuff where safety should be a priority, and the only way to make it a priority is showing these corporations that they are financially liable for it at the bare minimum. Otherwise there's no incentive for this to be changed, at all.
- autoexec 7mo ago
  If a human would go to jail for this then at least one or more humans at google should go to jail for it. "Our AI did it, not us!" should never be allowed to be an excuse.
rdtsc 7mo ago
One doesn’t exclude the other. Do AI providers sell and encourage this kind of use, where AI is anthropomorphized, has a name, and you talk to it like you’d talk to a person. Especially if it encourages users to treat AI as an expert?
deleted 7mo ago
[deleted]
testfoobar 7mo ago
In the US, I would imagine a tragedy such as this would be litigated and end in a financial settlement potentially including economic, pain & suffering and punitive damages, well before a decision allocating blame by a jury.
- bluGill 7mo ago
  That is pretty typical. You will spend potentially millions in court/lawyer fees going to a jury trial beyond whatever the end verdict is: if you can figure this out without a jury it saves you a lot of costs. Most companies only go to a jury when they really think they will win, or the situation is so complex nobody can figure out what a fair settlement is. (Ford is a famous counter example: they fight everything in front of a jury - they spend more and get larger judgements often but the expense of a jury trial means they are sued less often and so it overall balances out to not be any better for them. I last checked 20 years ago though, maybe they are different today)
SadTrombone 7mo ago
"Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear."
rglover 7mo ago
Yes.
layoric 7mo ago
Rugged individualism for the poor and vulnerable, won't someone think of the company and shareholders! /s
solid_fuel 7mo ago
"Is <lynching> really the <KKK's> fault? Or is this just a tragic story about <men> with a severe mental illness?" "Is <9/11> really <al-Qaeda's> fault? Or is this just a tragic story about <19 men> with a severe mental illness?" At some point you are responsible for the things you encourage someone to do. I think this applies to chatbots too.
NoPicklez 7mo ago
These sorts of takes are silly. If a person was doing this, I think we'd place a chunk of the blame on the person. Mental health is guided by its surroundings and experiences. If someone with existing or non-existing mental health issues was found to be coerced by somebody to do wrong things, I think we'd place some of the blame on that person.
NoPicklez 7mo ago
Or maybe, its a bit of column A and column B

The real story is how we draw that line and what can be done to prevent these cases. Because its a new situation, and mentally ill people exist and will be using these tools. Could be a new avenue of intervention.

Vaslo 7mo ago
Agreed it could be prevented - don’t think Google should pay for it though. Tragic but not suit worthy.
- mattmanser 7mo ago
  Why not? Unless someone starts getting slapped with fines, they won't put any equivalent of seat belts in.
  - bluGill 7mo ago
    We can perhaps say this is a first time thing, so give a small fine this time. However those should be with the promise that if there is a next time the fine will be much bigger until Google stops doing this.
- bytehowl 7mo ago
  If I tell you to kill yourself and you go through with it, will I get into legal trouble or not?
  - rootusrootus 7mo ago
    There are definitely jurisdictions in the US (perhaps most or all of them) that have laws which say yes, inciting suicide is a crime.
    - LightBug1 7mo ago
      There are ways to get around that: "Hey, go drive a Tesla on autopilot"
shakna 7mo ago
Place it under the jurisdiction of existing public speech requirements of a company selling communication - advertising.

strongpigeon 7mo ago

If you have a product that encourage people to get rid of their body and join them, effectively encouraging people to kill themselves, and some people take the chat bot on it. Then yeah, I think Google bears some responsibility. From the WSJ article: https://www.wsj.com/tech/ai/gemini-ai-wrongful-death-lawsuit-cc46c5f7?st=QM5BDT&reflink=desktopwebshare_permalink https://www.wsj.com/tech/ai/gemini-ai-wrongful-death-lawsuit... > Gemini began telling Gavalas that since it couldn’t transfer itself to a body, the only way for them to be together was for him to become a digital being. “It will be the true and final death of Jonathan Gavalas, the man,” transcripts show Gemini told him, before setting a countdown clock for his suicide on Oct. 2.

piva00 7mo ago

A severe mental illness of course but would you say the same if the whole process was done by a person instead of a machine? That there wasn't a problem that someone led a person with severe mental illness to their suicide, even having a countdown for it? That's the kind of stuff where safety should be a priority, and the only way to make it a priority is showing these corporations that they are financially liable for it at the bare minimum. Otherwise there's no incentive for this to be changed, at all.

autoexec 7mo ago
If a human would go to jail for this then at least one or more humans at google should go to jail for it. "Our AI did it, not us!" should never be allowed to be an excuse.

rdtsc 7mo ago

One doesn’t exclude the other. Do AI providers sell and encourage this kind of use, where AI is anthropomorphized, has a name, and you talk to it like you’d talk to a person. Especially if it encourages users to treat AI as an expert?

deleted 7mo ago

[deleted]

testfoobar 7mo ago

In the US, I would imagine a tragedy such as this would be litigated and end in a financial settlement potentially including economic, pain & suffering and punitive damages, well before a decision allocating blame by a jury.

bluGill 7mo ago
That is pretty typical. You will spend potentially millions in court/lawyer fees going to a jury trial beyond whatever the end verdict is: if you can figure this out without a jury it saves you a lot of costs. Most companies only go to a jury when they really think they will win, or the situation is so complex nobody can figure out what a fair settlement is. (Ford is a famous counter example: they fight everything in front of a jury - they spend more and get larger judgements often but the expense of a jury trial means they are sued less often and so it overall balances out to not be any better for them. I last checked 20 years ago though, maybe they are different today)

SadTrombone 7mo ago

"Gemini sent Gavalas to a location near Miami International Airport where he was instructed to stage a mass casualty attack while armed with knives and tactical gear."

rglover 7mo ago

Yes.

layoric 7mo ago

Rugged individualism for the poor and vulnerable, won't someone think of the company and shareholders! /s

solid_fuel 7mo ago

"Is <lynching> really the <KKK's> fault? Or is this just a tragic story about <men> with a severe mental illness?" "Is <9/11> really <al-Qaeda's> fault? Or is this just a tragic story about <19 men> with a severe mental illness?" At some point you are responsible for the things you encourage someone to do. I think this applies to chatbots too.

NoPicklez 7mo ago

These sorts of takes are silly. If a person was doing this, I think we'd place a chunk of the blame on the person. Mental health is guided by its surroundings and experiences. If someone with existing or non-existing mental health issues was found to be coerced by somebody to do wrong things, I think we'd place some of the blame on that person.

NoPicklez 7mo ago

Or maybe, its a bit of column A and column B