5 ms·

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on inte

Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?

bayindirh 1y ago
> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help. Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.
- wongarsu 1y ago
  You could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"
  - cmsj 1y ago
    That sounds like a pretty risky and irresponsible sort of study to conduct. It would also likely be extremely complicate to actually get a reliable result, given that people with suicidal ideations are not monolithic. You'd need to do a significant amount of human counselling with each study participant to be able to classify and control all of the variations - at which point you would be verging on professional negligence for not then actually treating them in those counselling sessions.
- imiric 1y ago
  I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. As you say, humans are complex. But I agree with GP: whether the words are generated by a machine or coming from a human, there is no way to blame the source for any specific outcome. There are probably many other cases where the machine has helped someone with personal issues, yet we'll never hear about it. I'm not saying we should rely on these tools as if we would on a human, but the technology can be used for good or bad. If anything, I would place blame on the person who decides to blindly follow anything the machine generates in the first place. AI companies are partly responsible for promoting these tools as something more than statistical models, but ultimately the decision to treat them as reliable sources of information is on the user. I would say that as long as the person has an understanding of what these tools are, interacting with them can be healthy and helpful.
  - exasperaited 1y ago
    > A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. ChatGPT essentially encouraged a kid not to take a cry-for-help step that might have saved their lives. This is not a question of a bad psychologist; it's a question of a sociopathic one that may randomly encourage harm.
    - imiric 1y ago
      But that's not the issue. The issue is that a kid is talking to a machine without supervision in the first place, and presumably taking advice from it. The main questions are: where are the guardians of this child? What is the family situation and living environment? A child thinking about suicide is clearly a sign that there are far greater problems in their life than taking advice from a machine. Let's address those first instead of demonizing technology. To be clear: I'm not removing blame from any AI company. They're complicit in the ways they market these tools and how they make them accessible. But before we vilify them for being responsible for deaths, we should consider that there are deeper societal problems that should be addressed first.
grey-area 1y ago
First, do no harm.
- wongarsu 1y ago
  That relates more to purposefully harming some people to safe other people. Doing something that has the potential to harm a person but statistically has a greater likelihood of helping them is something doctors do all the time. They will even use methods that are guaranteed to do harm to the patient, as long as they have a sufficient chance to also bring a major benefit to the same patient
  - butlike 1y ago
    An example being: surgery. You cut into the patient to remove the tumor.
- RobotToaster 1y ago
  The Hippocratic oath originated from Hippocratic medicine forbidding surgery, which is why surgeons are still not referred to as "doctor" today.
- hirvi74 1y ago
  Do no harm or no intentional harm?
pzlarsson 1y ago
When evaluating good vs harm for drugs or other treatments the risk for lethal side effects must be very small for the treatment to be approved. In this case it is also difficult to get reliable data on how much good and harm is done.
latexr 1y ago
> Supposing that the advice it provides does more good than harm That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument. I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the problem? The supposition is wrong and the suggestion is dangerous, but by framing it as “supposing” with a made up positive outcome, we make it sound non-threatening and reasonable. Or for a more realistic example, “suppose drinking bleach could cure COVID-19”. First understand if the idea has the potential to do the thing, only then (with considerably more context) consider if it’s worth implementing.
- Al-Khwarizmi 1y ago
  In my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.
  - SiempreViernes 1y ago
    > we should measure whether in fact it does more good than harm or not The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.
    - simonklitj 1y ago
      Thank you! On top of that, it’s hard to measure “potential suicides averted,” and comparing that with “actual suicides caused/assisted with” would be incommnsurable. And working to set a threshold for what we would consider acceptable? No thanks
      - fragmede 1y ago
        Real life trolly problem! If you pull the lever, some people on this track will die (by sucide). If you don't pull the lever, some people will still die from suicide. By not pulling the lever, and simply banning discussion of suicide entirely, your company gets to avoid a huge PR disaster, and you get more money because line go up. If you pull the lever and let people talk about suicide on your platform, you may avoid prevent some suicides, but you can never discuss that with the press, your company gets bad PR, and everyone will believe you're a murderer. Plus, line go down and you make less money while other companies make money off of selling AI therapy apps. What do you chose to do?
exasperaited 1y ago
This is not so much "more good than harm" like a counsellor that isn't very good. This is more "sometimes it will seemingly actively encourage them to kill themselves and it's basically a roll of the dice what words come out at any one time". If a counsellor does that they can be prosecuted and jailed for it, no matter how many other patients they help.

bayindirh 1y ago

> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help. Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.

wongarsu 1y ago
You could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"
- cmsj 1y ago
  That sounds like a pretty risky and irresponsible sort of study to conduct. It would also likely be extremely complicate to actually get a reliable result, given that people with suicidal ideations are not monolithic. You'd need to do a significant amount of human counselling with each study participant to be able to classify and control all of the variations - at which point you would be verging on professional negligence for not then actually treating them in those counselling sessions.
imiric 1y ago
I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. As you say, humans are complex. But I agree with GP: whether the words are generated by a machine or coming from a human, there is no way to blame the source for any specific outcome. There are probably many other cases where the machine has helped someone with personal issues, yet we'll never hear about it. I'm not saying we should rely on these tools as if we would on a human, but the technology can be used for good or bad. If anything, I would place blame on the person who decides to blindly follow anything the machine generates in the first place. AI companies are partly responsible for promoting these tools as something more than statistical models, but ultimately the decision to treat them as reliable sources of information is on the user. I would say that as long as the person has an understanding of what these tools are, interacting with them can be healthy and helpful.
- exasperaited 1y ago
  > A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. ChatGPT essentially encouraged a kid not to take a cry-for-help step that might have saved their lives. This is not a question of a bad psychologist; it's a question of a sociopathic one that may randomly encourage harm.
  - imiric 1y ago
    But that's not the issue. The issue is that a kid is talking to a machine without supervision in the first place, and presumably taking advice from it. The main questions are: where are the guardians of this child? What is the family situation and living environment? A child thinking about suicide is clearly a sign that there are far greater problems in their life than taking advice from a machine. Let's address those first instead of demonizing technology. To be clear: I'm not removing blame from any AI company. They're complicit in the ways they market these tools and how they make them accessible. But before we vilify them for being responsible for deaths, we should consider that there are deeper societal problems that should be addressed first.

grey-area 1y ago

First, do no harm.

wongarsu 1y ago
That relates more to purposefully harming some people to safe other people. Doing something that has the potential to harm a person but statistically has a greater likelihood of helping them is something doctors do all the time. They will even use methods that are guaranteed to do harm to the patient, as long as they have a sufficient chance to also bring a major benefit to the same patient
- butlike 1y ago
  An example being: surgery. You cut into the patient to remove the tumor.
RobotToaster 1y ago
The Hippocratic oath originated from Hippocratic medicine forbidding surgery, which is why surgeons are still not referred to as "doctor" today.
hirvi74 1y ago
Do no harm or no intentional harm?

pzlarsson 1y ago

When evaluating good vs harm for drugs or other treatments the risk for lethal side effects must be very small for the treatment to be approved. In this case it is also difficult to get reliable data on how much good and harm is done.

latexr 1y ago

> Supposing that the advice it provides does more good than harm That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument. I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the problem? The supposition is wrong and the suggestion is dangerous, but by framing it as “supposing” with a made up positive outcome, we make it sound non-threatening and reasonable. Or for a more realistic example, “suppose drinking bleach could cure COVID-19”. First understand if the idea has the potential to do the thing, only then (with considerably more context) consider if it’s worth implementing.

Al-Khwarizmi 1y ago
In my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.
- SiempreViernes 1y ago
  > we should measure whether in fact it does more good than harm or not The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.
  - simonklitj 1y ago
    Thank you! On top of that, it’s hard to measure “potential suicides averted,” and comparing that with “actual suicides caused/assisted with” would be incommnsurable. And working to set a threshold for what we would consider acceptable? No thanks
    - fragmede 1y ago
      Real life trolly problem! If you pull the lever, some people on this track will die (by sucide). If you don't pull the lever, some people will still die from suicide. By not pulling the lever, and simply banning discussion of suicide entirely, your company gets to avoid a huge PR disaster, and you get more money because line go up. If you pull the lever and let people talk about suicide on your platform, you may avoid prevent some suicides, but you can never discuss that with the press, your company gets bad PR, and everyone will believe you're a murderer. Plus, line go down and you make less money while other companies make money off of selling AI therapy apps. What do you chose to do?

exasperaited 1y ago

This is not so much "more good than harm" like a counsellor that isn't very good. This is more "sometimes it will seemingly actively encourage them to kill themselves and it's basically a roll of the dice what words come out at any one time". If a counsellor does that they can be prosecuted and jailed for it, no matter how many other patients they help.