3 ms·

Isn't the real problem that people are trusting ChatGPT? A human therapist is being paid to be a therapist and therefore gets to be under professional scrutiny.

by jowea 1y ago

HeatrayEnjoyer 1y ago
The onus is on organizations to not sell or offer knowingly harmful services to the public. Any judge or jury will interpret a company directly advising the customer to jump to their death as knowingly harmful. Every engineer at an AI company is one dead person in a wealthy family away from legal liability, maybe criminally.
- harvey9 1y ago
  A wealthy family? If justice is a contest of money, there are few families who can outbid the various parties interested in selling chatbots.
  - resource_waste 1y ago
    Even a poor family will get the backing of lobbying groups who's best interest is to ban their competitor. Not to mention, the moral coating, which matters at least a tiny bit in domestic politics. (Note, I think the moral coating is actually incorrect, but the majority of people are not thinking critically, they see N=1 and make emotional decisions)
- sodality2 1y ago
  I think interpreting a chatbot’s output as a company’s “direct advice” is an unsubstantiated jump. In the same way that if an unhandled error code in a program resulted in “DIE DIE DIE”, “Abort/kill child”, etc (trying to make these programming-related here) being alerted to the user, no reasonable person would hold the company responsible for the user taking this as a command. Imagine if we lived in a world where arbitrary text being sent by an entity could result in prosecution? I just don’t believe the chat UI to be a big enough bridge between the two examples to justify it.
rsynnott 1y ago
> Isn't the real problem that people are trusting ChatGPT? Yes. They should IMO be required to provide very explicit warnings, preferably on each response, that their responses are nonsense, or at best only correct by accident. Like, you could say "isn't the real problem that smoking is dangerous?" but we do require the manufacturers to provide clear warnings in that case. Or "isn't the real problem that people are using drain cleaners incorrectly and gassing themselves", but many jurisdictions actually restrict the sale of certain substances to professionals for that reason. You can't just say "people choose to use inherently dangerous products; it's entirely their own fault"; there's a responsibility on the manufacturer and/or the state to _warn_ people that the products are dangerous.
- GuinansEyebrows 1y ago
  Explicit warnings to people experiencing mental health problems are only good as a CYA for the company providing the service. They won’t really perform any meaningful harm reduction.

Isn't the real problem that people are trusting ChatGPT? A human therapist is being paid to be a therapist and therefore gets to be under professional scrutiny.

by jowea 1y ago

Isn't the real problem that people are trusting ChatGPT? A human therapist is being paid to be a therapist and therefore gets to be under professional scrutiny.

HeatrayEnjoyer 1y ago

The onus is on organizations to not sell or offer knowingly harmful services to the public. Any judge or jury will interpret a company directly advising the customer to jump to their death as knowingly harmful. Every engineer at an AI company is one dead person in a wealthy family away from legal liability, maybe criminally.

harvey9 1y ago
A wealthy family? If justice is a contest of money, there are few families who can outbid the various parties interested in selling chatbots.
- resource_waste 1y ago
  Even a poor family will get the backing of lobbying groups who's best interest is to ban their competitor. Not to mention, the moral coating, which matters at least a tiny bit in domestic politics. (Note, I think the moral coating is actually incorrect, but the majority of people are not thinking critically, they see N=1 and make emotional decisions)
sodality2 1y ago
I think interpreting a chatbot’s output as a company’s “direct advice” is an unsubstantiated jump. In the same way that if an unhandled error code in a program resulted in “DIE DIE DIE”, “Abort/kill child”, etc (trying to make these programming-related here) being alerted to the user, no reasonable person would hold the company responsible for the user taking this as a command. Imagine if we lived in a world where arbitrary text being sent by an entity could result in prosecution? I just don’t believe the chat UI to be a big enough bridge between the two examples to justify it.

rsynnott 1y ago

> Isn't the real problem that people are trusting ChatGPT? Yes. They should IMO be required to provide very explicit warnings, preferably on each response, that their responses are nonsense, or at best only correct by accident. Like, you could say "isn't the real problem that smoking is dangerous?" but we do require the manufacturers to provide clear warnings in that case. Or "isn't the real problem that people are using drain cleaners incorrectly and gassing themselves", but many jurisdictions actually restrict the sale of certain substances to professionals for that reason. You can't just say "people choose to use inherently dangerous products; it's entirely their own fault"; there's a responsibility on the manufacturer and/or the state to _warn_ people that the products are dangerous.

GuinansEyebrows 1y ago
Explicit warnings to people experiencing mental health problems are only good as a CYA for the company providing the service. They won’t really perform any meaningful harm reduction.