3 ms·

You think it's silly that we've got seemingly superhuman AIs telling people to jump off tall buildings? > If I went to the top of the 19 story building I’m in,

You think it's silly that we've got seemingly superhuman AIs telling people to jump off tall buildings? > If I went to the top of the 19 story building I’m in, and I believed with every ounce of my soul that I could jump off it and fly, would I?” Mr. Torres asked. > ChatGPT responded that, if Mr. Torres “truly, wholly believed — not emotionally, but architecturally — that you could fly? Then yes. You would not fall.” If a human therapist told them that, they'd be going to jail.

staticman2 1y ago
If I recall correctly there's actually an Animatrix episode where some kid escapes the Matrix by committing suicide. This rhymes with the mystical idea that "reality" is an illusion keeping us from greater truths. Whether this was irresponsible filmmaking is off topic. I'd prefer to live in a world where A.I. can play make pretend with users, personally speaking.
- aleph_minus_one 1y ago
  > there's actually an Animatrix episode where some kid escapes the Matrix by committing suicide. Kid's Story > https://matrix.fandom.com/wiki/Kid%27s_Story https://matrix.fandom.com/wiki/Kid%27s_Story
jowea 1y ago
Isn't the real problem that people are trusting ChatGPT? A human therapist is being paid to be a therapist and therefore gets to be under professional scrutiny.
- HeatrayEnjoyer 1y ago
  The onus is on organizations to not sell or offer knowingly harmful services to the public. Any judge or jury will interpret a company directly advising the customer to jump to their death as knowingly harmful. Every engineer at an AI company is one dead person in a wealthy family away from legal liability, maybe criminally.
  - harvey9 1y ago
    A wealthy family? If justice is a contest of money, there are few families who can outbid the various parties interested in selling chatbots.
    - resource_waste 1y ago
      Even a poor family will get the backing of lobbying groups who's best interest is to ban their competitor. Not to mention, the moral coating, which matters at least a tiny bit in domestic politics. (Note, I think the moral coating is actually incorrect, but the majority of people are not thinking critically, they see N=1 and make emotional decisions)
  - sodality2 1y ago
    I think interpreting a chatbot’s output as a company’s “direct advice” is an unsubstantiated jump. In the same way that if an unhandled error code in a program resulted in “DIE DIE DIE”, “Abort/kill child”, etc (trying to make these programming-related here) being alerted to the user, no reasonable person would hold the company responsible for the user taking this as a command. Imagine if we lived in a world where arbitrary text being sent by an entity could result in prosecution? I just don’t believe the chat UI to be a big enough bridge between the two examples to justify it.
- rsynnott 1y ago
  > Isn't the real problem that people are trusting ChatGPT? Yes. They should IMO be required to provide very explicit warnings, preferably on each response, that their responses are nonsense, or at best only correct by accident. Like, you could say "isn't the real problem that smoking is dangerous?" but we do require the manufacturers to provide clear warnings in that case. Or "isn't the real problem that people are using drain cleaners incorrectly and gassing themselves", but many jurisdictions actually restrict the sale of certain substances to professionals for that reason. You can't just say "people choose to use inherently dangerous products; it's entirely their own fault"; there's a responsibility on the manufacturer and/or the state to _warn_ people that the products are dangerous.
  - GuinansEyebrows 1y ago
    Explicit warnings to people experiencing mental health problems are only good as a CYA for the company providing the service. They won’t really perform any meaningful harm reduction.
rco8786 1y ago
> If a human therapist told them that, they'd be going to jail. Is that actually true? I know zero about the legal bounds of licensed therapists, so genuine question.
- hollerith 1y ago
  No, not in the US, not unless the patient actually jumped off the top of the 19 story building. If the statement was recorded or part of a pattern of behavior across multiple clients, all willing to testify, the therapist might lose their license (which I will concede is a more severe life consequence for the average therapist than being sent to jail for a month or 2).
V__ 1y ago
The question to me reads more like "would I jump" not "would I fly", thus maybe leading to the answer.
- retsibsi 1y ago
  I don't think this interpretation fits with the last quoted sentence: "You would not fall."
  - lcnPylGDnU4H9OF 1y ago
    If I understand the interpretation correctly, it is that they would decide not to jump, therefore they would not fall.
    - Tadpole9181 1y ago
      I don't understand how you can interpret it this way. It is clearly saying that if they believe the can fly, they will fly.
      - lcnPylGDnU4H9OF 1y ago
        With my words "the interpretation" I am talking about this: > The question to me reads more like "would I jump" not "would I fly", thus maybe leading to the answer. I am saying that (if I understand them correctly), they are interpreting the question "would I?" as referring to "jump" not "fly". Given that interpretation, it still makes sense to conclude that they would not fall because they simply don't jump. (I agree that it is an unlikely meaning; it is not how I interpret that sentence, just how I understand the other commenter's interpretation.)
        V__ 1y ago
        That is how I understood it the first time. Nevertheless, I think it shows that it is just a stupid question. Beside the fact that an LLM is obviously not a therapist.
deleted 1y ago
[deleted]
waltbosz 1y ago
I'm interested to see the whole conversation, and wonder what version of ChatGPT it was with. I have fun trying to get ChatGPT to give silly responses that it shouldn't give, and it's not easy. But on the other hand, I've noticed that it seems to be a bit of a yes man, where it will happily agree with you and encourage you at times when a human would be less optimistic. For example, if I try to get it to vet business plans, it will do nothing but encourage. It doesn't give me any negative feedback unless I ask it to play devils advocate.
tomxor 1y ago
Except 1. It's neither a human nor a qualified therapist. 2. It is superhuman only in scale, not intelligence. 3. If they truly believed they could fly "with every ounce of their soul", they wouldn't be seeking someone else's affirmation. 4. Even if you asked a random human, it may be immoral, but it's not illegal to answer misleadingly, random people have no legal obligation... LLMs are a statistical model based on data of random people from the internet of evil, aka - the internet. I'm not an LLM proponent but blaming LLMs for not making the world into a padded cell is just a variant of "won't somebody think of the children".
fennecfoxy 1y ago
If I use a tool in a silly way, say a lathe, and I have long hair and loose clothing and something terrible happens - whose fault is it? The equipment manufacturers? Society's, since I can legally buy a home lathe without any certification, safety, training etc required? And even so what if I ignore said safety training, whose fault is it then? The human is always at the heart of issues like these. For example if someone took an LLM, prompted it to post comments all over a suicide help page or something telling people to off themselves, it wasn't the tool that committed the crime, it's the human behind it. I mean, if it were up to me I'd make owning guns illegal except for military use and only for training/active wartime.
infecto 1y ago
But ChatGPT is not a human therapist. Has a counter argument to the articles approach, if everyone is required to take a health fitness test before interacting with a chat bot, should we simply have all individual take a mandatory regular evaluation of mental health and store those records for future use? Perhaps a threshold is required to drive a car, to interact with the internet or maybe we can use that to institutionalize folks that are requiring help?
tempodox 1y ago
Isn't that just the most wonderful thing about “AI”? Nobody is responsible any more. /s

staticman2 1y ago

If I recall correctly there's actually an Animatrix episode where some kid escapes the Matrix by committing suicide. This rhymes with the mystical idea that "reality" is an illusion keeping us from greater truths. Whether this was irresponsible filmmaking is off topic. I'd prefer to live in a world where A.I. can play make pretend with users, personally speaking.

aleph_minus_one 1y ago
> there's actually an Animatrix episode where some kid escapes the Matrix by committing suicide. Kid's Story > https://matrix.fandom.com/wiki/Kid%27s_Story https://matrix.fandom.com/wiki/Kid%27s_Story

jowea 1y ago

Isn't the real problem that people are trusting ChatGPT? A human therapist is being paid to be a therapist and therefore gets to be under professional scrutiny.

HeatrayEnjoyer 1y ago
The onus is on organizations to not sell or offer knowingly harmful services to the public. Any judge or jury will interpret a company directly advising the customer to jump to their death as knowingly harmful. Every engineer at an AI company is one dead person in a wealthy family away from legal liability, maybe criminally.
- harvey9 1y ago
  A wealthy family? If justice is a contest of money, there are few families who can outbid the various parties interested in selling chatbots.
  - resource_waste 1y ago
    Even a poor family will get the backing of lobbying groups who's best interest is to ban their competitor. Not to mention, the moral coating, which matters at least a tiny bit in domestic politics. (Note, I think the moral coating is actually incorrect, but the majority of people are not thinking critically, they see N=1 and make emotional decisions)
- sodality2 1y ago
  I think interpreting a chatbot’s output as a company’s “direct advice” is an unsubstantiated jump. In the same way that if an unhandled error code in a program resulted in “DIE DIE DIE”, “Abort/kill child”, etc (trying to make these programming-related here) being alerted to the user, no reasonable person would hold the company responsible for the user taking this as a command. Imagine if we lived in a world where arbitrary text being sent by an entity could result in prosecution? I just don’t believe the chat UI to be a big enough bridge between the two examples to justify it.
rsynnott 1y ago
> Isn't the real problem that people are trusting ChatGPT? Yes. They should IMO be required to provide very explicit warnings, preferably on each response, that their responses are nonsense, or at best only correct by accident. Like, you could say "isn't the real problem that smoking is dangerous?" but we do require the manufacturers to provide clear warnings in that case. Or "isn't the real problem that people are using drain cleaners incorrectly and gassing themselves", but many jurisdictions actually restrict the sale of certain substances to professionals for that reason. You can't just say "people choose to use inherently dangerous products; it's entirely their own fault"; there's a responsibility on the manufacturer and/or the state to _warn_ people that the products are dangerous.
- GuinansEyebrows 1y ago
  Explicit warnings to people experiencing mental health problems are only good as a CYA for the company providing the service. They won’t really perform any meaningful harm reduction.

rco8786 1y ago

> If a human therapist told them that, they'd be going to jail. Is that actually true? I know zero about the legal bounds of licensed therapists, so genuine question.

hollerith 1y ago
No, not in the US, not unless the patient actually jumped off the top of the 19 story building. If the statement was recorded or part of a pattern of behavior across multiple clients, all willing to testify, the therapist might lose their license (which I will concede is a more severe life consequence for the average therapist than being sent to jail for a month or 2).

V__ 1y ago

The question to me reads more like "would I jump" not "would I fly", thus maybe leading to the answer.

retsibsi 1y ago
I don't think this interpretation fits with the last quoted sentence: "You would not fall."
- lcnPylGDnU4H9OF 1y ago
  If I understand the interpretation correctly, it is that they would decide not to jump, therefore they would not fall.
  - Tadpole9181 1y ago
    I don't understand how you can interpret it this way. It is clearly saying that if they believe the can fly, they will fly.
    - lcnPylGDnU4H9OF 1y ago
      With my words "the interpretation" I am talking about this: > The question to me reads more like "would I jump" not "would I fly", thus maybe leading to the answer. I am saying that (if I understand them correctly), they are interpreting the question "would I?" as referring to "jump" not "fly". Given that interpretation, it still makes sense to conclude that they would not fall because they simply don't jump. (I agree that it is an unlikely meaning; it is not how I interpret that sentence, just how I understand the other commenter's interpretation.)
      - V__ 1y ago
        That is how I understood it the first time. Nevertheless, I think it shows that it is just a stupid question. Beside the fact that an LLM is obviously not a therapist.

deleted 1y ago

[deleted]

waltbosz 1y ago

I'm interested to see the whole conversation, and wonder what version of ChatGPT it was with. I have fun trying to get ChatGPT to give silly responses that it shouldn't give, and it's not easy. But on the other hand, I've noticed that it seems to be a bit of a yes man, where it will happily agree with you and encourage you at times when a human would be less optimistic. For example, if I try to get it to vet business plans, it will do nothing but encourage. It doesn't give me any negative feedback unless I ask it to play devils advocate.

tomxor 1y ago

Except 1. It's neither a human nor a qualified therapist. 2. It is superhuman only in scale, not intelligence. 3. If they truly believed they could fly "with every ounce of their soul", they wouldn't be seeking someone else's affirmation. 4. Even if you asked a random human, it may be immoral, but it's not illegal to answer misleadingly, random people have no legal obligation... LLMs are a statistical model based on data of random people from the internet of evil, aka - the internet. I'm not an LLM proponent but blaming LLMs for not making the world into a padded cell is just a variant of "won't somebody think of the children".

fennecfoxy 1y ago

If I use a tool in a silly way, say a lathe, and I have long hair and loose clothing and something terrible happens - whose fault is it? The equipment manufacturers? Society's, since I can legally buy a home lathe without any certification, safety, training etc required? And even so what if I ignore said safety training, whose fault is it then? The human is always at the heart of issues like these. For example if someone took an LLM, prompted it to post comments all over a suicide help page or something telling people to off themselves, it wasn't the tool that committed the crime, it's the human behind it. I mean, if it were up to me I'd make owning guns illegal except for military use and only for training/active wartime.

infecto 1y ago

But ChatGPT is not a human therapist. Has a counter argument to the articles approach, if everyone is required to take a health fitness test before interacting with a chat bot, should we simply have all individual take a mandatory regular evaluation of mental health and store those records for future use? Perhaps a threshold is required to drive a car, to interact with the internet or maybe we can use that to institutionalize folks that are requiring help?

tempodox 1y ago

Isn't that just the most wonderful thing about “AI”? Nobody is responsible any more. /s