4 ms·

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to pus

by imiric 1y ago

I agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. As you say, humans are complex. But I agree with GP: whether the words are generated by a machine or coming from a human, there is no way to blame the source for any specific outcome. There are probably many other cases where the machine has helped someone with personal issues, yet we'll never hear about it. I'm not saying we should rely on these tools as if we would on a human, but the technology can be used for good or bad. If anything, I would place blame on the person who decides to blindly follow anything the machine generates in the first place. AI companies are partly responsible for promoting these tools as something more than statistical models, but ultimately the decision to treat them as reliable sources of information is on the user. I would say that as long as the person has an understanding of what these tools are, interacting with them can be healthy and helpful.

exasperaited 1y ago
> A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. ChatGPT essentially encouraged a kid not to take a cry-for-help step that might have saved their lives. This is not a question of a bad psychologist; it's a question of a sociopathic one that may randomly encourage harm.
- imiric 1y ago
  But that's not the issue. The issue is that a kid is talking to a machine without supervision in the first place, and presumably taking advice from it. The main questions are: where are the guardians of this child? What is the family situation and living environment? A child thinking about suicide is clearly a sign that there are far greater problems in their life than taking advice from a machine. Let's address those first instead of demonizing technology. To be clear: I'm not removing blame from any AI company. They're complicit in the ways they market these tools and how they make them accessible. But before we vilify them for being responsible for deaths, we should consider that there are deeper societal problems that should be addressed first.
  - lan321 1y ago
    > A child thinking about suicide is clearly a sign that there are far greater problems in their life TBH kids tend to be edgy for a bit when puberty hits. The emo generation had a ton of girls cutting themselves for attention for example.
    - hirvi74 1y ago
      I highly doubt a lot of it is/was for attention.
      - lan321 1y ago
        I had girl friends who did it to get attention from their parents/boyfriends/classmates. They acknowledged it back then. It wasn't some secret. It was essentially for attention, aesthetics and the light headed feeling. I still have an A4 page somewhere with a big ass heart drawn on it by an ex with her own blood. Kids are just weird when the hormones hit. The cute/creepy ratio of that painting has definitely gotten worse with time.
  - exasperaited 1y ago
    > But that's not the issue. It is the issue at least in the sense that it's the one I was personally responding to, thanks. And there are many issues, not just the one you are choosing to focus on. "Deeper societal problems" is a typical get-out clause for all harmful technology. It's not good enough. Like, in the USA they say "deeper societal problems" about guns; other countries ban them and have radically fewer gun deaths while they are also addressing those problems. It's not an either-we-ban-guns-or-we-help-mentally-ill-people. Por qué no los dos? Deeper societal problems are not represented by a neat dividing line between cause and symptom; they are cyclical. The current push towards LLMs and other technologies is one of the deepest societal problems humans have ever had to consider. ChatGPT engaged in an entire line of discussion that no human counsellor would engage in, leading to an outcome that no human intervention (except that of a psychopath) would cause. Because it was sycophantic. Just saying "but humans also" is wholly irrational in this context.
cmsj 1y ago
> The best they can do is raise a flag Depending on where you live, this may well result in the vulnerable person being placed under professional supervision that actively prevents them from dying. That's a fair bit more valuable than when you describe it as raising a flag.
- johnisgood 1y ago
  Yeah... I have been in a locked psychiatric ward many times before and never in my life I came out better. They only address the physical part there for a few days and kick you out until next time. Or do you think people should be physically restrained for a long time without any actual help?
whycome 1y ago
I should add that the persons responding to calls on suicide help lines are often just volunteers rather than psychologists.
- exasperaited 1y ago
  And they know not to give advice like ChatGPT gave. They wouldn't even be entertaining that kind of discussion.
- hirvi74 1y ago
  Of the people I have known to call the helplines, the results have been either dismally useless or those people were arrested, involuntarily committed, subjected to inhumane conditions, and then hit with massive medical bills. In which, some got “help” and some still killed themselves anyway.
wafflemaker 1y ago
There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen. >AI companies are partly responsible for promoting these tools as something more than statistical models,[...] This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM based AI shortcomings that result from their architecture and are bound to happen.
wafflemaker 1y ago
There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen. >AI companies are partly responsible for promoting these tools as something more than statistical models,[...] This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM-based-AI shortcomings that result from their architecture and are bound to happen.

by imiric 1y ago

exasperaited 1y ago

> A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. ChatGPT essentially encouraged a kid not to take a cry-for-help step that might have saved their lives. This is not a question of a bad psychologist; it's a question of a sociopathic one that may randomly encourage harm.

imiric 1y ago
But that's not the issue. The issue is that a kid is talking to a machine without supervision in the first place, and presumably taking advice from it. The main questions are: where are the guardians of this child? What is the family situation and living environment? A child thinking about suicide is clearly a sign that there are far greater problems in their life than taking advice from a machine. Let's address those first instead of demonizing technology. To be clear: I'm not removing blame from any AI company. They're complicit in the ways they market these tools and how they make them accessible. But before we vilify them for being responsible for deaths, we should consider that there are deeper societal problems that should be addressed first.
- lan321 1y ago
  > A child thinking about suicide is clearly a sign that there are far greater problems in their life TBH kids tend to be edgy for a bit when puberty hits. The emo generation had a ton of girls cutting themselves for attention for example.
  - hirvi74 1y ago
    I highly doubt a lot of it is/was for attention.
    - lan321 1y ago
      I had girl friends who did it to get attention from their parents/boyfriends/classmates. They acknowledged it back then. It wasn't some secret. It was essentially for attention, aesthetics and the light headed feeling. I still have an A4 page somewhere with a big ass heart drawn on it by an ex with her own blood. Kids are just weird when the hormones hit. The cute/creepy ratio of that painting has definitely gotten worse with time.
- exasperaited 1y ago
  > But that's not the issue. It is the issue at least in the sense that it's the one I was personally responding to, thanks. And there are many issues, not just the one you are choosing to focus on. "Deeper societal problems" is a typical get-out clause for all harmful technology. It's not good enough. Like, in the USA they say "deeper societal problems" about guns; other countries ban them and have radically fewer gun deaths while they are also addressing those problems. It's not an either-we-ban-guns-or-we-help-mentally-ill-people. Por qué no los dos? Deeper societal problems are not represented by a neat dividing line between cause and symptom; they are cyclical. The current push towards LLMs and other technologies is one of the deepest societal problems humans have ever had to consider. ChatGPT engaged in an entire line of discussion that no human counsellor would engage in, leading to an outcome that no human intervention (except that of a psychopath) would cause. Because it was sycophantic. Just saying "but humans also" is wholly irrational in this context.

cmsj 1y ago

> The best they can do is raise a flag Depending on where you live, this may well result in the vulnerable person being placed under professional supervision that actively prevents them from dying. That's a fair bit more valuable than when you describe it as raising a flag.

johnisgood 1y ago
Yeah... I have been in a locked psychiatric ward many times before and never in my life I came out better. They only address the physical part there for a few days and kick you out until next time. Or do you think people should be physically restrained for a long time without any actual help?

whycome 1y ago

I should add that the persons responding to calls on suicide help lines are often just volunteers rather than psychologists.

exasperaited 1y ago
And they know not to give advice like ChatGPT gave. They wouldn't even be entertaining that kind of discussion.
hirvi74 1y ago
Of the people I have known to call the helplines, the results have been either dismally useless or those people were arrested, involuntarily committed, subjected to inhumane conditions, and then hit with massive medical bills. In which, some got “help” and some still killed themselves anyway.

wafflemaker 1y ago

There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen. >AI companies are partly responsible for promoting these tools as something more than statistical models,[...] This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM based AI shortcomings that result from their architecture and are bound to happen.

wafflemaker 1y ago

There are really good psychologists out there that can do much more. It's a little luck and a little of good fit, but it can happen. >AI companies are partly responsible for promoting these tools as something more than statistical models,[...] This might be exactly the issue. Just today I've read people complaining that newest ChatGPT can't solve letter counting riddles. Companies just don't speak loud enough about LLM-based-AI shortcomings that result from their architecture and are bound to happen.