11 ms·

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it,

We would need the big picture, though... maybe it caused that death (which is awful) but it's also saving lives? If there are that many people confiding in it, I wouldn't be surprised if it actually prevents some suicides with encouraging comments, and that's not going to make the news. Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the positive impact on suicides is greater than negative or vice versa (not a social scientist so I have no idea what the methodology would look like, but if should be doable... or if it currently isn't, we should find the way).

computably 1y ago
> Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the positive impact on suicides is greater than negative or vice versa That is the literal opposite of how medical treatment is regulated. Treatments should be tested and studied before availability to the general public. It's irresponsible in the extreme to suggest this.
grey-area 1y ago
A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.
- butlike 1y ago
  Yet, if you ask the word generator to generate words in the form of advice, like any machine or code, it will do exactly what you tell it to do. The fact people are asking implies a lack of common sense by your definition. Sertraline can increase suicidal thoughts in teens. Should anti-depressants not be allowed near suicidal/depressed teens?
  - grey-area 1y ago
    Should anti-depressants not be allowed near suicidal/depressed teens? Well certainly not without careful monitoring and medical advice, no of course not!
- Al-Khwarizmi 1y ago
  Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?
  - bayindirh 1y ago
    > Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help. Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.
    - wongarsu 1y ago
      You could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"
      - cmsj 1y ago
        That sounds like a pretty risky and irresponsible sort of study to conduct. It would also likely be extremely complicate to actually get a reliable result, given that people with suicidal ideations are not monolithic. You'd need to do a significant amount of human counselling with each study participant to be able to classify and control all of the variations - at which point you would be verging on professional negligence for not then actually treating them in those counselling sessions.
- ben_w 1y ago
  I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No. By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much. Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense". Now this part: > Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix. I agree with you on… …with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research. More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.
  - probably_wrong 1y ago
    While I agree with most of your comment, I'd like to dispute the story about GPT-2. Yes, they did claim that they wouldn't release GPT-2 due to unforeseen risks, but... a. they did end up releasing it, b. they explicitly stated that they wouldn't release GPT-3[1] for marketing/financial reasons, and c. it being dangerous didn't stop them from offering the service for a profit. I think the quotes around "open" are well deserved. [1] Edit: it was GPT-4, not GPT-3.
    - ben_w 1y ago
      > they did end up releasing it, After studying it extensively with real-world feedback. From everything I've seen, the statement wasn't "will never release", it was vaguer than that. > they explicitly stated that they wouldn't release GPT-3 for marketing/financial reasons Not seen this, can you give a link? > it being dangerous didn't stop them from offering the service for a profit. Please do be cynical about how honest they were being — I mean, look at the whole of Big Tech right now — but the story they gave was self-consistent: [Paraphrased!] (a) "We do research" (they do), "This research costs a lot of money" (it does), and (b) "As software devs, we all know what 'agile' is and how that keeps product aligned with stakeholder interest." (they do) "And the world is our stakeholder, so we need to release updates for the world to give us feedback." (???) That last bit may be wishful thinking, I don't want to give the false impression that I think they can do no wrong (I've been let down by such optimism a few other times), but it is my impression of what they were claiming.
- courseofaction 1y ago
  Replace 'word generator with no intelligence or understanding based on the contents of the internet' with 'for-profit health care system'. In retrospect, from experience, I'd take the LLM.
  - omnimus 1y ago
    'not-for-profit healthcare system' has to surely be better better goal/solution than LLM
    - fragmede 1y ago
      Lemme get right on vibecoding that! Maybe three days, max, before I'll have an MVP. When can I expect your cheque funding my non-profit? It'll have a quadrillion dollar valuation by the end of the month, and you'll want to get in on the ground floor, so better act fast!
  - grey-area 1y ago
    Seems like an argument against allowing the profit motive into important life-changing decisions, like whether or not you'd commit suicide, or your medical treatment.
- wafflemaker 1y ago
  Let's look at the problem from perspective of regular people. YMMV, but in countries I know most about, Poland and Norway (albeit a little less so for Norway) it's not about ChatGPT vs Therapist. It's about ChatGPT vs nothing. I know people who earn above average income and still spend a significant (north of 20%) portion of their income on therapy/meds. And many don't, because mental health isn't that important to them. Or rather - they're not aware of how much helpful it can be to attend therapy. Or they just can't afford the luxury (that I claim it is) of private mental health treatment. ADHD diagnosis took 2.5y from start to getting meds, in Norway. Many kids grow up before their wait time in queue for pediatric psychologist is over. It's not ChatGPT vs shrink. It's ChatGPT vs nothing or your uncle who tells you depression and ADHD are made up and you kids these days have it all too easy.
  - butlike 1y ago
    As someone who lives in America, and is prescribed meds for ADHD; 2.5 years from asking for help to receiving medication _feels_ right to me in this case. The medications have a pretty negative side effect profile in my experience, and so all options should be weighed before prescribing ADHD-specific medication, imo
  - hitarpetar 1y ago
    you know ChatGPT can't prescribe Adderall right?
- lazide 1y ago
  Bleach should also not be allowed near suicidal teens. But how do you tell before it matters?
  - fragmede 1y ago
    Plastic bags shouldn't be allowed near suicidal teens. Scarves shouldn't be. Underwear is also a strangulation hazard for the truly desperate. Anything long sleeved even. Knives of any kind, including butter. Cars, obviously. Bleach is the least of your problems.
    - lazide 1y ago
      Yeah, if we know they’re suicidal, it’s legitimately grippy socks time I guess? But there is zero actually effective way to do that as an online platform. And plenty of ways that would cause more harm (statistically). My comment was more ‘how the hell would you know in a way anyone could actually do anything reasonable, anyway?’. People spam ‘Reddit cares’ as a harassment technique, claiming people are suicidal all the time. How much should the LLM try to guess? If they use all ‘depressed’ words? What does that even mean? What happens if someone reports a user is suicidal, and we don’t do anything? Are we now on the hook if they succeed - or fail and sue us? Do we just make a button that says ‘I’m intending to self harm’ that locks them out of the system?
      - butlike 1y ago
        Why are we imprisoning suicidal people? That will surely add incentive to have someone raise their hand and ask for help: taking their freedoms away...
        lazide 1y ago
        Why do we put people in a controlled environment where their available actions are heavily restricted and their ability to use anything they could hurt themselves is taken away? When they are a known risk of hurting themselves or others? What else do you propose?
- IshKebab 1y ago
  > with no intelligence Damn I thought we'd got over that stochastic parrot nonsense finally...
- terminalshort 1y ago
  > A word generator with no intelligence or understanding I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"
  - grey-area 1y ago
    Sure ask it to write an interesting novel or a symphony, and present it to humans without editing. The majority of literate humans will easily tell the difference between that and human output. And it’s not allowed to be too derivative. When AI gets there (and I’m confident it will, though not confident LLMs will), I think that’s convincing evidence of intelligence and creativity.
    - terminalshort 1y ago
      I accept that test other than the "too derivative" part which is an avenue for subjective bias. AI has passed that test for art already: https://www.astralcodexten.com/p/ai-art-turing-test https://www.astralcodexten.com/p/ai-art-turing-test As for a novel that is currently beyond the LLMs capabilities due to context windows, but I wouldn't be surprised if it could do short stories that pass this Turing test right now.
      - grey-area 1y ago
        For art I'd want to see a consistent body of work with meaning from an AI, not heavily derivative copies of other styles and subject matter. But I'd agree AI art is further along, perhaps just because it is easier for us to fill in the gaps and attribute meaning where there is none.
bayindirh 1y ago
Maybe it's causing even more deaths than we know, and these doesn't make the news either? If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be surprised if these actually transports some people with comfort, and that's not going to make the news.
- Al-Khwarizmi 1y ago
  > Maybe it's causing even more deaths than we know, and these doesn't make the news either? Of course, and that's part of why I say that we need to measure the impact. It could be net positive or negative, we won't know if we don't find out. > If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be surprised if these actually transports some people with comfort, and that's not going to make the news. I'm not advocating for not improving security, I'm arguing against a comment that said that "ChatGPT should be nowhere near anyone dealing with psychological issues", because it can cause death. Following your analogy, cars objectively cause deaths (and not only of people with psychological issues, but of people in general) and we don't say that "they should be nowhere near a person". We improve their safety even though zero deaths is probably impossible, which we accept because they are useful. This is a big-picture approach.
jrflowers 1y ago
[flagged]
- david-gpu 1y ago
  They didn't say it was impossible, or that we should do nothing. Learn how to have a constructive dialogue, please.
  - jrflowers 1y ago
    You make a good point. While they absolutely and unequivocally said that it is currently impossible to tell whether the suicides are bad or not, they also sort of wondered aloud if in the future we might be able to develop a methodology to determine whether the suicides are bad or not. This is an important distinction becau
alex-moon 1y ago
I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good enough. In such cases, where a new approach offers to replace an existing approach, the burden of proof is on the challenger, not the incumbent. This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed tests. You understand then, hopefully, why your comments here are dangerous...? I have no doubt you have no malicious intent here - you're right that these decisions need to be based on data - but you're not taking into account that the (potentially extremely harmful) challenger already has a foothold in the field. I know that you will want to hear this from experts in the "relevant field" rather than myself, so here is a write-up from Stanford on the subject: https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in-mental-health-care https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in...
- butlike 1y ago
  What are you talking about? I can grow food myself, and I can build a car from scratch and take it on the highway. Are there repercussions? Sure, but nothing inherently stops me from doing it. The problem here is there's no measurable "win condition" for when a person gets good information that helps them. They remain alive, which was their previous state. This is hard to measure. Now, should people be able to google their symptoms and try and help themselves? This dovetails into a deeper philosophical discussion, but I'm not entirely convinced "seek professional help" is ALWAYS the answer. ALWAYS and NEVER are _very_ long timeframes, and we should be careful when using them.
- RobRivera 1y ago
  I would take it a step back and posit simply, "why does a human require certification to practice therapy, and a computer program does not?" There should be some liability for malpractrice even if generated by an llm
- 1y ago
toofy 1y ago
people are overcomplicating this, the big picture is simple af: if a therapist was ever found to have said to this to a suicidal person, they would be immediately stripped of their license and maybe jailed.
- cal85 1y ago
  True. But it feels like a fairer comparison would be with a huge healthcare company that failed to vet one of its therapists properly, so a crazy pro-suicide therapist slipped through the net. Would we petition to shut down the whole company for this rare event? I suppose it would depend on whether the company could demonstrate what it is doing to ensure it doesn’t happen again.
  - wat10000 1y ago
    Maybe you shouldn't shut down OpenAI over this. But each instance of a particular ChatGPT model is the same as all the others. This is like a company that has a magical superhuman therapist that can see a million patients a day. If they're found to be encouraging suicide, then they need to be stopped from providing therapy. The fact that this is the company's only source of revenue might mean that the company has to shut down over this, but that's just a consequence of putting all your eggs in one basket.
- terminalshort 1y ago
  But you would have to be a therapist. If a suicidal person went up to a stranger and started a conversation, there would be no consequences. That's more analogous to ChatGPT.
ml-anon 1y ago
This entire comment section is full of wide eyed nonsense like this. It’s honestly frightening that we are even humoring this point of view.
- nihzm 1y ago
  Since as you say this utilitarian view is rather common, perhaps it would good to show _why_ this is problematic by presenting a counterargument. The basic premise under GP's statements is that although not perfect, we should use the technology in such a way that it maximizes the well being of the largest number of people, even if comes at the expense of a few. But therein lies a problem: we cannot really measure well being (or utility). This becomes obvious if you look at individuals instead of the aggregate: imagine LLM therapy becomes widespread and a famous high profile person and your (not famous) daughter end up in "the few" for which LLM therapy goes terribly wrong and commit suicide. The loss of the famous person will cause thousands (perhaps millions) people to be a bit sad, and the loss of your daughter will cause you unimaginable pain. Which one is greater? Can they even be be compared? And how many people with a successful LLM therapy are enough to compensate for either one? Unmeasurable well-being then makes these moral calculations at best inexact and at worst completely meaningless. And if they are truly meaningless, how can they inform your LLM therapy policy decisions? Suppose for the sake of the argument we accept the above, and there is a way to measure well being. Then would it be just? Justice is a fuzzy concept, but imagine we reverse the example above: many people lose their lives because of bad LLM therapy, but one very famous person in the entertainment industry is saved by LLM therapy. Let's suppose then that this famous persons' well being plus the millions of spectators' improved well-being (through their entertainment) is worth enough to compensate the people who died. This means saving a famous funny person justifies the death of many. This does not feel just, does it? There is a vast amount of literature on this topic (criticisms of utilitarianism).
  - ml-anon 1y ago
    This is either incredible satire or you’re a lunatic.
    - 1y ago
computerthings 1y ago
[dead]
hitarpetar 1y ago
> someone in the relevant field should study whether the positive impact on suicides is greater than negative or vice versa we already have an approval process for medical interventions. are you suggesting the government shut ChatGPT down until the FDA can investigate it's use for therapy? because if so I can get behind that
wat10000 1y ago
Our standard approach for new medical treatments is to require proof of safety and efficacy before it's made available to the general public. This is because it's very, very easy for promising-looking treatments to end up being harmful. "Before declaring that it shouldn't be near anyone with psychological issues" is backwards. Before providing it to people with psychological issues, someone should study whether the positive impact is greater than the negative. Trouble is, this is such a generalized tool that it's very hard to do that.
wfleming 1y ago
If a therapist helped 99/100 patients but tacitly encouraged the 100th to commit suicide* they would still lose their license. * ignoring the case of ethical assisted suicide for reasons of terminal illness and such, which doesn’t seem relevant to the case discussed here.
FuckButtons 1y ago
Let’s maybe not give the benefit of the doubt to the startup which has shown itself to have the moral scruples of vault-tec just because what they’re doing might work out fine for some of the people they’re experimenting on.

computably 1y ago

> Before declaring that it shouldn't be near anyone with psychological issues, someone in the relevant field should study whether the positive impact on suicides is greater than negative or vice versa That is the literal opposite of how medical treatment is regulated. Treatments should be tested and studied before availability to the general public. It's irresponsible in the extreme to suggest this.

grey-area 1y ago

A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind. This is basic common sense. Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.

butlike 1y ago
Yet, if you ask the word generator to generate words in the form of advice, like any machine or code, it will do exactly what you tell it to do. The fact people are asking implies a lack of common sense by your definition. Sertraline can increase suicidal thoughts in teens. Should anti-depressants not be allowed near suicidal/depressed teens?
- grey-area 1y ago
  Should anti-depressants not be allowed near suicidal/depressed teens? Well certainly not without careful monitoring and medical advice, no of course not!
Al-Khwarizmi 1y ago
Supposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?
- bayindirh 1y ago
  > Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help. Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.
  - wongarsu 1y ago
    You could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"
    - cmsj 1y ago
      That sounds like a pretty risky and irresponsible sort of study to conduct. It would also likely be extremely complicate to actually get a reliable result, given that people with suicidal ideations are not monolithic. You'd need to do a significant amount of human counselling with each study participant to be able to classify and control all of the variations - at which point you would be verging on professional negligence for not then actually treating them in those counselling sessions.
ben_w 1y ago
I'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No. By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much. Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense". Now this part: > Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix. I agree with you on… …with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research. More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.
- probably_wrong 1y ago
  While I agree with most of your comment, I'd like to dispute the story about GPT-2. Yes, they did claim that they wouldn't release GPT-2 due to unforeseen risks, but... a. they did end up releasing it, b. they explicitly stated that they wouldn't release GPT-3[1] for marketing/financial reasons, and c. it being dangerous didn't stop them from offering the service for a profit. I think the quotes around "open" are well deserved. [1] Edit: it was GPT-4, not GPT-3.
  - ben_w 1y ago
    > they did end up releasing it, After studying it extensively with real-world feedback. From everything I've seen, the statement wasn't "will never release", it was vaguer than that. > they explicitly stated that they wouldn't release GPT-3 for marketing/financial reasons Not seen this, can you give a link? > it being dangerous didn't stop them from offering the service for a profit. Please do be cynical about how honest they were being — I mean, look at the whole of Big Tech right now — but the story they gave was self-consistent: [Paraphrased!] (a) "We do research" (they do), "This research costs a lot of money" (it does), and (b) "As software devs, we all know what 'agile' is and how that keeps product aligned with stakeholder interest." (they do) "And the world is our stakeholder, so we need to release updates for the world to give us feedback." (???) That last bit may be wishful thinking, I don't want to give the false impression that I think they can do no wrong (I've been let down by such optimism a few other times), but it is my impression of what they were claiming.
courseofaction 1y ago
Replace 'word generator with no intelligence or understanding based on the contents of the internet' with 'for-profit health care system'. In retrospect, from experience, I'd take the LLM.
- omnimus 1y ago
  'not-for-profit healthcare system' has to surely be better better goal/solution than LLM
  - fragmede 1y ago
    Lemme get right on vibecoding that! Maybe three days, max, before I'll have an MVP. When can I expect your cheque funding my non-profit? It'll have a quadrillion dollar valuation by the end of the month, and you'll want to get in on the ground floor, so better act fast!
- grey-area 1y ago
  Seems like an argument against allowing the profit motive into important life-changing decisions, like whether or not you'd commit suicide, or your medical treatment.
wafflemaker 1y ago
Let's look at the problem from perspective of regular people. YMMV, but in countries I know most about, Poland and Norway (albeit a little less so for Norway) it's not about ChatGPT vs Therapist. It's about ChatGPT vs nothing. I know people who earn above average income and still spend a significant (north of 20%) portion of their income on therapy/meds. And many don't, because mental health isn't that important to them. Or rather - they're not aware of how much helpful it can be to attend therapy. Or they just can't afford the luxury (that I claim it is) of private mental health treatment. ADHD diagnosis took 2.5y from start to getting meds, in Norway. Many kids grow up before their wait time in queue for pediatric psychologist is over. It's not ChatGPT vs shrink. It's ChatGPT vs nothing or your uncle who tells you depression and ADHD are made up and you kids these days have it all too easy.
- butlike 1y ago
  As someone who lives in America, and is prescribed meds for ADHD; 2.5 years from asking for help to receiving medication _feels_ right to me in this case. The medications have a pretty negative side effect profile in my experience, and so all options should be weighed before prescribing ADHD-specific medication, imo
- hitarpetar 1y ago
  you know ChatGPT can't prescribe Adderall right?
lazide 1y ago
Bleach should also not be allowed near suicidal teens. But how do you tell before it matters?
- fragmede 1y ago
  Plastic bags shouldn't be allowed near suicidal teens. Scarves shouldn't be. Underwear is also a strangulation hazard for the truly desperate. Anything long sleeved even. Knives of any kind, including butter. Cars, obviously. Bleach is the least of your problems.
  - lazide 1y ago
    Yeah, if we know they’re suicidal, it’s legitimately grippy socks time I guess? But there is zero actually effective way to do that as an online platform. And plenty of ways that would cause more harm (statistically). My comment was more ‘how the hell would you know in a way anyone could actually do anything reasonable, anyway?’. People spam ‘Reddit cares’ as a harassment technique, claiming people are suicidal all the time. How much should the LLM try to guess? If they use all ‘depressed’ words? What does that even mean? What happens if someone reports a user is suicidal, and we don’t do anything? Are we now on the hook if they succeed - or fail and sue us? Do we just make a button that says ‘I’m intending to self harm’ that locks them out of the system?
    - butlike 1y ago
      Why are we imprisoning suicidal people? That will surely add incentive to have someone raise their hand and ask for help: taking their freedoms away...
      - lazide 1y ago
        Why do we put people in a controlled environment where their available actions are heavily restricted and their ability to use anything they could hurt themselves is taken away? When they are a known risk of hurting themselves or others? What else do you propose?
IshKebab 1y ago
> with no intelligence Damn I thought we'd got over that stochastic parrot nonsense finally...
terminalshort 1y ago
> A word generator with no intelligence or understanding I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"
- grey-area 1y ago
  Sure ask it to write an interesting novel or a symphony, and present it to humans without editing. The majority of literate humans will easily tell the difference between that and human output. And it’s not allowed to be too derivative. When AI gets there (and I’m confident it will, though not confident LLMs will), I think that’s convincing evidence of intelligence and creativity.
  - terminalshort 1y ago
    I accept that test other than the "too derivative" part which is an avenue for subjective bias. AI has passed that test for art already: https://www.astralcodexten.com/p/ai-art-turing-test https://www.astralcodexten.com/p/ai-art-turing-test As for a novel that is currently beyond the LLMs capabilities due to context windows, but I wouldn't be surprised if it could do short stories that pass this Turing test right now.
    - grey-area 1y ago
      For art I'd want to see a consistent body of work with meaning from an AI, not heavily derivative copies of other styles and subject matter. But I'd agree AI art is further along, perhaps just because it is easier for us to fill in the gaps and attribute meaning where there is none.

bayindirh 1y ago

Maybe it's causing even more deaths than we know, and these doesn't make the news either? If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be surprised if these actually transports some people with comfort, and that's not going to make the news.

Al-Khwarizmi 1y ago
> Maybe it's causing even more deaths than we know, and these doesn't make the news either? Of course, and that's part of why I say that we need to measure the impact. It could be net positive or negative, we won't know if we don't find out. > If we think this way, then we don't need to improve safety of anything (cars, trains, planes, ships, etc.) because we would need the big picture, though... maybe these vehicles cause death (which is awful), but it's also transporting people to their destinations alive. If there are that many people using these, I wouldn't be surprised if these actually transports some people with comfort, and that's not going to make the news. I'm not advocating for not improving security, I'm arguing against a comment that said that "ChatGPT should be nowhere near anyone dealing with psychological issues", because it can cause death. Following your analogy, cars objectively cause deaths (and not only of people with psychological issues, but of people in general) and we don't say that "they should be nowhere near a person". We improve their safety even though zero deaths is probably impossible, which we accept because they are useful. This is a big-picture approach.

jrflowers 1y ago

[flagged]

david-gpu 1y ago
They didn't say it was impossible, or that we should do nothing. Learn how to have a constructive dialogue, please.
- jrflowers 1y ago
  You make a good point. While they absolutely and unequivocally said that it is currently impossible to tell whether the suicides are bad or not, they also sort of wondered aloud if in the future we might be able to develop a methodology to determine whether the suicides are bad or not. This is an important distinction becau

alex-moon 1y ago

I suspect you've never done therapy yourself. Most people who have worked with a professional therapist understand intuitively why the only helpful feedback from an LLM to someone who needs professional help is: get professional help. AIs are really good at doing something to about 80%. When the stakes are life or death, as they are with someone who is suicidal, that is a good example of a time when 80% isn't good enough. In such cases, where a new approach offers to replace an existing approach, the burden of proof is on the challenger, not the incumbent. This is why we have safety regulations, why we don't let people drive cars that haven't passed tests, build with materials that haven't passed tests, eat food that hasn't passed tests. You understand then, hopefully, why your comments here are dangerous...? I have no doubt you have no malicious intent here - you're right that these decisions need to be based on data - but you're not taking into account that the (potentially extremely harmful) challenger already has a foothold in the field. I know that you will want to hear this from experts in the "relevant field" rather than myself, so here is a write-up from Stanford on the subject: https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in-mental-health-care https://hai.stanford.edu/news/exploring-the-dangers-of-ai-in...

butlike 1y ago
What are you talking about? I can grow food myself, and I can build a car from scratch and take it on the highway. Are there repercussions? Sure, but nothing inherently stops me from doing it. The problem here is there's no measurable "win condition" for when a person gets good information that helps them. They remain alive, which was their previous state. This is hard to measure. Now, should people be able to google their symptoms and try and help themselves? This dovetails into a deeper philosophical discussion, but I'm not entirely convinced "seek professional help" is ALWAYS the answer. ALWAYS and NEVER are _very_ long timeframes, and we should be careful when using them.
RobRivera 1y ago
I would take it a step back and posit simply, "why does a human require certification to practice therapy, and a computer program does not?" There should be some liability for malpractrice even if generated by an llm
1y ago

toofy 1y ago

people are overcomplicating this, the big picture is simple af: if a therapist was ever found to have said to this to a suicidal person, they would be immediately stripped of their license and maybe jailed.

cal85 1y ago
True. But it feels like a fairer comparison would be with a huge healthcare company that failed to vet one of its therapists properly, so a crazy pro-suicide therapist slipped through the net. Would we petition to shut down the whole company for this rare event? I suppose it would depend on whether the company could demonstrate what it is doing to ensure it doesn’t happen again.
- wat10000 1y ago
  Maybe you shouldn't shut down OpenAI over this. But each instance of a particular ChatGPT model is the same as all the others. This is like a company that has a magical superhuman therapist that can see a million patients a day. If they're found to be encouraging suicide, then they need to be stopped from providing therapy. The fact that this is the company's only source of revenue might mean that the company has to shut down over this, but that's just a consequence of putting all your eggs in one basket.
terminalshort 1y ago
But you would have to be a therapist. If a suicidal person went up to a stranger and started a conversation, there would be no consequences. That's more analogous to ChatGPT.

ml-anon 1y ago

This entire comment section is full of wide eyed nonsense like this. It’s honestly frightening that we are even humoring this point of view.

nihzm 1y ago
Since as you say this utilitarian view is rather common, perhaps it would good to show _why_ this is problematic by presenting a counterargument. The basic premise under GP's statements is that although not perfect, we should use the technology in such a way that it maximizes the well being of the largest number of people, even if comes at the expense of a few. But therein lies a problem: we cannot really measure well being (or utility). This becomes obvious if you look at individuals instead of the aggregate: imagine LLM therapy becomes widespread and a famous high profile person and your (not famous) daughter end up in "the few" for which LLM therapy goes terribly wrong and commit suicide. The loss of the famous person will cause thousands (perhaps millions) people to be a bit sad, and the loss of your daughter will cause you unimaginable pain. Which one is greater? Can they even be be compared? And how many people with a successful LLM therapy are enough to compensate for either one? Unmeasurable well-being then makes these moral calculations at best inexact and at worst completely meaningless. And if they are truly meaningless, how can they inform your LLM therapy policy decisions? Suppose for the sake of the argument we accept the above, and there is a way to measure well being. Then would it be just? Justice is a fuzzy concept, but imagine we reverse the example above: many people lose their lives because of bad LLM therapy, but one very famous person in the entertainment industry is saved by LLM therapy. Let's suppose then that this famous persons' well being plus the millions of spectators' improved well-being (through their entertainment) is worth enough to compensate the people who died. This means saving a famous funny person justifies the death of many. This does not feel just, does it? There is a vast amount of literature on this topic (criticisms of utilitarianism).
- ml-anon 1y ago
  This is either incredible satire or you’re a lunatic.
  - 1y ago

computerthings 1y ago

[dead]

hitarpetar 1y ago

> someone in the relevant field should study whether the positive impact on suicides is greater than negative or vice versa we already have an approval process for medical interventions. are you suggesting the government shut ChatGPT down until the FDA can investigate it's use for therapy? because if so I can get behind that

wat10000 1y ago

Our standard approach for new medical treatments is to require proof of safety and efficacy before it's made available to the general public. This is because it's very, very easy for promising-looking treatments to end up being harmful. "Before declaring that it shouldn't be near anyone with psychological issues" is backwards. Before providing it to people with psychological issues, someone should study whether the positive impact is greater than the negative. Trouble is, this is such a generalized tool that it's very hard to do that.

wfleming 1y ago

If a therapist helped 99/100 patients but tacitly encouraged the 100th to commit suicide* they would still lose their license. * ignoring the case of ethical assisted suicide for reasons of terminal illness and such, which doesn’t seem relevant to the case discussed here.

FuckButtons 1y ago

Let’s maybe not give the benefit of the doubt to the startup which has shown itself to have the moral scruples of vault-tec just because what they’re doing might work out fine for some of the people they’re experimenting on.