7 ms·
A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to
by grey-area 1y ago
A word generator with no intelligence or understanding based on the contents of the internet should not be allowed near suicidal teens, nor should it attempt to offer advice of any kind.
This is basic common sense.
Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix.
- butlike 1y agoYet, if you ask the word generator to generate words in the form of advice, like any machine or code, it will do exactly what you tell it to do. The fact people are asking implies a lack of common sense by your definition. Sertraline can increase suicidal thoughts in teens. Should anti-depressants not be allowed near suicidal/depressed teens?
- grey-area 1y agoShould anti-depressants not be allowed near suicidal/depressed teens? Well certainly not without careful monitoring and medical advice, no of course not!
- Al-Khwarizmi 1y agoSupposing that the advice it provides does more good than harm, why? What's the objective reason? If it can save lives, who cares if the advice is based on intelligence and understanding or on regurgitating internet content?
- bayindirh 1y ago> Supposing that the advice it provides does more good than harm, why? Because a human, esp. a confused and depressive human being is a complex thing. Much more complex than a stable, healthy human. Words encouraging a healthy person can break a depressed person further. Statistically positive words can deepen wounds, and push people more to the edge. Dark corners of human nature is twisted, hard to navigate and full of distortions. Simple words don't and can't help. Humans are not machines, brains are not mathematical formulae. We're not deterministic. We need to leave this fantasy behind.
- wongarsu 1y agoYou could make the same arguments to say that humans should never talk to suicidal people. And that really sounds counterproductive Also it's side-stepping the question, isn't it? "Supposing that the advice it provides does more good than harm" already supposes that LLMs navigate this somehow. Maybe because they are so great, maybe by accident, maybe because just having someone nonjudgmental to talk to has a net-positive effect. The question posed is really "if LLMs lead some people to suicide but saved a greater number of people from suicide, and we verify this hypothesis with studies, would there still be an argument against LLMs talking to suicidal people"
- cmsj 1y agoThat sounds like a pretty risky and irresponsible sort of study to conduct. It would also likely be extremely complicate to actually get a reliable result, given that people with suicidal ideations are not monolithic. You'd need to do a significant amount of human counselling with each study participant to be able to classify and control all of the variations - at which point you would be verging on professional negligence for not then actually treating them in those counselling sessions.
- imiric 1y agoI agree with your concerns, but I think you're overestimating the value of a human intervening in these scenarios. A human can also say the wrong things to push someone in a certain direction. A psychologist, or anyone else for that matter, can't stop someone from committing suicide if they've already made up their mind about it. They're educated on human psychology, but they're not miracle workers. The best they can do is raise a flag if they suspect self-harm, but then again, so could a machine. As you say, humans are complex. But I agree with GP: whether the words are generated by a machine or coming from a human, there is no way to blame the source for any specific outcome. There are probably many other cases where the machine has helped someone with personal issues, yet we'll never hear about it. I'm not saying we should rely on these tools as if we would on a human, but the technology can be used for good or bad. If anything, I would place blame on the person who decides to blindly follow anything the machine generates in the first place. AI companies are partly responsible for promoting these tools as something more than statistical models, but ultimately the decision to treat them as reliable sources of information is on the user. I would say that as long as the person has an understanding of what these tools are, interacting with them can be healthy and helpful.
- grey-area 1y agoFirst, do no harm.
- wongarsu 1y agoThat relates more to purposefully harming some people to safe other people. Doing something that has the potential to harm a person but statistically has a greater likelihood of helping them is something doctors do all the time. They will even use methods that are guaranteed to do harm to the patient, as long as they have a sufficient chance to also bring a major benefit to the same patient
- butlike 1y agoAn example being: surgery. You cut into the patient to remove the tumor.
- RobotToaster 1y agoThe Hippocratic oath originated from Hippocratic medicine forbidding surgery, which is why surgeons are still not referred to as "doctor" today.
- hirvi74 1y agoDo no harm or no intentional harm?
- pzlarsson 1y agoWhen evaluating good vs harm for drugs or other treatments the risk for lethal side effects must be very small for the treatment to be approved. In this case it is also difficult to get reliable data on how much good and harm is done.
- latexr 1y ago> Supposing that the advice it provides does more good than harm That unsubstantiated supposition is doing a lot of heavy lifting and that’s a dangerous and unproductive way to frame the argument. I’ll make a purposefully exaggerated example. Say a school wants to add cyanide to every meal and defends the decision with “supposing it helps students concentrate and be quieter in the classroom, why not?”. See the problem? The supposition is wrong and the suggestion is dangerous, but by framing it as “supposing” with a made up positive outcome, we make it sound non-threatening and reasonable. Or for a more realistic example, “suppose drinking bleach could cure COVID-19”. First understand if the idea has the potential to do the thing, only then (with considerably more context) consider if it’s worth implementing.
- Al-Khwarizmi 1y agoIn my previous post up the thread I said that we should measure whether in fact it does more good than harm or not. That's the context of my comment, I'm not saying we should just take it for granted without looking.
- SiempreViernes 1y ago> we should measure whether in fact it does more good than harm or not The demonstrable harms include assisting suicide, there's is no way to ethically continue the measurement because continuing the measurements in their current form will with certainty result in further deaths.
- simonklitj 1y agoThank you! On top of that, it’s hard to measure “potential suicides averted,” and comparing that with “actual suicides caused/assisted with” would be incommnsurable. And working to set a threshold for what we would consider acceptable? No thanks
- fragmede 1y agoReal life trolly problem! If you pull the lever, some people on this track will die (by sucide). If you don't pull the lever, some people will still die from suicide. By not pulling the lever, and simply banning discussion of suicide entirely, your company gets to avoid a huge PR disaster, and you get more money because line go up. If you pull the lever and let people talk about suicide on your platform, you may avoid prevent some suicides, but you can never discuss that with the press, your company gets bad PR, and everyone will believe you're a murderer. Plus, line go down and you make less money while other companies make money off of selling AI therapy apps. What do you chose to do?
- exasperaited 1y agoThis is not so much "more good than harm" like a counsellor that isn't very good. This is more "sometimes it will seemingly actively encourage them to kill themselves and it's basically a roll of the dice what words come out at any one time". If a counsellor does that they can be prosecuted and jailed for it, no matter how many other patients they help.
- ben_w 1y agoI'll gladly diss LLMs in a whole bunch of ways, but "common sense"? No. By the "common sense" definitions, LLMs have "intelligence" and "understanding", that's why they get used so much. Not that this makes the "common sense" definitions useful for all questions. One of the worse things about LLMs, in my opinion, is that they're mostly a pile of "common sense". Now this part: > Add in the commercial incentives of 'Open'AI to promote usage for anything and everything and you have a toxic mix. I agree with you on… …with the exception of one single word: It's quite cliquish to put scare quotes around the "Open" part on a discussion about them publishing research. More so given that people started doing this in response to them saying "let's be cautious, we don't know what the risks are yet and we can't un-publish model weights" with GPT-2, and oh look, here it is being dangerous.
- probably_wrong 1y agoWhile I agree with most of your comment, I'd like to dispute the story about GPT-2. Yes, they did claim that they wouldn't release GPT-2 due to unforeseen risks, but... a. they did end up releasing it, b. they explicitly stated that they wouldn't release GPT-3[1] for marketing/financial reasons, and c. it being dangerous didn't stop them from offering the service for a profit. I think the quotes around "open" are well deserved. [1] Edit: it was GPT-4, not GPT-3.
- ben_w 1y ago> they did end up releasing it, After studying it extensively with real-world feedback. From everything I've seen, the statement wasn't "will never release", it was vaguer than that. > they explicitly stated that they wouldn't release GPT-3 for marketing/financial reasons Not seen this, can you give a link? > it being dangerous didn't stop them from offering the service for a profit. Please do be cynical about how honest they were being — I mean, look at the whole of Big Tech right now — but the story they gave was self-consistent: [Paraphrased!] (a) "We do research" (they do), "This research costs a lot of money" (it does), and (b) "As software devs, we all know what 'agile' is and how that keeps product aligned with stakeholder interest." (they do) "And the world is our stakeholder, so we need to release updates for the world to give us feedback." (???) That last bit may be wishful thinking, I don't want to give the false impression that I think they can do no wrong (I've been let down by such optimism a few other times), but it is my impression of what they were claiming.
- courseofaction 1y agoReplace 'word generator with no intelligence or understanding based on the contents of the internet' with 'for-profit health care system'. In retrospect, from experience, I'd take the LLM.
- omnimus 1y ago'not-for-profit healthcare system' has to surely be better better goal/solution than LLM
- fragmede 1y agoLemme get right on vibecoding that! Maybe three days, max, before I'll have an MVP. When can I expect your cheque funding my non-profit? It'll have a quadrillion dollar valuation by the end of the month, and you'll want to get in on the ground floor, so better act fast!
- grey-area 1y agoSeems like an argument against allowing the profit motive into important life-changing decisions, like whether or not you'd commit suicide, or your medical treatment.
- wafflemaker 1y agoLet's look at the problem from perspective of regular people. YMMV, but in countries I know most about, Poland and Norway (albeit a little less so for Norway) it's not about ChatGPT vs Therapist. It's about ChatGPT vs nothing. I know people who earn above average income and still spend a significant (north of 20%) portion of their income on therapy/meds. And many don't, because mental health isn't that important to them. Or rather - they're not aware of how much helpful it can be to attend therapy. Or they just can't afford the luxury (that I claim it is) of private mental health treatment. ADHD diagnosis took 2.5y from start to getting meds, in Norway. Many kids grow up before their wait time in queue for pediatric psychologist is over. It's not ChatGPT vs shrink. It's ChatGPT vs nothing or your uncle who tells you depression and ADHD are made up and you kids these days have it all too easy.
- butlike 1y agoAs someone who lives in America, and is prescribed meds for ADHD; 2.5 years from asking for help to receiving medication _feels_ right to me in this case. The medications have a pretty negative side effect profile in my experience, and so all options should be weighed before prescribing ADHD-specific medication, imo
- hitarpetar 1y agoyou know ChatGPT can't prescribe Adderall right?
- lazide 1y agoBleach should also not be allowed near suicidal teens. But how do you tell before it matters?
- fragmede 1y agoPlastic bags shouldn't be allowed near suicidal teens. Scarves shouldn't be. Underwear is also a strangulation hazard for the truly desperate. Anything long sleeved even. Knives of any kind, including butter. Cars, obviously. Bleach is the least of your problems.
- lazide 1y agoYeah, if we know they’re suicidal, it’s legitimately grippy socks time I guess? But there is zero actually effective way to do that as an online platform. And plenty of ways that would cause more harm (statistically). My comment was more ‘how the hell would you know in a way anyone could actually do anything reasonable, anyway?’. People spam ‘Reddit cares’ as a harassment technique, claiming people are suicidal all the time. How much should the LLM try to guess? If they use all ‘depressed’ words? What does that even mean? What happens if someone reports a user is suicidal, and we don’t do anything? Are we now on the hook if they succeed - or fail and sue us? Do we just make a button that says ‘I’m intending to self harm’ that locks them out of the system?
- butlike 1y agoWhy are we imprisoning suicidal people? That will surely add incentive to have someone raise their hand and ask for help: taking their freedoms away...
- lazide 1y agoWhy do we put people in a controlled environment where their available actions are heavily restricted and their ability to use anything they could hurt themselves is taken away? When they are a known risk of hurting themselves or others? What else do you propose?
- IshKebab 1y ago> with no intelligence Damn I thought we'd got over that stochastic parrot nonsense finally...
- terminalshort 1y ago> A word generator with no intelligence or understanding I will take this seriously when you propose a test that can distinguish between that and something with actual "intelligence or understanding"
- grey-area 1y agoSure ask it to write an interesting novel or a symphony, and present it to humans without editing. The majority of literate humans will easily tell the difference between that and human output. And it’s not allowed to be too derivative. When AI gets there (and I’m confident it will, though not confident LLMs will), I think that’s convincing evidence of intelligence and creativity.
- terminalshort 1y agoI accept that test other than the "too derivative" part which is an avenue for subjective bias. AI has passed that test for art already: https://www.astralcodexten.com/p/ai-art-turing-test https://www.astralcodexten.com/p/ai-art-turing-test As for a novel that is currently beyond the LLMs capabilities due to context windows, but I wouldn't be surprised if it could do short stories that pass this Turing test right now.
- grey-area 1y agoFor art I'd want to see a consistent body of work with meaning from an AI, not heavily derivative copies of other styles and subject matter. But I'd agree AI art is further along, perhaps just because it is easier for us to fill in the gaps and attribute meaning where there is none.