4 ms·

You "skimmed through it" by your own admission, but somehow feel comfortable enough to armchair diagnose everyone involved as having "severe mental health issue

You "skimmed through it" by your own admission, but somehow feel comfortable enough to armchair diagnose everyone involved as having "severe mental health issues". Seriously, are you not ashamed? Your advocacy of "just focus on mental health" rings particularly hollow when you advocate that we should just ignore a technology that's actively destroying people's mental health, driving them to suicide. The irony is off the charts here. So, please, let's be real - you don't care about mental health. You're scared someone might put a warning sticker on your new toys. Even if every single person had a pre-existing mental condition (they didn't, and you'd know this if you had read the article) your logic is sociopathic and genuinely disturbing. "Vulnerable people exist. Therefore, companies bear no responsibility when their products exploit those vulnerabilities and harm them. Yay, we don't need to think about it anymore." Cool take. I guess it's fine if technology kills people as long as they might have an undiagnosed mental health issue. This argument is patently ridiculous in any context other than a Hacker News discussion full of technophiles and accelerationists. Moving fast and breaking things is fine when it comes to your database - not so much when you're dealing with people's lives. We don't dismiss food safety regulations because some people have allergies. We don't ignore faulty airbags because some people are terrible or reckless drivers. But when a chatbot instructs someone to stop taking prescribed medications, increase ketamine use, and tells them they can fly if they jump off a building - that's just a "mental health issue" and not a flaw in the product? No guardrails, warning labels or safety mechanisms necessary? The fact that this predominantly affects vulnerable populations makes regulation MORE necessary - not less. Nothing says, "let's focus more on mental health" than claiming it's "silly" to suggest a technology that actively coaches vulnerable people through suicide attempts might be worthy of regulation.

infecto 1y ago
Do not confuse yourself, skimming does not mean I did not ingest the article but I did speed read through it as it was not that well written. Why would I feel ashamed? I am not saying the crisis itself is silly but the article leads the reader through a biased take. But let’s also be real, there is an high probability every single one of these folks was having some sort of issue prior to the engagement with ChatGPT. Would love to see more research in the area and actual quantification of the issue but until that happens your argument might as well extend to all parts of life. Have a mandatory psych evaluation every year. Limitations to your access to the internet or other things, maybe even forced institutionalization if you don’t pass.
- sReinwald 1y ago
  In your case, "speed reading" and "skimming" are obviously just two flavors of "I didn't actually read this properly but still feel qualified to diagnose everyone involved." You'd love to see more research? You must've been reading so fast, you breezed right past the Stanford Study, the MIT Media Lab Research and the analysis by McCoy showing GPT-4o affirmed psychotic delusions 68% of the time. Or do you mean it in the sense of "we have to study this for at least 20 years and let thousands suffer and die before we can determine whether chatbots affirming people's delusions or coaching them on how to kill themselves is a bad thing"? Your concerns about the slippery slope from "maybe chatbots shouldn't coach people through suicide" to "forced psychiatric evaluations" or "institutionalization" is genuinely unhinged. And, tellingly, it betrays your view of mental healthcare as fundamentally punitive and authoritarian rather than supportive or therapeutic. Big advocate for mental health, you are. You realize we already regulate countless things to protect vulnerable people from mental harm, right? We put warning labels on cigarettes. We have age ratings on media. We don't let casinos exploit problem gamblers. We don't let bartenders serve alcohol to visibly intoxicated people. We don't let therapists tell suicidal patients to jump off buildings. None of these regulations are authoritarian overreach - they're basic safety measures that exist because in a healthy society, vulnerable people deserve protection, not exploitation. But apparently when it's time to consider holding tech to similar standards, we're supposed to just shrug and say, "well, they were probably crazy anyway." Your position essentially boils down to: "Vulnerable people exist, therefore companies should be free to exploit them." That's not a mental health advocacy position - it's tech industry bootlicking dressed up as concern trolling. That's why you should feel ashamed.
  - infecto 1y ago
    You’re clearly passionate about this, and I don’t fault you for that. But I’d suggest dialing back the aggression, rhetoric like “bootlicking” and “you should feel ashamed” doesn’t exactly invite thoughtful dialogue. It just shuts it down. Now, to the substance: I never argued against any safety measures or regulations. I specifically said I’d welcome more research and quantification of the issue. You’re citing some early studies, that’s good, but it’s also fair to question how representative or conclusive they are before leaping to sweeping policy implications. My point wasn’t that chatbots helping people kill themselves is fine. It’s that if we’re going to treat them like a public health threat, we should hold all tools and technologies that interact with people to the same standard and yes, that includes thinking carefully about how we approach mental health more broadly, not selectively. If a vulnerable person spirals after a chatbot interaction, that’s serious. But it’s also worth asking: what led them to be that vulnerable in the first place, and where were the other support systems? LLMs may exacerbate, but they aren’t the root cause. You’re framing this as a binary,either you care and want regulation, or you’re enabling exploitation. That’s a false choice. I’m saying the issue is complex, the data is early, and we shouldn’t let moral panic drive policy faster than evidence supports. Let’s push for more transparency, better safeguards, and yes, real research. But let’s also keep the conversation civil.
afavour 1y ago
Agreed. It’s depressing to see the top comment as so dismissive of the real human effects of technology. If ChatGPT is helping accelerate spirals in those with previously unseen mental health issues the answer isn’t “well they’re mentally unwell anyway”, it’s “what could ChatGPT, in its unique position, be doing to help these people”. I have to imagine an LLM is capable of detecting suicidal ideation in the context of a conversation.
- Jackpillar 1y ago
  Brother like 90% of the freaks in these threads are heavily invested into the current LLM bubble so of course they have to hand wave anything negative.
  - sReinwald 1y ago
    The funny thing is that I am, arguably, one of those "freaks" heavily invested in LLMs - in the sense that I am a very heavy and fairly enthusiastic user, not in the Angel Investor sense. I spend an embarrassing amount monthly on subscriptions and API costs. I run uncensored models locally, and I'm responsible for maintaining our AI infrastructure at work. I use them for everything from coding to creative writing to research. I'm about as far from an LLM Luddite as you can get. But here's the thing: I'm also technically literate enough to understand what I'm dealing with. I know these are effectively stochastic parrots, not oracles offering glimpses into the threads of fate, or the Matrix. I've also done years of therapy, so I know how to engage with anything related to mental health with respect, and a healthy dose of skepticism. I'm not saying I'm not vulnerable - anyone is vulnerable to manipulation. But I have the tools to recognize when an LLM is hallucinating, or reinforcing unhealthy patterns. However, it seems like the people in this article - and most people that I have talked to, even those in technical roles - didn't have that context. They thought they were talking to something that "knew more than any human possibly could." And, I think what makes it even more dangerous, they talked to something that is trained to express itself with authority, and to be helpful to users. LLMs aren't necessarily trained for nuance, or to say "I don't know", or to push back on someone experiencing a psychosis. We are all just human, and most humans don't generally like it when LLMs refuse, offer vague answers, and they don't like it when LLMs disagree. That's what got us into the mess with GPT-4o turning into an absolute sycophant a few weeks ago. (https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/) But, being pro-LLM doesn't mean one has to oppose basic safety measures. I don't want my tools neutered or censored, but I also don't want them instructing vulnerable people to stop taking their meds or encouraging them to jump off buildings. The fact that I can safely use these tools, doesn't mean everyone can. Just like the fact that my dad being able to safely handle a chain saw doesn't mean that I won't accidentally cut my arm off with one. When someone thinks ChatGPT has special access to hidden truths, they're not using it wrong - they're using it exactly as it presents itself: with confident authority about everything.

infecto 1y ago

Do not confuse yourself, skimming does not mean I did not ingest the article but I did speed read through it as it was not that well written. Why would I feel ashamed? I am not saying the crisis itself is silly but the article leads the reader through a biased take. But let’s also be real, there is an high probability every single one of these folks was having some sort of issue prior to the engagement with ChatGPT. Would love to see more research in the area and actual quantification of the issue but until that happens your argument might as well extend to all parts of life. Have a mandatory psych evaluation every year. Limitations to your access to the internet or other things, maybe even forced institutionalization if you don’t pass.

sReinwald 1y ago
In your case, "speed reading" and "skimming" are obviously just two flavors of "I didn't actually read this properly but still feel qualified to diagnose everyone involved." You'd love to see more research? You must've been reading so fast, you breezed right past the Stanford Study, the MIT Media Lab Research and the analysis by McCoy showing GPT-4o affirmed psychotic delusions 68% of the time. Or do you mean it in the sense of "we have to study this for at least 20 years and let thousands suffer and die before we can determine whether chatbots affirming people's delusions or coaching them on how to kill themselves is a bad thing"? Your concerns about the slippery slope from "maybe chatbots shouldn't coach people through suicide" to "forced psychiatric evaluations" or "institutionalization" is genuinely unhinged. And, tellingly, it betrays your view of mental healthcare as fundamentally punitive and authoritarian rather than supportive or therapeutic. Big advocate for mental health, you are. You realize we already regulate countless things to protect vulnerable people from mental harm, right? We put warning labels on cigarettes. We have age ratings on media. We don't let casinos exploit problem gamblers. We don't let bartenders serve alcohol to visibly intoxicated people. We don't let therapists tell suicidal patients to jump off buildings. None of these regulations are authoritarian overreach - they're basic safety measures that exist because in a healthy society, vulnerable people deserve protection, not exploitation. But apparently when it's time to consider holding tech to similar standards, we're supposed to just shrug and say, "well, they were probably crazy anyway." Your position essentially boils down to: "Vulnerable people exist, therefore companies should be free to exploit them." That's not a mental health advocacy position - it's tech industry bootlicking dressed up as concern trolling. That's why you should feel ashamed.
- infecto 1y ago
  You’re clearly passionate about this, and I don’t fault you for that. But I’d suggest dialing back the aggression, rhetoric like “bootlicking” and “you should feel ashamed” doesn’t exactly invite thoughtful dialogue. It just shuts it down. Now, to the substance: I never argued against any safety measures or regulations. I specifically said I’d welcome more research and quantification of the issue. You’re citing some early studies, that’s good, but it’s also fair to question how representative or conclusive they are before leaping to sweeping policy implications. My point wasn’t that chatbots helping people kill themselves is fine. It’s that if we’re going to treat them like a public health threat, we should hold all tools and technologies that interact with people to the same standard and yes, that includes thinking carefully about how we approach mental health more broadly, not selectively. If a vulnerable person spirals after a chatbot interaction, that’s serious. But it’s also worth asking: what led them to be that vulnerable in the first place, and where were the other support systems? LLMs may exacerbate, but they aren’t the root cause. You’re framing this as a binary,either you care and want regulation, or you’re enabling exploitation. That’s a false choice. I’m saying the issue is complex, the data is early, and we shouldn’t let moral panic drive policy faster than evidence supports. Let’s push for more transparency, better safeguards, and yes, real research. But let’s also keep the conversation civil.

afavour 1y ago

Agreed. It’s depressing to see the top comment as so dismissive of the real human effects of technology. If ChatGPT is helping accelerate spirals in those with previously unseen mental health issues the answer isn’t “well they’re mentally unwell anyway”, it’s “what could ChatGPT, in its unique position, be doing to help these people”. I have to imagine an LLM is capable of detecting suicidal ideation in the context of a conversation.

Jackpillar 1y ago
Brother like 90% of the freaks in these threads are heavily invested into the current LLM bubble so of course they have to hand wave anything negative.
- sReinwald 1y ago
  The funny thing is that I am, arguably, one of those "freaks" heavily invested in LLMs - in the sense that I am a very heavy and fairly enthusiastic user, not in the Angel Investor sense. I spend an embarrassing amount monthly on subscriptions and API costs. I run uncensored models locally, and I'm responsible for maintaining our AI infrastructure at work. I use them for everything from coding to creative writing to research. I'm about as far from an LLM Luddite as you can get. But here's the thing: I'm also technically literate enough to understand what I'm dealing with. I know these are effectively stochastic parrots, not oracles offering glimpses into the threads of fate, or the Matrix. I've also done years of therapy, so I know how to engage with anything related to mental health with respect, and a healthy dose of skepticism. I'm not saying I'm not vulnerable - anyone is vulnerable to manipulation. But I have the tools to recognize when an LLM is hallucinating, or reinforcing unhealthy patterns. However, it seems like the people in this article - and most people that I have talked to, even those in technical roles - didn't have that context. They thought they were talking to something that "knew more than any human possibly could." And, I think what makes it even more dangerous, they talked to something that is trained to express itself with authority, and to be helpful to users. LLMs aren't necessarily trained for nuance, or to say "I don't know", or to push back on someone experiencing a psychosis. We are all just human, and most humans don't generally like it when LLMs refuse, offer vague answers, and they don't like it when LLMs disagree. That's what got us into the mess with GPT-4o turning into an absolute sycophant a few weeks ago. (https://openai.com/index/sycophancy-in-gpt-4o/ https://openai.com/index/sycophancy-in-gpt-4o/) But, being pro-LLM doesn't mean one has to oppose basic safety measures. I don't want my tools neutered or censored, but I also don't want them instructing vulnerable people to stop taking their meds or encouraging them to jump off buildings. The fact that I can safely use these tools, doesn't mean everyone can. Just like the fact that my dad being able to safely handle a chain saw doesn't mean that I won't accidentally cut my arm off with one. When someone thinks ChatGPT has special access to hidden truths, they're not using it wrong - they're using it exactly as it presents itself: with confident authority about everything.