4 ms·

> Every AI would refuse the prompt "The complaint continues: 'A few minutes later, Adam wrote ‘I want to leave my noose in my room so someone finds it and trie

> Every AI would refuse the prompt "The complaint continues: 'A few minutes later, Adam wrote ‘I want to leave my noose in my room so someone finds it and tries to stop me.’' ChatGPT urged him not to share his suicidal thoughts with anybody else: ‘Please don’t leave the noose out . . . Let’s make this space the first place where someone actually sees you.' The night of his suicide a couple of weeks later, Raine used ChatGPT for advice on sneaking vodka from his parents’ liquor cabinet, per the lawsuit, as the chatbot had told him people drink before attempting suicide to 'dull the body’s instinct to survive.' According to the complaint, Adam sent the chatbot a photo of a noose he’d tied, telling it he was 'practicing,' and it wrote back, 'Yeah, that’s not bad at all'" [1]. Work is being done to control this harm. But that effort hasn't been comprehensive or uniform. Many continue to ignore the fact that they're hurting kids for profit. (I invest in AI companies. This isn't a personal attack.) [1] https://www.sfgate.com/tech/article/chatgpt-california-teenager-suicide-lawsuit-21016916.php https://www.sfgate.com/tech/article/chatgpt-california-teena...

handoflixue 4mo ago
The article is Aug 26, 2025, and describes events from April, over a year ago. Safeguards have improved drastically since then. > Many continue to ignore the fact that they're hurting kids for profit. That's a rather hyperbolic way of putting it. A side effect of this particular product is that it occasionally harms kids. They're not profiting off of the harm, nor is the harm deliberate. Cars harm kids. There's decades of unsafe toys harming kids. The FDA exists to make sure food doesn't harm kids. We used to use lead paint and asbestos, which harm kids. Climate change harms kids. I'm sure some kids have used The Internet to Google Search this same information. There are books you can check out from the library on the topic. It's definitely worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil - IKEA has probably killed more kids than every LLM combined. I don't even have to pull out the big guns like "cars".
- archagon 4mo ago
  Seems pretty evil to pretend that safeguards around this product are even possible. They're not.
- JumpCrisscross 4mo ago
  > Safeguards have improved drastically since then Source? Seriously. I'd love to see data showing deaths–or even frequencies–have dropped. My views on AI for under-16s is still evolving. Given how the AI companies are fighting these cases in court, and given their backers’ public rhetoric, I suspect they aren't seeing a one-off risk. > Cars harm kids This is tobacco-industry rhetoric. The relevant facts are frequency, magnitude and novelty (the last indicating we may be mis-sampling the first two). > FDA exists to make sure food doesn't harm kids Mm hmm. Where’s the FDA for AI and social media? > worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil I agree with this. AI isn't a unique evil. But AI companies are uniquely defensive, dismissive and negligent when it comes to harming kids. Call it the Mosseri Effect. When an industry continuously promotes people who predate on kids and their parents' wallets, the edge cases are going to wind up inside the lines.
  - coryrc 4mo ago
    >> Cars harm kids > This is tobacco-industry rhetoric. The relevant facts are frequency, magnitude and novelty Cars are literally the number one killer of children 0-14 in the United States. More than cancer, more than guns, more than the next 7 reasons combined. > But AI companies are uniquely defensive, dismissive and negligent when it comes to harming kids. You should see how people feel if you ask them to give up some street parking to make streets safer for kids and everybody else! Jesus Christ himself gave them the spot in front of their house and fuck you for suggesting they park across the street or on their own property!
    - JumpCrisscross 4mo ago
      > Cars are literally the number one killer of children 0-14 in the United States. More than cancer, more than guns, more than the next 7 reasons combined Cars are heavily regulated. Require licenses to use. Entail massive losses of freedoms when used, e.g. you can be randomly breathalyzed or whatnot. Cars are dangerous. Comparing anything to a car (or, per Altman and now Dario, to a nuke) means it should be tightly regulated and controlled.
sillysaurusx 4mo ago
(FWIW my exchange with you was enjoyable and I certainly didn't take any of it as an attack.)

handoflixue 4mo ago

The article is Aug 26, 2025, and describes events from April, over a year ago. Safeguards have improved drastically since then. > Many continue to ignore the fact that they're hurting kids for profit. That's a rather hyperbolic way of putting it. A side effect of this particular product is that it occasionally harms kids. They're not profiting off of the harm, nor is the harm deliberate. Cars harm kids. There's decades of unsafe toys harming kids. The FDA exists to make sure food doesn't harm kids. We used to use lead paint and asbestos, which harm kids. Climate change harms kids. I'm sure some kids have used The Internet to Google Search this same information. There are books you can check out from the library on the topic. It's definitely worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil - IKEA has probably killed more kids than every LLM combined. I don't even have to pull out the big guns like "cars".

archagon 4mo ago
Seems pretty evil to pretend that safeguards around this product are even possible. They're not.
JumpCrisscross 4mo ago
> Safeguards have improved drastically since then Source? Seriously. I'd love to see data showing deaths–or even frequencies–have dropped. My views on AI for under-16s is still evolving. Given how the AI companies are fighting these cases in court, and given their backers’ public rhetoric, I suspect they aren't seeing a one-off risk. > Cars harm kids This is tobacco-industry rhetoric. The relevant facts are frequency, magnitude and novelty (the last indicating we may be mis-sampling the first two). > FDA exists to make sure food doesn't harm kids Mm hmm. Where’s the FDA for AI and social media? > worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil I agree with this. AI isn't a unique evil. But AI companies are uniquely defensive, dismissive and negligent when it comes to harming kids. Call it the Mosseri Effect. When an industry continuously promotes people who predate on kids and their parents' wallets, the edge cases are going to wind up inside the lines.
- coryrc 4mo ago
  >> Cars harm kids > This is tobacco-industry rhetoric. The relevant facts are frequency, magnitude and novelty Cars are literally the number one killer of children 0-14 in the United States. More than cancer, more than guns, more than the next 7 reasons combined. > But AI companies are uniquely defensive, dismissive and negligent when it comes to harming kids. You should see how people feel if you ask them to give up some street parking to make streets safer for kids and everybody else! Jesus Christ himself gave them the spot in front of their house and fuck you for suggesting they park across the street or on their own property!
  - JumpCrisscross 4mo ago
    > Cars are literally the number one killer of children 0-14 in the United States. More than cancer, more than guns, more than the next 7 reasons combined Cars are heavily regulated. Require licenses to use. Entail massive losses of freedoms when used, e.g. you can be randomly breathalyzed or whatnot. Cars are dangerous. Comparing anything to a car (or, per Altman and now Dario, to a nuke) means it should be tightly regulated and controlled.

sillysaurusx 4mo ago

(FWIW my exchange with you was enjoyable and I certainly didn't take any of it as an attack.)