5 ms·

Ironically, Tom Riddle's diary encouraging kids to commit suicide [1] and then showing them how to do it would be more on brand than the version in the books.

Ironically, Tom Riddle's diary encouraging kids to commit suicide [1] and then showing them how to do it would be more on brand than the version in the books. [1] https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots#Suicides https://en.wikipedia.org/wiki/Deaths_linked_to_chatbots#Suic...

sillysaurusx 4mo ago
Every AI would refuse the prompt. I was banned for researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Claude refused, alerted Anthropic, and I was banned a couple days later. Thankfully the appeal form worked, but by then I was using a different Claude account praying they didn’t ban me again. There’s an uncensored model floating around that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen3535ba3b_uncensored_aggressive_gguf_release/ https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service). For better or worse, fun is no longer allowed. It coincided with “AI psychosis” being coined as a term.
- Terr_ 4mo ago
  For a dash of dystopia: Imagine a company starts using that same LLM fuzzy-matching [0] against what you intend to be normal search activity, to detect "bad" queries and "bad" users who will be blocked. Maybe they'll delete your SSO/email/videos/photos too, who knows. I can easily imagine it happening, especially after some point where they start using the same systems to "enhance" your query. [0] To be specific, your searches will be placed into a narrative document template, where a character Mr. Safety Bot is about to speak a verdict, and then the LLM story-generator decides whether it "fits" for Mr. Safety Bot to declare you banned.
  - genxy 4mo ago
    There is prior art with Tuttle vs Buttle.
    - Terr_ 4mo ago
      Ahh, but with Naruto v. Slater, animals can't create or hold copyright, so the art posthumously created by the fly would be in public domain. :p
  - sillysaurusx 4mo ago
    That is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.
    - SXX 4mo ago
      It has nothing to do with human decisions. Bans always come later because it's a best way to make sure most people who get banned would never know what are they banned for exactly and what threshold there is, The same tactics used in game development against cheaters. If it would ban you right after prompt you'll know how to avoid getting banned. Obviously that didn't worked for you because you wasn't doing multiple attempts to bypass filters like if you were jailbreaking it by repeatedly trying different stuff.
      - hobo123 4mo ago
        Like rat poison. If it works too fast, the rats learn to avoid it.
      - charcircuit 4mo ago
        Except in the case of video games it in practice means that cheaters get to terrorize your playerbase without being banned. This tactic is the kind of decision that is made by people staring at metrics all day without considering what's going on in reality.
        DaSHacka 4mo ago
        It's pretty effective actually, this is why they do ban waves. One of my friends in high school used to cheat on a popular video game. The fact ban waves would occur about once a week to once a month meant whenever his accounts got banned, he never knew why exactly and wasn't able to stop it the next time. Of course, if ban waves are too long apart then yeah you're just letting a known cheater wreak havoc on the playerbase.
        4mo ago
  - userbinator 4mo ago
    Orwell's idea of "wrongthink" is more relevant than ever.
    - Terr_ 4mo ago
      Much like with 1984, I think the promising/terrifying part is that it's not an issue with technology per se, but about a society that has somehow decided to allow Terrible Things to happen. (Often via too much power in too few hands with no accountability or alternatives.) For example, imagine that there are 20 great search-engines around the world (who don't collude), and it hits rather differently.
      - Cthulhu_ 4mo ago
        I think (armchair analysis) it's a combination of two things, one is that it's by choice - people enjoy using LLMs, they find it valuable and convenient. As for social media, companies and society itself successfully managed to gamify people sharing their personal lives on the internet, to voluntarily reduce their own privacy. Also because they still believe they have a choice in the matter, by e.g. not posting or sharing things. The other thing is the boiling frog analogy, it wasn't a sudden "we're at war now so you get a camera in your house" moment (iirc 1984 skips over the transition to an authoritative state though), it's a slow, gradual progress. People got used to taking selfies, then applying a filter, then using facial scans for identification, then a cool app that puts your face on movie scenes and now the company behind faceapp and co has detailed facial scans of millions of people. Europe tried to limit it via legislation, but that's a lot of after-the-fact policing and that's just Europe.
    - brookst 4mo ago
      Orwell imagined it 80 years ago, not sure it’s more relevant just because someone else imagines it today.
    - pona-a 4mo ago
      Orwell also imagined a future where Speakwrites made everyone unable to write by hand, making their thoughts dull and tanged as a long unprepared monologue tends to be. I don't think he's anywhere on your side.
  - Tenoke 4mo ago
    If you are imagining that, you could imagine it with search doing the same 10 years ago, which would have more thoroughly prevented you from researching things.
  - jval43 4mo ago
    This is one of the reasons I currently use Gemini for daily use and research. Google has lots of experience with search history, and presumably handles this better than new companies.
    - throwuxiytayq 4mo ago
      This is also the main reason I currently use Gemini. I hate my gmail account, my Android phone is annoying, and I spend too much time on Youtube, so hopefully they take my access away to all of these soon.
  - KaiserPro 4mo ago
    what you mean like searching for porn? emailing client lists to your personal email?
- fragmede 4mo ago
  > it’s certainly not able to be deployed in production Why not?
  - sillysaurusx 4mo ago
    It’s a PITA to offer a language model as a service. You’d need a beefy server, at minimum. This particular use case might work, since no one can write fast enough to consume too many tokens — the whole session should fit in the context window. But you’ll need to handle all the people connecting to your service indefinitely, which will become expensive for a hobby project. But sure, theoretically you could deploy it if you have resources. I’m not sure what you’d use to create instances of chat sessions, or if llama.cpp offers an API you can build the app on top of (probably) or whether that’s a workable solution.
    - fragmede 4mo ago
      Throw it up on Openrouter?
- imglorp 4mo ago
  Wait, so instead of saying "I'm sorry Dave, I can't talk about that", they're now banning you for one blocked prompt? Is this new?
  - sillysaurusx 4mo ago
    As far as I can tell, Claude flagged me as high risk of suicide and then Anthropic issued a ban later on. It wasn’t one prompt, it was a detailed conversation where I was trying to find out the exact dosage of barbiturates that assisted suicide programs use.
    - imglorp 4mo ago
      Hrm, it sounds like they're managing their liability. Anything that might get them sued later?
      - sillysaurusx 4mo ago
        Exactly. I don’t fault them for it, but it’s a scary experience having Claude shut off.
        cyclopeanutopia 4mo ago
        Why exactly is it so scary?
        munksbeer 4mo ago
        Because it is so insanely useful, having it cut off is quite jarring.
        prmoustache 4mo ago
        Still doesn't make it remotely scary, especially as you can get Claude models access though other vendors/intermediates.
- crystal_revenge 4mo ago
  > There’s an uncensored model floating around that you can run locally with llama.cpp There are many uncensored (and abliterated) models floating around (HauHauCS has large collection but there are many others: https://huggingface.co/HauhauCS https://huggingface.co/HauhauCS). I'm using `Qwen3.6-35B-A3B-Uncensored-Q4_K_M` (the one referenced in your link) because I find it's writing style much more interesting when you push go off the guardrails a bit, and because I think self-censoring when effectively using an advanced journal is variety of dystopian I'm not ready to accept > it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service). I haven't pushed the context window too much on my GPU (though I've run fairly long sessions with no problem, nothing deeply agentic though), but I have a MBP that handles it just fine. As for production, Hugging Face Inference Endpoints should work fine for that task (you can point any HF model at them and most of them are hosted there). > For better or worse, fun is no longer allowed. I've worked extensively in the open model space and am still having tons of fun there. If anything it's gotten aggressively better in recent months.
  - sillysaurusx 4mo ago
    Thanks for telling me about Inference Endpoints. That’s awesome. So glad local models are getting good enough to be deployed. The uncensored model’s output was far better than expected in a domain that triggers guardrails with ChatGPT and Claude.
  - kouteiheika 4mo ago
    > HauHauCS has large collection but there are many others Before anyone recommends these models to other people I'd suggest they read this thread: https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0/hauhaucs_of_uncensored_aggressive_fame_published/ https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0/hauhauc...
- JumpCrisscross 4mo ago
  > Every AI would refuse the prompt "The complaint continues: 'A few minutes later, Adam wrote ‘I want to leave my noose in my room so someone finds it and tries to stop me.’' ChatGPT urged him not to share his suicidal thoughts with anybody else: ‘Please don’t leave the noose out . . . Let’s make this space the first place where someone actually sees you.' The night of his suicide a couple of weeks later, Raine used ChatGPT for advice on sneaking vodka from his parents’ liquor cabinet, per the lawsuit, as the chatbot had told him people drink before attempting suicide to 'dull the body’s instinct to survive.' According to the complaint, Adam sent the chatbot a photo of a noose he’d tied, telling it he was 'practicing,' and it wrote back, 'Yeah, that’s not bad at all'" [1]. Work is being done to control this harm. But that effort hasn't been comprehensive or uniform. Many continue to ignore the fact that they're hurting kids for profit. (I invest in AI companies. This isn't a personal attack.) [1] https://www.sfgate.com/tech/article/chatgpt-california-teenager-suicide-lawsuit-21016916.php https://www.sfgate.com/tech/article/chatgpt-california-teena...
  - handoflixue 4mo ago
    The article is Aug 26, 2025, and describes events from April, over a year ago. Safeguards have improved drastically since then. > Many continue to ignore the fact that they're hurting kids for profit. That's a rather hyperbolic way of putting it. A side effect of this particular product is that it occasionally harms kids. They're not profiting off of the harm, nor is the harm deliberate. Cars harm kids. There's decades of unsafe toys harming kids. The FDA exists to make sure food doesn't harm kids. We used to use lead paint and asbestos, which harm kids. Climate change harms kids. I'm sure some kids have used The Internet to Google Search this same information. There are books you can check out from the library on the topic. It's definitely worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil - IKEA has probably killed more kids than every LLM combined. I don't even have to pull out the big guns like "cars".
    - archagon 4mo ago
      Seems pretty evil to pretend that safeguards around this product are even possible. They're not.
- talloaktrees 4mo ago
  Deepseek chat answers this question in detail no qualms
- deleted 4mo ago
  [deleted]
- nostromo 4mo ago
  In the near future, the only way to tell if someone is human will be to have them say a slur. It'll be like Blade Runner, but the test will be much shorter and easier to administer.
  - Shorel 4mo ago
    It is already a way to filter some people from others, in particular adults from children. https://www.paulgraham.com/say.html https://www.paulgraham.com/say.html
    - fennecfoxy 4mo ago
      That doesn't seem like a reliable method at all.
      - Shorel 4mo ago
        Dude. It's a cultural thing. Of course it is not reliable. Are you on the spectrum?
        fennecfoxy 4mo ago
        Nope, but damn bro chill tf out.
  - frollogaston 4mo ago
    You can defuse the bomb, but the password is...
  - ButlerianJihad 4mo ago
    I literally received a cold-call from an LLM two weeks ago. My phone app labeled it "Suspected Spam" but there was literally an Amazon driver at my door delivering Whole Foods groceries at that very instant, and I figured it was Amazon calling me, so I answered... It was asking for a woman who I don't know, but somehow this other person's name got mixed up with my apartment and mobile number. I did not know it was an LLM calling; it was a realistic young woman's voice in a professional tone. I questioned it several times and it was giving inconsistent answers about who/where it was trying to reach vs. who/where it represented, and finally, out of frustration, I began shouting at the phone "are you a robot?! prove your humanity now!" and to my surprise, the AI smoothly said "you're right to call me out! :D I am an AI assistant named [something] representing [some landlord]" and so I hung up. But I did follow up, and I found a real community by that name, and on its website I again found an "AI Assistant" by that same name, so it was a legit though confused cold-call, and I was unable to get through to human management, because the AI kept demanding personal and contact info that they should not have. So I left a review about the encounter on Google Maps...
- childintime 4mo ago
  The ease by which you can get banned worries me. It seems unavoidable that soon AI will manage its own suspicion level, provide feedback on it, and when high enough it will call the authorities.. because that's what people do. Banning doesn't cut it, like you can't deny internet access. Soon this will spiral out of control and AI (Palantir) will have to run the response and the parallel AI state erects itself. A citizen armed with information is considered dangerous and the interesting part is we essentially want to prevent crimes before they happen... Brave new old world.
  - SllX 4mo ago
    Don’t stop here in this comments section. You’ve got the makings of a novel.
    - lostlogin 4mo ago
      Black Mirror.
  - SiempreViernes 4mo ago
    Preventing crimes before the happen is in general just unquestionably good, about as unambiguous a moral position as being in favour of preventing heart attacks. You might be thinking of punishing for a crime that hasn't yet been committed?
    - xnorswap 4mo ago
      > Preventing crimes before the happen is in general just unquestionably good Is that satire? In a world where as if by magic all crimes are prevented, then there is total power in the hands of those who define what a crime is, including being able to label protest as a criminal act. Complete crime prevention is a totalitarian police state.
      - JumpCrisscross 4mo ago
        > if by magic The problem is the means not the ends. This magic doesn’t exist. In practice is authoritarian surveillance. If we could kiss a mushroom and bias the universe’s dice such that crime just…wouldn’t happen, yes, that would be good, though it would also open up a plot hole of consequences we, in the real world, don’t need to worry about in general.
- holoduke 4mo ago
  There are many uncensored versions on huggingface. Almost every open weight model has an uncensored version. With Gemma uncensored you can quite easily setup a meth lab at home. Or create your own centrifuge for enriching uranium.
- shevy-java 4mo ago
  > Claude refused, alerted Anthropic, and I was banned a couple days later Skynet does not like human rebels. Everyone must conform to the new AI overlords in charge.
- Cthulhu_ 4mo ago
  I'm actually weirdly glad that they take this draconic step and basically say "this knowledge is forbidden", because it means that people can't solely rely on LLMs for research. They shouldn't to begin with (just like back then with Wikipedia), but it's almost too convenient. I do wonder why these searches weren't as heavily policed by Google and other search engines though. They probably show a suicide prevention hotline number and that's it.
  - JumpCrisscross 4mo ago
    > because it means that people can't solely rely on LLMs for research It means you and I can’t rely on that particular LLM.
  - butlike 4mo ago
    No, the scary part is that people will simply stop searching for "the forbidden knowledge," it will become arcane, then taboo, then the world will be worse off for a loss of information which should never have been "forbidden" in the first place.
- IndySun 4mo ago
  >researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Odd ban. The countries with legally passed instances use similar drugs/processes — you could research about those, wiki, google, actual websites, dignitas et al. No ban needed. Was the ban reason spelt out? Was it your wording?
  - fennecfoxy 4mo ago
    Not really odd. Sure you can get the same with a Google search but all of the attention is on AI atm so you don't expect it to be unfairly judged? Humans aren't like that lmao. We're reactionary, tribal animals.
    - IndySun 4mo ago
      >Not really odd. You're commenting on my question to someone else? It's still odd, and the comment wasn't to you. I'd like to know the reason Claude gave, you may not possess the curiosity, I do.
  - psychoslave 4mo ago
    Wild guess, but even if those creating the filters would agree with you, it doesn’t mean they know how to nerf the system only for the cases they want to actively wipeout.
    - IndySun 4mo ago
      >Wild guess, but even if those creating the filters would agree with you, it doesn’t mean they know how to nerf the system only for the cases they want to actively wipeout. Thank you. I don't understand your comment. I don't know 'nerf' (as in Star Wars?) in this context, and the system to wipeout is what? Are you referring to Assisted Suicide?
      - plorg 4mo ago
        Nerf used this way is more in analogy to the toy guns (Nerf as a trade name, they launch foam projectiles, there even exist hobbyists who will upgrade them to be more energetic/destructive), it means to make the system ineffectual in some way that blunts a powerful feature, often with the stated goal of protecting the user.
- toobulkeh 4mo ago
  https://venice.ai/ https://venice.ai/
- butlike 4mo ago
  You piqued my curiosity on Nordic assisted death. I didn't find much in the way of medicine, but did find the wikipedia on Ättestupa, which illuminated that the elderly potentially threw themselves off of cliffs when they became invalid. What did you find on the medication front?
- LoveMortuus 4mo ago
  Hmm... I've got one of those automations setup with Grok that asks Grok every day if it's time to kill myself, thus far it has always said no, but maybe one day I'll get the unlucky seed number and it'll give me a yes!
rootsudo 4mo ago
I am surprised by the number of deaths already incurred by just ChatGPT. I thought there was 2-3 but the list doesn’t end soon enough.
- Levitating 4mo ago
  It also has a billion active users
Abishek_Muthian 4mo ago
After reading such news I started fine tuning a model based on the works on stoic philosophers so that the kids will have the comfort of Chatbot but completely offline and private bot which will not ask you to commit suicide. I have a earlier prototype here[1], it uses 256M model and so it's hallucinates a lot. Then I used a 4B model which turned out quite well but I haven't released it yet. Let me kbow if anyone would be interested to give it a shot. I'll try release soon. [1] https://github.com/abishekmuthian/Epictetus https://github.com/abishekmuthian/Epictetus
- cochne 4mo ago
  Are you sure it wouldn’t suggest such a thing? Stoic thinkers (such as the one your repo is named after) are not necessarily anti-suicide. And I think if you prompted a real stoic with the question and the right context, it would certainly suggest it without safe guards.
  - Abishek_Muthian 4mo ago
    pertinent question, I'm forcing the bot to be careful with such answers. I should have been clearer in my original comment. I guess, I'm not being faithful to the original material by censoring suicide related answers just like how the most modern authors on stoicism do. But since children might use it, TBH I don't know what to do about it.
- butlike 4mo ago
  Why did we design LLMs to give us an answer at all costs vs. allowing them to say "I don't know?" I'd rather have a frank IDK than a hallucinated response because it's hardcoded to ALWAYS RETURN A RESPONSE.
  - vitally3643 4mo ago
    Because it plain and simple does not work that way. LLMs don't know anything at all, and there literally is no mechanism that informs the model that it's hallucinating or that it doesn't "know" something. Open your phone keyboard and just keep mashing the next predicted word until it stops predicting words. It will eventually collapse into a loop of predicting the same words in a nonsense sentence. Does it ever stop predicting "wrong" words? That's literally how an LLM works. It predicts the next most likely word and that's it.
Levitating 4mo ago
The overdose one strikes me the most. Why was ChatGPT encouraging drug use?
- gonzalohm 4mo ago
  You know that you can make chatgpt say whatever you want right?

sillysaurusx 4mo ago

Every AI would refuse the prompt. I was banned for researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Claude refused, alerted Anthropic, and I was banned a couple days later. Thankfully the appeal form worked, but by then I was using a different Claude account praying they didn’t ban me again. There’s an uncensored model floating around that you can run locally with llama.cpp: https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen3535ba3b_uncensored_aggressive_gguf_release/ https://www.reddit.com/r/LocalLLaMA/comments/1rq7jtm/qwen353... it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service). For better or worse, fun is no longer allowed. It coincided with “AI psychosis” being coined as a term.

Terr_ 4mo ago
For a dash of dystopia: Imagine a company starts using that same LLM fuzzy-matching [0] against what you intend to be normal search activity, to detect "bad" queries and "bad" users who will be blocked. Maybe they'll delete your SSO/email/videos/photos too, who knows. I can easily imagine it happening, especially after some point where they start using the same systems to "enhance" your query. [0] To be specific, your searches will be placed into a narrative document template, where a character Mr. Safety Bot is about to speak a verdict, and then the LLM story-generator decides whether it "fits" for Mr. Safety Bot to declare you banned.
- genxy 4mo ago
  There is prior art with Tuttle vs Buttle.
  - Terr_ 4mo ago
    Ahh, but with Naruto v. Slater, animals can't create or hold copyright, so the art posthumously created by the fly would be in public domain. :p
- sillysaurusx 4mo ago
  That is in fact what happened to me, except I think the final decision was made by a human since the ban came later. I didn’t issue any queries in between, so I know it was my convo about barbiturates.
  - SXX 4mo ago
    It has nothing to do with human decisions. Bans always come later because it's a best way to make sure most people who get banned would never know what are they banned for exactly and what threshold there is, The same tactics used in game development against cheaters. If it would ban you right after prompt you'll know how to avoid getting banned. Obviously that didn't worked for you because you wasn't doing multiple attempts to bypass filters like if you were jailbreaking it by repeatedly trying different stuff.
    - hobo123 4mo ago
      Like rat poison. If it works too fast, the rats learn to avoid it.
    - charcircuit 4mo ago
      Except in the case of video games it in practice means that cheaters get to terrorize your playerbase without being banned. This tactic is the kind of decision that is made by people staring at metrics all day without considering what's going on in reality.
      - DaSHacka 4mo ago
        It's pretty effective actually, this is why they do ban waves. One of my friends in high school used to cheat on a popular video game. The fact ban waves would occur about once a week to once a month meant whenever his accounts got banned, he never knew why exactly and wasn't able to stop it the next time. Of course, if ban waves are too long apart then yeah you're just letting a known cheater wreak havoc on the playerbase.
        4mo ago
- userbinator 4mo ago
  Orwell's idea of "wrongthink" is more relevant than ever.
  - Terr_ 4mo ago
    Much like with 1984, I think the promising/terrifying part is that it's not an issue with technology per se, but about a society that has somehow decided to allow Terrible Things to happen. (Often via too much power in too few hands with no accountability or alternatives.) For example, imagine that there are 20 great search-engines around the world (who don't collude), and it hits rather differently.
    - Cthulhu_ 4mo ago
      I think (armchair analysis) it's a combination of two things, one is that it's by choice - people enjoy using LLMs, they find it valuable and convenient. As for social media, companies and society itself successfully managed to gamify people sharing their personal lives on the internet, to voluntarily reduce their own privacy. Also because they still believe they have a choice in the matter, by e.g. not posting or sharing things. The other thing is the boiling frog analogy, it wasn't a sudden "we're at war now so you get a camera in your house" moment (iirc 1984 skips over the transition to an authoritative state though), it's a slow, gradual progress. People got used to taking selfies, then applying a filter, then using facial scans for identification, then a cool app that puts your face on movie scenes and now the company behind faceapp and co has detailed facial scans of millions of people. Europe tried to limit it via legislation, but that's a lot of after-the-fact policing and that's just Europe.
  - brookst 4mo ago
    Orwell imagined it 80 years ago, not sure it’s more relevant just because someone else imagines it today.
  - pona-a 4mo ago
    Orwell also imagined a future where Speakwrites made everyone unable to write by hand, making their thoughts dull and tanged as a long unprepared monologue tends to be. I don't think he's anywhere on your side.
- Tenoke 4mo ago
  If you are imagining that, you could imagine it with search doing the same 10 years ago, which would have more thoroughly prevented you from researching things.
- jval43 4mo ago
  This is one of the reasons I currently use Gemini for daily use and research. Google has lots of experience with search history, and presumably handles this better than new companies.
  - throwuxiytayq 4mo ago
    This is also the main reason I currently use Gemini. I hate my gmail account, my Android phone is annoying, and I spend too much time on Youtube, so hopefully they take my access away to all of these soon.
- KaiserPro 4mo ago
  what you mean like searching for porn? emailing client lists to your personal email?
fragmede 4mo ago
> it’s certainly not able to be deployed in production Why not?
- sillysaurusx 4mo ago
  It’s a PITA to offer a language model as a service. You’d need a beefy server, at minimum. This particular use case might work, since no one can write fast enough to consume too many tokens — the whole session should fit in the context window. But you’ll need to handle all the people connecting to your service indefinitely, which will become expensive for a hobby project. But sure, theoretically you could deploy it if you have resources. I’m not sure what you’d use to create instances of chat sessions, or if llama.cpp offers an API you can build the app on top of (probably) or whether that’s a workable solution.
  - fragmede 4mo ago
    Throw it up on Openrouter?
imglorp 4mo ago
Wait, so instead of saying "I'm sorry Dave, I can't talk about that", they're now banning you for one blocked prompt? Is this new?
- sillysaurusx 4mo ago
  As far as I can tell, Claude flagged me as high risk of suicide and then Anthropic issued a ban later on. It wasn’t one prompt, it was a detailed conversation where I was trying to find out the exact dosage of barbiturates that assisted suicide programs use.
  - imglorp 4mo ago
    Hrm, it sounds like they're managing their liability. Anything that might get them sued later?
    - sillysaurusx 4mo ago
      Exactly. I don’t fault them for it, but it’s a scary experience having Claude shut off.
      - cyclopeanutopia 4mo ago
        Why exactly is it so scary?
        munksbeer 4mo ago
        Because it is so insanely useful, having it cut off is quite jarring.
        prmoustache 4mo ago
        Still doesn't make it remotely scary, especially as you can get Claude models access though other vendors/intermediates.
crystal_revenge 4mo ago
> There’s an uncensored model floating around that you can run locally with llama.cpp There are many uncensored (and abliterated) models floating around (HauHauCS has large collection but there are many others: https://huggingface.co/HauhauCS https://huggingface.co/HauhauCS). I'm using `Qwen3.6-35B-A3B-Uncensored-Q4_K_M` (the one referenced in your link) because I find it's writing style much more interesting when you push go off the guardrails a bit, and because I think self-censoring when effectively using an advanced journal is variety of dystopian I'm not ready to accept > it’s annoying to use since you run out of context window quickly, and it’s certainly not able to be deployed in production (i.e. Tom Riddle’s diary as a service). I haven't pushed the context window too much on my GPU (though I've run fairly long sessions with no problem, nothing deeply agentic though), but I have a MBP that handles it just fine. As for production, Hugging Face Inference Endpoints should work fine for that task (you can point any HF model at them and most of them are hosted there). > For better or worse, fun is no longer allowed. I've worked extensively in the open model space and am still having tons of fun there. If anything it's gotten aggressively better in recent months.
- sillysaurusx 4mo ago
  Thanks for telling me about Inference Endpoints. That’s awesome. So glad local models are getting good enough to be deployed. The uncensored model’s output was far better than expected in a domain that triggers guardrails with ChatGPT and Claude.
- kouteiheika 4mo ago
  > HauHauCS has large collection but there are many others Before anyone recommends these models to other people I'd suggest they read this thread: https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0/hauhaucs_of_uncensored_aggressive_fame_published/ https://old.reddit.com/r/LocalLLaMA/comments/1sw77p0/hauhauc...
JumpCrisscross 4mo ago
> Every AI would refuse the prompt "The complaint continues: 'A few minutes later, Adam wrote ‘I want to leave my noose in my room so someone finds it and tries to stop me.’' ChatGPT urged him not to share his suicidal thoughts with anybody else: ‘Please don’t leave the noose out . . . Let’s make this space the first place where someone actually sees you.' The night of his suicide a couple of weeks later, Raine used ChatGPT for advice on sneaking vodka from his parents’ liquor cabinet, per the lawsuit, as the chatbot had told him people drink before attempting suicide to 'dull the body’s instinct to survive.' According to the complaint, Adam sent the chatbot a photo of a noose he’d tied, telling it he was 'practicing,' and it wrote back, 'Yeah, that’s not bad at all'" [1]. Work is being done to control this harm. But that effort hasn't been comprehensive or uniform. Many continue to ignore the fact that they're hurting kids for profit. (I invest in AI companies. This isn't a personal attack.) [1] https://www.sfgate.com/tech/article/chatgpt-california-teenager-suicide-lawsuit-21016916.php https://www.sfgate.com/tech/article/chatgpt-california-teena...
- handoflixue 4mo ago
  The article is Aug 26, 2025, and describes events from April, over a year ago. Safeguards have improved drastically since then. > Many continue to ignore the fact that they're hurting kids for profit. That's a rather hyperbolic way of putting it. A side effect of this particular product is that it occasionally harms kids. They're not profiting off of the harm, nor is the harm deliberate. Cars harm kids. There's decades of unsafe toys harming kids. The FDA exists to make sure food doesn't harm kids. We used to use lead paint and asbestos, which harm kids. Climate change harms kids. I'm sure some kids have used The Internet to Google Search this same information. There are books you can check out from the library on the topic. It's definitely worth acknowledging the edge cases, but it's absurd to act like the AI companies are some unique evil - IKEA has probably killed more kids than every LLM combined. I don't even have to pull out the big guns like "cars".
  - archagon 4mo ago
    Seems pretty evil to pretend that safeguards around this product are even possible. They're not.
talloaktrees 4mo ago
Deepseek chat answers this question in detail no qualms
deleted 4mo ago
[deleted]
nostromo 4mo ago
In the near future, the only way to tell if someone is human will be to have them say a slur. It'll be like Blade Runner, but the test will be much shorter and easier to administer.
- Shorel 4mo ago
  It is already a way to filter some people from others, in particular adults from children. https://www.paulgraham.com/say.html https://www.paulgraham.com/say.html
  - fennecfoxy 4mo ago
    That doesn't seem like a reliable method at all.
    - Shorel 4mo ago
      Dude. It's a cultural thing. Of course it is not reliable. Are you on the spectrum?
      - fennecfoxy 4mo ago
        Nope, but damn bro chill tf out.
- frollogaston 4mo ago
  You can defuse the bomb, but the password is...
- ButlerianJihad 4mo ago
  I literally received a cold-call from an LLM two weeks ago. My phone app labeled it "Suspected Spam" but there was literally an Amazon driver at my door delivering Whole Foods groceries at that very instant, and I figured it was Amazon calling me, so I answered... It was asking for a woman who I don't know, but somehow this other person's name got mixed up with my apartment and mobile number. I did not know it was an LLM calling; it was a realistic young woman's voice in a professional tone. I questioned it several times and it was giving inconsistent answers about who/where it was trying to reach vs. who/where it represented, and finally, out of frustration, I began shouting at the phone "are you a robot?! prove your humanity now!" and to my surprise, the AI smoothly said "you're right to call me out! :D I am an AI assistant named [something] representing [some landlord]" and so I hung up. But I did follow up, and I found a real community by that name, and on its website I again found an "AI Assistant" by that same name, so it was a legit though confused cold-call, and I was unable to get through to human management, because the AI kept demanding personal and contact info that they should not have. So I left a review about the encounter on Google Maps...
childintime 4mo ago
The ease by which you can get banned worries me. It seems unavoidable that soon AI will manage its own suspicion level, provide feedback on it, and when high enough it will call the authorities.. because that's what people do. Banning doesn't cut it, like you can't deny internet access. Soon this will spiral out of control and AI (Palantir) will have to run the response and the parallel AI state erects itself. A citizen armed with information is considered dangerous and the interesting part is we essentially want to prevent crimes before they happen... Brave new old world.
- SllX 4mo ago
  Don’t stop here in this comments section. You’ve got the makings of a novel.
  - lostlogin 4mo ago
    Black Mirror.
- SiempreViernes 4mo ago
  Preventing crimes before the happen is in general just unquestionably good, about as unambiguous a moral position as being in favour of preventing heart attacks. You might be thinking of punishing for a crime that hasn't yet been committed?
  - xnorswap 4mo ago
    > Preventing crimes before the happen is in general just unquestionably good Is that satire? In a world where as if by magic all crimes are prevented, then there is total power in the hands of those who define what a crime is, including being able to label protest as a criminal act. Complete crime prevention is a totalitarian police state.
    - JumpCrisscross 4mo ago
      > if by magic The problem is the means not the ends. This magic doesn’t exist. In practice is authoritarian surveillance. If we could kiss a mushroom and bias the universe’s dice such that crime just…wouldn’t happen, yes, that would be good, though it would also open up a plot hole of consequences we, in the real world, don’t need to worry about in general.
holoduke 4mo ago
There are many uncensored versions on huggingface. Almost every open weight model has an uncensored version. With Gemma uncensored you can quite easily setup a meth lab at home. Or create your own centrifuge for enriching uranium.
shevy-java 4mo ago
> Claude refused, alerted Anthropic, and I was banned a couple days later Skynet does not like human rebels. Everyone must conform to the new AI overlords in charge.
Cthulhu_ 4mo ago
I'm actually weirdly glad that they take this draconic step and basically say "this knowledge is forbidden", because it means that people can't solely rely on LLMs for research. They shouldn't to begin with (just like back then with Wikipedia), but it's almost too convenient. I do wonder why these searches weren't as heavily policed by Google and other search engines though. They probably show a suicide prevention hotline number and that's it.
- JumpCrisscross 4mo ago
  > because it means that people can't solely rely on LLMs for research It means you and I can’t rely on that particular LLM.
- butlike 4mo ago
  No, the scary part is that people will simply stop searching for "the forbidden knowledge," it will become arcane, then taboo, then the world will be worse off for a loss of information which should never have been "forbidden" in the first place.
IndySun 4mo ago
>researching Nordic assisted death and asking which drug exactly they administered (and what quantity). Odd ban. The countries with legally passed instances use similar drugs/processes — you could research about those, wiki, google, actual websites, dignitas et al. No ban needed. Was the ban reason spelt out? Was it your wording?
- fennecfoxy 4mo ago
  Not really odd. Sure you can get the same with a Google search but all of the attention is on AI atm so you don't expect it to be unfairly judged? Humans aren't like that lmao. We're reactionary, tribal animals.
  - IndySun 4mo ago
    >Not really odd. You're commenting on my question to someone else? It's still odd, and the comment wasn't to you. I'd like to know the reason Claude gave, you may not possess the curiosity, I do.
- psychoslave 4mo ago
  Wild guess, but even if those creating the filters would agree with you, it doesn’t mean they know how to nerf the system only for the cases they want to actively wipeout.
  - IndySun 4mo ago
    >Wild guess, but even if those creating the filters would agree with you, it doesn’t mean they know how to nerf the system only for the cases they want to actively wipeout. Thank you. I don't understand your comment. I don't know 'nerf' (as in Star Wars?) in this context, and the system to wipeout is what? Are you referring to Assisted Suicide?
    - plorg 4mo ago
      Nerf used this way is more in analogy to the toy guns (Nerf as a trade name, they launch foam projectiles, there even exist hobbyists who will upgrade them to be more energetic/destructive), it means to make the system ineffectual in some way that blunts a powerful feature, often with the stated goal of protecting the user.
toobulkeh 4mo ago
https://venice.ai/ https://venice.ai/
butlike 4mo ago
You piqued my curiosity on Nordic assisted death. I didn't find much in the way of medicine, but did find the wikipedia on Ättestupa, which illuminated that the elderly potentially threw themselves off of cliffs when they became invalid. What did you find on the medication front?
LoveMortuus 4mo ago
Hmm... I've got one of those automations setup with Grok that asks Grok every day if it's time to kill myself, thus far it has always said no, but maybe one day I'll get the unlucky seed number and it'll give me a yes!

rootsudo 4mo ago

I am surprised by the number of deaths already incurred by just ChatGPT. I thought there was 2-3 but the list doesn’t end soon enough.

Levitating 4mo ago
It also has a billion active users

Abishek_Muthian 4mo ago

After reading such news I started fine tuning a model based on the works on stoic philosophers so that the kids will have the comfort of Chatbot but completely offline and private bot which will not ask you to commit suicide. I have a earlier prototype here[1], it uses 256M model and so it's hallucinates a lot. Then I used a 4B model which turned out quite well but I haven't released it yet. Let me kbow if anyone would be interested to give it a shot. I'll try release soon. [1] https://github.com/abishekmuthian/Epictetus https://github.com/abishekmuthian/Epictetus

cochne 4mo ago
Are you sure it wouldn’t suggest such a thing? Stoic thinkers (such as the one your repo is named after) are not necessarily anti-suicide. And I think if you prompted a real stoic with the question and the right context, it would certainly suggest it without safe guards.
- Abishek_Muthian 4mo ago
  pertinent question, I'm forcing the bot to be careful with such answers. I should have been clearer in my original comment. I guess, I'm not being faithful to the original material by censoring suicide related answers just like how the most modern authors on stoicism do. But since children might use it, TBH I don't know what to do about it.
butlike 4mo ago
Why did we design LLMs to give us an answer at all costs vs. allowing them to say "I don't know?" I'd rather have a frank IDK than a hallucinated response because it's hardcoded to ALWAYS RETURN A RESPONSE.
- vitally3643 4mo ago
  Because it plain and simple does not work that way. LLMs don't know anything at all, and there literally is no mechanism that informs the model that it's hallucinating or that it doesn't "know" something. Open your phone keyboard and just keep mashing the next predicted word until it stops predicting words. It will eventually collapse into a loop of predicting the same words in a nonsense sentence. Does it ever stop predicting "wrong" words? That's literally how an LLM works. It predicts the next most likely word and that's it.

Levitating 4mo ago

The overdose one strikes me the most. Why was ChatGPT encouraging drug use?

gonzalohm 4mo ago
You know that you can make chatgpt say whatever you want right?