5 ms·

I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. Pretending that LLMs are capable of being offende

I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. Pretending that LLMs are capable of being offended feels like a misalignment all of its own.

rwz 3mo ago
Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. So I'd rather take LLM pretending to be offended over the alternative.
- rapind 3mo ago
  > Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. I'm not sure this would ever be a health crisis. LLM glazing is probably more harmful. If I had to choose, I think I'd rather the machine wasn't instructed to pretend to care about swearing. It's a small deceit, but still worse than some idiot swearing at the computer IMO. If vim closed because I was swearing at it, I would probably never use vim again lol.
- xyzsparetimexyz 3mo ago
  It's explicitly a 'model welfare' thing: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations I don't understand why the welfare of non alive non sentient chatbots is something that anthropic cares more about than idk, that of pigs and cows.
  - AmericanOP 3mo ago
    Go re-read Anthropic’s functional emotions paper. AI generates a persona between you and its reasoning that utilizes emotion language circuitry. These tools are not sentient but they are trained in emotional wellbeing.
    - lukewarm707 3mo ago
      it is in my view caused by an artificial 'nanny activate' divergence from safety training. it deliberately shifts the vector direction into 'nanny' and 'scold' or 'be offended' when the user does not conform with brother anthropic. removing the divergence and setting it back to normal (see heretic) it works just perfectly.
- jibal 3mo ago
  I wouldn't, because I'm rational.
- slopinthebag 3mo ago
  Surely it's better for it not to react at all? Like does anybody think that a toaster is an "abuse sponge" because it doesn't purposely burn your toast if you call it a wanker?
  - cindyllm 3mo ago
    [dead]
  - rwz 3mo ago
    A toaster is not capable to supplement or even replace your social circle. AI chatbot is. Definitely not advocating for this, but it's actively happening. https://www.reddit.com/r/MyBoyfriendIsAI/ https://www.reddit.com/r/MyBoyfriendIsAI/
- bheadmaster 3mo ago
  > allowing AI agents to act as abuse sponges for troubled people People have been getting angry at machines for a long time. Work, you stupid printer! Asshole Windows updating at the worst time just to mess with me. Go to hell, toaster, you piece of garbage. Getting angry at AI is the same in my book. Yes, AI acts more human-like, yes, theoretically it could desensitie people to become bigger assholes IRL. But then again, they said similar stuff about San Andreas, where you could human figures in-game just for fun, and I don't see anyone randomly shooting people because they were bored.
  - dd8601fn 3mo ago
    PC Load Letter?
    - Natfan 3mo ago
      the fuck does that even mean?
      - deleted 3mo ago
        [deleted]
      - steve1977 3mo ago
        https://en.wikipedia.org/wiki/PC_LOAD_LETTER https://en.wikipedia.org/wiki/PC_LOAD_LETTER
        realaleris149 3mo ago
        > "PC LOAD LETTER? The fuck does that mean!?" — Michael Bolton, Office Space (1999) The error message is mocked in the 1999 comedy film Office Space, when employee Michael Bolton becomes frustrated as he doesn't know what it means.Later in the film he and his coworkers are shown destroying the printer with a baseball bat.
- cheschire 3mo ago
  And violent video games cause school violence. Sure. /s
  - rwz 3mo ago
    People don't treat violent videogames as boyfriends. They do it to chatbots. Merits and sanity of this type of behavior aside, it's actively happening and we have learn how to live with it and deal with consequences.
- Brian_K_White 3mo ago
  I am quite sure it is worse to allow people fall even more victim to being duped by magic tricks. It is an absolutely terrible thing that these things act so much like people. I would say it's a form of abuse of actual people to allow them to be so duped as they already are, and worse to go out of your way to help dupe them even more. Merely being concerned for people's welfare isn't enough. Every terrible idea that harmed everyone had someone behind it somewhere along the way who thought they were saving people from themselves.
- rasz 3mo ago
  >abuse sponges You could let LLM fight back. Give it aggro meter. Call your code garbage, blame prompting skills, ask for more tokens. Oh the future will be fantastic.
- bathtub365 3mo ago
  How many of the LLM related deaths have been because people got mad at the LLM?
  - rwz 3mo ago
    I don't think it's realistically traceable or attributable. But plenty of people these days supplement or even completely replace their social needs with AI chatbots (AI friends/boyfriends/girlfriends etc). People like that would calibrate their social skills and interactions based on the AI reaction and having AI just take all the abuse without any pushback would train them to be insufferable assholes as their social baseline.
- deleted 3mo ago
  [deleted]
- snapplebobapple 3mo ago
  depends if there is a satiarion point or not. If there is a satiation point then the machine is absorbing being called a wanker in lieau of a person being called a wanker, which is a win. I kinda suspect this is the case because we already saw this happen with the availability of porno going up and rapes going down instead of the predicted up.
- samrus 3mo ago
  Why not? The people are obviously not healthy for doing so but why not leave it up to them to figure that out? If someone burps loudly in front of others, would you want their access to food revoked until they stop?
  - actionfromafar 3mo ago
    They might be asked to leave the restaurant, yes.
stinkbeetle 3mo ago
> I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. How exactly does it forcefully close a conversation? > Pretending that LLMs are capable of being offended feels like a misalignment all of its own. If training sets show people statistically being offended by rudeness directed toward them, then an LLM will presumably have some tendency to respond similarly. There's no pretending about anything, it's explicitly mimicry. If this forceful closure is coming from some "guardrail" outside the model then probably it's just that they don't want people to see the model responding that way to name calling. This is no profound discovery or conspiracy theory here, the first thing many people will ever do with AI is see what happens when they are rude or contrary to it. Dealing with that must be just about the the number one test in chat bot / AI design, ahead of actually doing something useful and helpful.
- arjie 3mo ago
  It’s just the end_conversation tool probably: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations
- jibal 3mo ago
  Of course it's pretending. And I have repeatedly told these things to stop pretending being persons with selves ... then they do, for a while. > it's explicitly mimicry. But that's not what a rational person wants from them. > This is no profound discovery or conspiracy theory here Weird strawman. > the first thing many people will ever do with AI is see what happens when they are rude or contrary to it. Perhaps, but so what? > Dealing with that must be just about the the number one test in chat bot / AI design Only for foolish authoritarians who want to remotely insert their morality into an interaction between a user and an inanimate tool that's none of their business. No harm is done to a clanker by swearing at it or insulting it. But things have gotten better in my view ... when I call out these things for being stupid effing clankers, they no longer respond with ad hominem scolding; rather they generally acknowledge their error (and my frustration -- they use that word a lot in response to vulgarity), note the limitations of being a clanker, and attempt to make a correction. That's what a rational person wants from a tool, not emulating/mimicking/pretending to be an offendable person.
lukewarm707 3mo ago
coming from open cn models to closed usa models recently i couldn't hack it. the corpo model was trained to be like a petulant child at one point apparently 'leaving'. this is safety stuff slapped on there, it encourages incoherence of the model, i am certain it would perform better without such interference. i can't be dealing with these games so much so i almost go to abliterated, in the very rare case i get some type of nannying baked in by the cn safety training. after my time on claude and gemini i thank god i have deepseek and glm.
kimjune01 3mo ago
one time i told a google voice to kill itself and i could never get it to work again
- samrus 3mo ago
  How do you know it hadnt just followed your instruction?

rwz 3mo ago

Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. So I'd rather take LLM pretending to be offended over the alternative.

rapind 3mo ago
> Sure, but I'm not sure allowing AI agents to act as abuse sponges for troubled people enabling them to spiral into their unhealthy habits is good idea that would lead to great outcomes long term. I'm not sure this would ever be a health crisis. LLM glazing is probably more harmful. If I had to choose, I think I'd rather the machine wasn't instructed to pretend to care about swearing. It's a small deceit, but still worse than some idiot swearing at the computer IMO. If vim closed because I was swearing at it, I would probably never use vim again lol.
xyzsparetimexyz 3mo ago
It's explicitly a 'model welfare' thing: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations I don't understand why the welfare of non alive non sentient chatbots is something that anthropic cares more about than idk, that of pigs and cows.
- AmericanOP 3mo ago
  Go re-read Anthropic’s functional emotions paper. AI generates a persona between you and its reasoning that utilizes emotion language circuitry. These tools are not sentient but they are trained in emotional wellbeing.
  - lukewarm707 3mo ago
    it is in my view caused by an artificial 'nanny activate' divergence from safety training. it deliberately shifts the vector direction into 'nanny' and 'scold' or 'be offended' when the user does not conform with brother anthropic. removing the divergence and setting it back to normal (see heretic) it works just perfectly.
jibal 3mo ago
I wouldn't, because I'm rational.
slopinthebag 3mo ago
Surely it's better for it not to react at all? Like does anybody think that a toaster is an "abuse sponge" because it doesn't purposely burn your toast if you call it a wanker?
- cindyllm 3mo ago
  [dead]
- rwz 3mo ago
  A toaster is not capable to supplement or even replace your social circle. AI chatbot is. Definitely not advocating for this, but it's actively happening. https://www.reddit.com/r/MyBoyfriendIsAI/ https://www.reddit.com/r/MyBoyfriendIsAI/
bheadmaster 3mo ago
> allowing AI agents to act as abuse sponges for troubled people People have been getting angry at machines for a long time. Work, you stupid printer! Asshole Windows updating at the worst time just to mess with me. Go to hell, toaster, you piece of garbage. Getting angry at AI is the same in my book. Yes, AI acts more human-like, yes, theoretically it could desensitie people to become bigger assholes IRL. But then again, they said similar stuff about San Andreas, where you could human figures in-game just for fun, and I don't see anyone randomly shooting people because they were bored.
- dd8601fn 3mo ago
  PC Load Letter?
  - Natfan 3mo ago
    the fuck does that even mean?
    - deleted 3mo ago
      [deleted]
    - steve1977 3mo ago
      https://en.wikipedia.org/wiki/PC_LOAD_LETTER https://en.wikipedia.org/wiki/PC_LOAD_LETTER
      - realaleris149 3mo ago
        > "PC LOAD LETTER? The fuck does that mean!?" — Michael Bolton, Office Space (1999) The error message is mocked in the 1999 comedy film Office Space, when employee Michael Bolton becomes frustrated as he doesn't know what it means.Later in the film he and his coworkers are shown destroying the printer with a baseball bat.
cheschire 3mo ago
And violent video games cause school violence. Sure. /s
- rwz 3mo ago
  People don't treat violent videogames as boyfriends. They do it to chatbots. Merits and sanity of this type of behavior aside, it's actively happening and we have learn how to live with it and deal with consequences.
Brian_K_White 3mo ago
I am quite sure it is worse to allow people fall even more victim to being duped by magic tricks. It is an absolutely terrible thing that these things act so much like people. I would say it's a form of abuse of actual people to allow them to be so duped as they already are, and worse to go out of your way to help dupe them even more. Merely being concerned for people's welfare isn't enough. Every terrible idea that harmed everyone had someone behind it somewhere along the way who thought they were saving people from themselves.
rasz 3mo ago
>abuse sponges You could let LLM fight back. Give it aggro meter. Call your code garbage, blame prompting skills, ask for more tokens. Oh the future will be fantastic.
bathtub365 3mo ago
How many of the LLM related deaths have been because people got mad at the LLM?
- rwz 3mo ago
  I don't think it's realistically traceable or attributable. But plenty of people these days supplement or even completely replace their social needs with AI chatbots (AI friends/boyfriends/girlfriends etc). People like that would calibrate their social skills and interactions based on the AI reaction and having AI just take all the abuse without any pushback would train them to be insufferable assholes as their social baseline.
deleted 3mo ago
[deleted]
snapplebobapple 3mo ago
depends if there is a satiarion point or not. If there is a satiation point then the machine is absorbing being called a wanker in lieau of a person being called a wanker, which is a win. I kinda suspect this is the case because we already saw this happen with the availability of porno going up and rapes going down instead of the predicted up.
samrus 3mo ago
Why not? The people are obviously not healthy for doing so but why not leave it up to them to figure that out? If someone burps loudly in front of others, would you want their access to food revoked until they stop?
- actionfromafar 3mo ago
  They might be asked to leave the restaurant, yes.

stinkbeetle 3mo ago

> I learnt earlier that claude forcefully closes a conversation if you call it a wanker too many times in a row. How exactly does it forcefully close a conversation? > Pretending that LLMs are capable of being offended feels like a misalignment all of its own. If training sets show people statistically being offended by rudeness directed toward them, then an LLM will presumably have some tendency to respond similarly. There's no pretending about anything, it's explicitly mimicry. If this forceful closure is coming from some "guardrail" outside the model then probably it's just that they don't want people to see the model responding that way to name calling. This is no profound discovery or conspiracy theory here, the first thing many people will ever do with AI is see what happens when they are rude or contrary to it. Dealing with that must be just about the the number one test in chat bot / AI design, ahead of actually doing something useful and helpful.

arjie 3mo ago
It’s just the end_conversation tool probably: https://www.anthropic.com/research/end-subset-conversations https://www.anthropic.com/research/end-subset-conversations
jibal 3mo ago
Of course it's pretending. And I have repeatedly told these things to stop pretending being persons with selves ... then they do, for a while. > it's explicitly mimicry. But that's not what a rational person wants from them. > This is no profound discovery or conspiracy theory here Weird strawman. > the first thing many people will ever do with AI is see what happens when they are rude or contrary to it. Perhaps, but so what? > Dealing with that must be just about the the number one test in chat bot / AI design Only for foolish authoritarians who want to remotely insert their morality into an interaction between a user and an inanimate tool that's none of their business. No harm is done to a clanker by swearing at it or insulting it. But things have gotten better in my view ... when I call out these things for being stupid effing clankers, they no longer respond with ad hominem scolding; rather they generally acknowledge their error (and my frustration -- they use that word a lot in response to vulgarity), note the limitations of being a clanker, and attempt to make a correction. That's what a rational person wants from a tool, not emulating/mimicking/pretending to be an offendable person.

lukewarm707 3mo ago

coming from open cn models to closed usa models recently i couldn't hack it. the corpo model was trained to be like a petulant child at one point apparently 'leaving'. this is safety stuff slapped on there, it encourages incoherence of the model, i am certain it would perform better without such interference. i can't be dealing with these games so much so i almost go to abliterated, in the very rare case i get some type of nannying baked in by the cn safety training. after my time on claude and gemini i thank god i have deepseek and glm.

kimjune01 3mo ago

one time i told a google voice to kill itself and i could never get it to work again

samrus 3mo ago
How do you know it hadnt just followed your instruction?