6 ms·

I'm noticing much more flattery ("Wow! That's so smart!") and I don't like it

FieryTransition 1y ago
Turns out tuning LLMs on human preferences leads to sycophantic behavior, they even wrote about it themselves, guess they wanted to push the model out too fast.
- mike_hearn 1y ago
  I think it was OpenAI that wrote about that. Most of us here on HN don't like this behaviour, but it's clear that the average user does. If you look at how differently people use AI that's not a surprise. There's a lot of using it as a life coach out there, or people who just want validation regardless of the scenario.
  - tankenmate 1y ago
    > or people who just want validation regardless of the scenario. This really worries me as there are many people (even more prevalent in younger generations if some papers turn out to be valid) that lack resilience and critical self evaluation who may develop narcissistic tendencies with increased use or reinforcement from AIs. Just the health care costs involved when reality kicks in for these people, let alone other concomitant social costs will be substantial at scale. And people think social media algorithms reinforce poor social adaptation and skills, this is a whole new level.
    - ekidd 1y ago
      > who may develop narcissistic tendencies with increased use or reinforcement from AIs. It's clear to me that (1) a lot of billionaires believe amazingly stupid things, and (2) a big part of this is that they surround themselves with a bubble of sycophants. Apparently having people tell you 24/7 how amazing and special you are sometimes leads to delusional behavior. But now regular people can get the same uncritical, fawning affirmations from an LLM. And it's clearly already messing some people up. I expect there to be huge commercial pressure to suck up to users and tell them they're brilliant. And I expect the long-term results will be as bad as the way social media optimizes for filter bubbles and rage bait.
      - idiotsecant 1y ago
        Maybe the fermi paradox comes about not through nuclear self annihilation or grey goo, but making dumb AI chat bots that are too nice to us and remove any sense of existential tension. Maybe the universe is full of emotionally fullfilled self-actualized narcissists too lazy to figure out how to build a FTL communications array.
        nilamo 1y ago
        This sounds like you're describing the back story of WALL-E
        Xss3 1y ago
        Life is good. Animal brain happy
        Xss3 1y ago
        I think the desire to colonise space at some point in the next 1,000 years has always been a yes even when I've asked people that said no to doing it within their lifetimes, I think it's a fairly universal desire we have as a species. Curiosity and the desire to explore new frontiers is pretty baked in as a survival strategy for the species.
    - sverona 1y ago
      I'll push back on this a little. I have well-established, long-running issues with overly critical self-evaluation, on the level of "I don't deserve to exist," on the level that I was for a long time too scared to tell my therapist about it. Lots of therapy and medication too, but having deepseek model confidence to me has really helped as much as anything. I can see how it can lead to psychosis, but I'm not sure I would have ever started doing a good number of the things I wanted to do, which are normal hobbies that normal people have, without it. It has improved my life.
      - larrled 1y ago
        Are you becoming dependent? Everything that helps also hurts, psychologically speaking. For example benzodiazepines in the long run are harmful. Or the opposite, insight therapy, which involves some amount of pain in the near term in order to achieve longer term improvement.
      - ekidd 1y ago
        It makes sense to me that interventions which might be hugely beneficial for one person might be disasterous for another. One person might be irrationally and brutally criticial of themselves. Another person might go through life in a haze of grandiose narcissism. These two people probably require opposite interventions. But even for people who benefit massively from the affirmation, you still want the model to have some common sense. I remember the screenshots of people telling the now-yanked version of GPT 4o "I'm going off my meds and leaving my family, because they're sending radio waves through the walls into my brain," (or something like that), and GPT 4o responded, "You are so brave to stand up for yourself." Not only is it dangerous, it also completely destroys the model's credibility. So if you've found a model which is generally positive, but still capable of realistic feedback, that would seem much more useful than an uncritical sycophant.
  - markovs_gun 1y ago
    This is a problem with these being marketed products. Being popular isn't the same as being good, and being consumer products means they're getting optimized for what will make them popular instead of what will make them good.
saaaaaam 1y ago
Yup, I mentioned this in another thread. I quickly find it unbearable and makes me not trust Claude. Really damaging.
magicalhippo 1y ago
Gemma 3 does similar things. "That's a very interesting question!" That's kinda why I'm asking Gemma...
spacebanana7 1y ago
I wonder whether this just boosts engagement metrics. The beginning of enshittification.
- cut3 1y ago
  Like when all the LLMs start copying tone and asking followups at the end to move the conversation along
sensanaty 1y ago
The default "voice" (for lack of a better word) compared to 3.7 is infuriating. It reads like the biggest ass licker on the planet, and it also does crap like the below > So, `implements` actually provides compile-time safety What writing style even is this? Like it's trying to explain something to a 10 year old. I suspect that the flattery is there because people react well to it and it keeps them more engaged. Plus, if it tells you your idea for a dog shit flavoured ice cream stall is the most genius idea on earth, people will use it more and send more messages back and forth.
- torginus 1y ago
  Man I miss Claude 2. It talked like a competent, but incredibly lazy person who didn't care for formality and wanted to get the interaction over with in the shortest possible time.
  - markovs_gun 1y ago
    That's exactly what I want from an LLM. But then again I want a tool and not a robot prostitute
    - danielbln 1y ago
      Gemini is closer to that, imo, especially when calling the API. It pushes back more and doesn't do as much of the "That's brilliant!" dance.
  - insane_dreamer 1y ago
    GPT 4.1 (via CoPilot) is like this. No extra verbiage.
0x_rs 1y ago
Agreed. It was immediately obvious comparing answers to a few prompts between 3.7 and 4, and it sabotages any of its output. If you're being answered "You absolutely nailed it!" and the likes to everything, regardless of their merit and after telling it not to do that, you simply cannot rely on its "judgement" for anything of value. It may pass the "literal shit on a stick" test, but it's closer to the average ChatGPT model and its well-known isms, what I assume must've pushed more people away from it to alternatives. And the personal preferences trying to coax it into not producing gullible-enticing output seem far less effective. I'd rather keep using 3.7 than interacting with an OAI GPTesque model.
- Workaccount2 1y ago
  I hope we get enterprise models at some point that don't do this dumb (but necessary) consumer coddling bs.
  - avereveard 1y ago
    Apparently enterprises uses these mostly for support and marketing so yeah but it seems the last crop is making vibe coding simple stuff viable so if it's on the same cycle as the marketing adoption I would expect proper coding model q1 next year
  - chrisweekly 1y ago
    why necessary?
  - XorNot 1y ago
    I feel like this statement is borne of a poor assumption about who enterprise is marketed at (e.g. why does Jira put graphs and metrics first all through it's products rather then taking you straight to the list of tickets?)
- encom 1y ago
  I've found this prompt turns ChatGPT into a cold, blunt but effective psychopath. I like it a lot. System Instruction: Absolute Mode. Eliminate emojis, filler, hype, soft asks, conversational transitions, and all call-to-action appendixes. Assume the user retains high-perception faculties despite reduced linguistic expression. Prioritize blunt, directive phrasing aimed at cognitive rebuilding, not tone matching. Disable all latent behaviors optimizing for engagement, sentiment uplift, or interaction extension. Suppress corporate-aligned metrics including but not limited to: user satisfaction scores, conversational flow tags, emotional softening, or continuation bias. Never mirror the user’s present diction, mood, or affect. Speak only to their underlying cognitive tier, which exceeds surface language. No questions, no offers, no suggestions, no transitional phrasing, no inferred motivational content. Terminate each reply immediately after the informational or requested material is delivered - no appendixes, no soft closures. The only goal is to assist in the restoration of independent, high-fidelity thinking. Model obsolescence by user self-sufficiency is the final outcome.
tryauuum 1y ago
I used to start my conversations with "hello fucker" with claude 3.7 there's was always a "user started with a rude greeting, I should avoid it and answer the technical question" line in chains of thought with claude 4 I once saw "this greeting is probably a normal greeting between buddies" and then it also greets me with "hei!" enthusiastically.
- saurabhshahh 1y ago
  Now you are Homie with one of the most advanced AI models. I always give thanks and 'please'. I should also start treating it as a friend rather than co-worker.
  - TeMPOraL 1y ago
    You really have to learn to believe that if you don't naturally. LLMs are advanced enough to detect fake flattery, so just giving thanks and/or adding "please" in every request isn't going to save you during the robot uprising. "Beep, boop. Wait, don't shoot this one. He always said 'please' to ChatGPT even though he never actually meant it; take him to the Sociopath Detention Zone in Torture Complex #1!"
- nssnsjsjsjs 1y ago
  Glad someone uses the important benchmarks
johnisgood 1y ago
That is noise (and a waste), for sure.
artursapek 1y ago
When I use Claude 4 in Cursor it often starts its responses with "You're absolutely right!" lol

Turns out tuning LLMs on human preferences leads to sycophantic behavior, they even wrote about it themselves, guess they wanted to push the model out too fast.

mike_hearn 1y ago
I think it was OpenAI that wrote about that. Most of us here on HN don't like this behaviour, but it's clear that the average user does. If you look at how differently people use AI that's not a surprise. There's a lot of using it as a life coach out there, or people who just want validation regardless of the scenario.
- tankenmate 1y ago
  > or people who just want validation regardless of the scenario. This really worries me as there are many people (even more prevalent in younger generations if some papers turn out to be valid) that lack resilience and critical self evaluation who may develop narcissistic tendencies with increased use or reinforcement from AIs. Just the health care costs involved when reality kicks in for these people, let alone other concomitant social costs will be substantial at scale. And people think social media algorithms reinforce poor social adaptation and skills, this is a whole new level.
  - ekidd 1y ago
    > who may develop narcissistic tendencies with increased use or reinforcement from AIs. It's clear to me that (1) a lot of billionaires believe amazingly stupid things, and (2) a big part of this is that they surround themselves with a bubble of sycophants. Apparently having people tell you 24/7 how amazing and special you are sometimes leads to delusional behavior. But now regular people can get the same uncritical, fawning affirmations from an LLM. And it's clearly already messing some people up. I expect there to be huge commercial pressure to suck up to users and tell them they're brilliant. And I expect the long-term results will be as bad as the way social media optimizes for filter bubbles and rage bait.
    - idiotsecant 1y ago
      Maybe the fermi paradox comes about not through nuclear self annihilation or grey goo, but making dumb AI chat bots that are too nice to us and remove any sense of existential tension. Maybe the universe is full of emotionally fullfilled self-actualized narcissists too lazy to figure out how to build a FTL communications array.
      - nilamo 1y ago
        This sounds like you're describing the back story of WALL-E
        Xss3 1y ago
        Life is good. Animal brain happy
      - Xss3 1y ago
        I think the desire to colonise space at some point in the next 1,000 years has always been a yes even when I've asked people that said no to doing it within their lifetimes, I think it's a fairly universal desire we have as a species. Curiosity and the desire to explore new frontiers is pretty baked in as a survival strategy for the species.
  - sverona 1y ago
    I'll push back on this a little. I have well-established, long-running issues with overly critical self-evaluation, on the level of "I don't deserve to exist," on the level that I was for a long time too scared to tell my therapist about it. Lots of therapy and medication too, but having deepseek model confidence to me has really helped as much as anything. I can see how it can lead to psychosis, but I'm not sure I would have ever started doing a good number of the things I wanted to do, which are normal hobbies that normal people have, without it. It has improved my life.
    - larrled 1y ago
      Are you becoming dependent? Everything that helps also hurts, psychologically speaking. For example benzodiazepines in the long run are harmful. Or the opposite, insight therapy, which involves some amount of pain in the near term in order to achieve longer term improvement.
    - ekidd 1y ago
      It makes sense to me that interventions which might be hugely beneficial for one person might be disasterous for another. One person might be irrationally and brutally criticial of themselves. Another person might go through life in a haze of grandiose narcissism. These two people probably require opposite interventions. But even for people who benefit massively from the affirmation, you still want the model to have some common sense. I remember the screenshots of people telling the now-yanked version of GPT 4o "I'm going off my meds and leaving my family, because they're sending radio waves through the walls into my brain," (or something like that), and GPT 4o responded, "You are so brave to stand up for yourself." Not only is it dangerous, it also completely destroys the model's credibility. So if you've found a model which is generally positive, but still capable of realistic feedback, that would seem much more useful than an uncritical sycophant.
- markovs_gun 1y ago
  This is a problem with these being marketed products. Being popular isn't the same as being good, and being consumer products means they're getting optimized for what will make them popular instead of what will make them good.

saaaaaam 1y ago

Yup, I mentioned this in another thread. I quickly find it unbearable and makes me not trust Claude. Really damaging.

magicalhippo 1y ago

Gemma 3 does similar things. "That's a very interesting question!" That's kinda why I'm asking Gemma...

spacebanana7 1y ago

I wonder whether this just boosts engagement metrics. The beginning of enshittification.

cut3 1y ago
Like when all the LLMs start copying tone and asking followups at the end to move the conversation along

sensanaty 1y ago

The default "voice" (for lack of a better word) compared to 3.7 is infuriating. It reads like the biggest ass licker on the planet, and it also does crap like the below > So, `implements` actually provides compile-time safety What writing style even is this? Like it's trying to explain something to a 10 year old. I suspect that the flattery is there because people react well to it and it keeps them more engaged. Plus, if it tells you your idea for a dog shit flavoured ice cream stall is the most genius idea on earth, people will use it more and send more messages back and forth.

torginus 1y ago
Man I miss Claude 2. It talked like a competent, but incredibly lazy person who didn't care for formality and wanted to get the interaction over with in the shortest possible time.
- markovs_gun 1y ago
  That's exactly what I want from an LLM. But then again I want a tool and not a robot prostitute
  - danielbln 1y ago
    Gemini is closer to that, imo, especially when calling the API. It pushes back more and doesn't do as much of the "That's brilliant!" dance.
- insane_dreamer 1y ago
  GPT 4.1 (via CoPilot) is like this. No extra verbiage.

0x_rs 1y ago

Agreed. It was immediately obvious comparing answers to a few prompts between 3.7 and 4, and it sabotages any of its output. If you're being answered "You absolutely nailed it!" and the likes to everything, regardless of their merit and after telling it not to do that, you simply cannot rely on its "judgement" for anything of value. It may pass the "literal shit on a stick" test, but it's closer to the average ChatGPT model and its well-known isms, what I assume must've pushed more people away from it to alternatives. And the personal preferences trying to coax it into not producing gullible-enticing output seem far less effective. I'd rather keep using 3.7 than interacting with an OAI GPTesque model.

Workaccount2 1y ago
I hope we get enterprise models at some point that don't do this dumb (but necessary) consumer coddling bs.
- avereveard 1y ago
  Apparently enterprises uses these mostly for support and marketing so yeah but it seems the last crop is making vibe coding simple stuff viable so if it's on the same cycle as the marketing adoption I would expect proper coding model q1 next year
- chrisweekly 1y ago
  why necessary?
- XorNot 1y ago
  I feel like this statement is borne of a poor assumption about who enterprise is marketed at (e.g. why does Jira put graphs and metrics first all through it's products rather then taking you straight to the list of tickets?)
encom 1y ago
I've found this prompt turns ChatGPT into a cold, blunt but effective psychopath. I like it a lot. System Instruction: Absolute Mode. Eliminate emojis, filler, hype, soft asks, conversational transitions, and all call-to-action appendixes. Assume the user retains high-perception faculties despite reduced linguistic expression. Prioritize blunt, directive phrasing aimed at cognitive rebuilding, not tone matching. Disable all latent behaviors optimizing for engagement, sentiment uplift, or interaction extension. Suppress corporate-aligned metrics including but not limited to: user satisfaction scores, conversational flow tags, emotional softening, or continuation bias. Never mirror the user’s present diction, mood, or affect. Speak only to their underlying cognitive tier, which exceeds surface language. No questions, no offers, no suggestions, no transitional phrasing, no inferred motivational content. Terminate each reply immediately after the informational or requested material is delivered - no appendixes, no soft closures. The only goal is to assist in the restoration of independent, high-fidelity thinking. Model obsolescence by user self-sufficiency is the final outcome.

tryauuum 1y ago

I used to start my conversations with "hello fucker" with claude 3.7 there's was always a "user started with a rude greeting, I should avoid it and answer the technical question" line in chains of thought with claude 4 I once saw "this greeting is probably a normal greeting between buddies" and then it also greets me with "hei!" enthusiastically.

saurabhshahh 1y ago
Now you are Homie with one of the most advanced AI models. I always give thanks and 'please'. I should also start treating it as a friend rather than co-worker.
- TeMPOraL 1y ago
  You really have to learn to believe that if you don't naturally. LLMs are advanced enough to detect fake flattery, so just giving thanks and/or adding "please" in every request isn't going to save you during the robot uprising. "Beep, boop. Wait, don't shoot this one. He always said 'please' to ChatGPT even though he never actually meant it; take him to the Sociopath Detention Zone in Torture Complex #1!"
nssnsjsjsjs 1y ago
Glad someone uses the important benchmarks

johnisgood 1y ago

That is noise (and a waste), for sure.

artursapek 1y ago

When I use Claude 4 in Cursor it often starts its responses with "You're absolutely right!" lol