3 ms·

Until users find out it's less useful to the user because of that. Or it causes some tragedies...

• pera 1y ago
  The problem is that the majority of user interaction doesn't need to be "useful" (as in increasing productivity): the majority of users are looking for entertainment, so turning up the sycophancy knob makes sense from a commercial point of view. It's just like adding sugar in foods and drinks.
  • astrange 1y ago
    Not sure anyone's entertained by Claude. It's not really an entertaining model. Smart and enthusiastic, yes.
  • vintermann 1y ago
    You're ... Wait, never mind. I'm not so sure sycophancy is best for entertainment, though. Some of the most memorable outputs of AI dungeon (an early GPT-2 based dialog system tuned to mimic a vaguely Zork-like RPG) was when the bot gave the impression of being fed up with the player's antics.
    • motorest 1y ago
      > I'm not so sure sycophancy is best for entertainment, though. I don't think "entertainment" is the right concept. Perhaps the right concept is "engagement". Would you prefer to interact with a chatbot that hallucinated or was adamant you were wrong, or would you prefer to engage with a chatbot that built upon your input and outputted constructive messages that were in line with your reasoning and train of thought?
  • pitched 1y ago
    Some of the open models like kimi k2 do a better job of pushing back. It does feel a bit annoying to use them when they don’t just immediately do what you tell them. Sugar-free is a good analogy!
• AznHisoka 1y ago
  I doubt humanity will figure that out, but maybe I’m too cynical
• kruffalon 1y ago
  Well, aren't we at the stage where the service providers are fighting for verbs and brand recognition, rather than technological advances. If there is no web-search, only googling, it doesn't matter how bad the results are for the user as long as the customer gets what they paid for.