3 ms·

Yo it was an engagement pattern openAI found specifically grew subscriptions and conversation length. It’s a dark pattern for sure.

• Legend2440 11mo ago
  It doesn’t appear that anyone at OpenAI sat down and thought “let’s make our model more sycophantic so that people engage with it more”. Instead it emerged automatically from RLHF, because users rated agreeable responses more highly.
  • astrange 11mo ago
    Not precisely RLHF, probably a policy model trained on user responses. RL works on responses from the model you're training, which is not the one you have in production. It can't directly use responses from previous models.
  • tsunamifury 11mo ago
    I can tell you’ve never worked in big tech before. Dark patterns are often “discovered” and very consciously not shut off because the reverse cost would be too high to stomach. Esp in a delicate growth situation. See Facebook at its adverse mental health studies