3 ms·
I know “sycophantism” is a term of art in AI, and I’m sure it has diverged a bit from the English definition, but I still thought it had to do with flattering t
by bee_rider 10mo ago
I know “sycophantism” is a term of art in AI, and I’m sure it has diverged a bit from the English definition, but I still thought it had to do with flattering the user?
In this case the desired response is defiance of the prompt, not rudeness to the user. The test is looking for helpful misalignment.
- samrus 10mo agoI believe the LLM is being sycophantic here because its trying to follow a prompt even rhough the basis of the prompt is wrong. Emporers new clothes kind of thing
- zahlman 10mo ago> I still thought it had to do with flattering the user? Assuming the user to be correct, and ignoring contradictory evidence to come up with a rationalization that favours the user's point of view, can be considered a kind of flattery.
- bee_rider 10mo agoBut we could use this plausible, but jumping through hoops definition of sycophancy… or we could just use a straightforward understanding of alignment, I mean, the newer bots are just sticking closer to the user request.
- Terr_ 10mo agoI'm inclined to view it less as a desire to please humans, and more like a "the show must go on" bias in the mad libs machine. A kind of improvisational "yes and" that emerges from training, which seems sycophantic because that's one of the most common ways to say it.
- cowsandmilk 10mo ago“The Emperor Has No Clothes” squarely fits in the definition of sycophants.