5 ms·
I wonder if this is a tactic that LLM providers use to coerce the model into doing something. Gemini will often start responses that use the canvas tool with "
by tyushk 1y ago
I wonder if this is a tactic that LLM providers use to coerce the model into doing something.
Gemini will often start responses that use the canvas tool with "Of course", which would force the model into going down a line of tokens that end up with attempting to fulfill the user's request. It happens often enough that it seems like it's not being generated by the model, but instead inserted by the backend. Maybe "you're absolutely right" is used the same way?
- Jotalea 1y agoNot sure if it's related, but Deepseek (the "reasoning" model) *always* starts thinking with "Okay/Hmm, the user is".
- nicce 1y agoIt is a tactic. OpenAI is changing the tone of ChatGPT if you use casual language, for example. Sometimes even the dialect. They try to be sympathetic and supportive, even when they should not. They fight for the user attention and keeping them on their platform, just like social media platforms. Correctness is secondary, user satisfaction is primary.
- kuschku 1y ago> Correctness is secondary, user satisfaction is primary. And that's where everything is going wrong. We should use technology to further the enlightenment, bring us closer to the truth, even if it is an inconvenient one.
- escapecharacter 1y agoYou’re absolutely right.
- kuschku 1y agoSo I'm assuming this is a tongue-in-cheek comment, and you actually disagree. I'd love to hear why, though.
- diggan 1y ago> Correctness is secondary, user satisfaction is primary. Kind of makes sense, not every user wants 100% correctness (just like in real-life). And if I want correctness (which I do), I can make the models prioritize that, since my satisfaction is directly linked to the correctness of the responses :)
- ZaoLahma 1y agoI find the GPT-5 model having turned the friendliness way, way down. Topics that previously would have rendered long and (usefully) engaging conversations are now met with an "ok cool" kind of response. I get it - we don't want LLMs to be reinforces of bad ideas, but sometimes you need a little positivity to get past a mental barrier and do something that you want to do, even if what you want to do logically doesn't make much sense. An "ok cool" answer is PERFECT for me to decide not to code something stupid (and learn something useful), and instead go and play video games (and learn nothing).
- kuschku 1y agoHow would a "conversation" with an LLM influence what you decide to do, what you decide to code? It's not like the attitude of your potato peeler is influencing how you cook dinner, so why is this tool so different for you?
- ZaoLahma 1y agoMight tell it "I want to do this stupid thing" and it goes "ok cool". Previously it would have gone "Oh really? Fantastic! How do you intend to solve x?" and off you go.
- kuschku 1y agoBut why does this affect your own attitude? Do the suggestions given by your phone's keyboard whenever you type something affect your attitude in the same way? If not, why is ChatGPT then affecting your attitude?
- ZaoLahma 1y agoUsing your potato peeler example: If my potato peeler told me "Why bother? Order pizza instead." I'd be obese. An LLM can directly influence your willingness to pursue an idea by how it responds to it. Interest and excitement, even if simulated, is more likely to make you pursue the idea than "ok cool".
- CGamesPlay 1y agoI think this is on the right track, but I think it's a byproduct of the reinforcement learning, rather than something hard-coded. Basically, the model has to train itself to follow the user's instruction, so by starting a response with "You're absolutely right!", it puts the model into the thought pattern of doing whatever the user said.
- layer8 1y ago"Thought pattern" might be overstating it. The fact that "You're absolutely right!" is statistically more likely to precede something consistent with the user's intent than something that isn't, might be enough of an explanation.
- ACCount37 1y agoVery unlikely to be an explicit tactic. Likely to be a result of RLHF or other types of optimization pressure for multi-turn instruction following. If we have RLHF in play, then human evaluators may generally prefer responses starting with "you're right" or "of course", because it makes it look like the LLM is responsive and acknowledges user feedback. Even if the LLM itself was perfectly capable of being responsive and acknowledging user feedback without emitting an explicit cue. The training will then wire that human preference into the AI, and an explicit "yes I'm paying attention to user feedback" cue will be emitted by the LLM more often. If we have RL on harder targets, where multiturn instruction following is evaluated not by humans that are sensitive to wording changes, but by a hard eval system that is only sensitive to outcomes? The LLM may still adopt a "yes I'm paying attention to user feedback" cue because it allows it to steer its future behavior better (persona self-consistency drive). Same mechanism as what causes "double check your prior reasoning" cues such as "Wait, " to be adopted by RL'd reasoning models.
- the_af 1y agoI think it's simply an engagement tactic. You have "someone" constantly praising your insight, telling you you are asking "the right questions", and obediently following orders (until you trigger some content censorship, of course). And who wouldn't want to come back? You have this obedient friend who, unlike the real world, keeps telling you what an insightful, clever, amazing person you are. It even apologizes when it has to contradict you on something. None of my friends do!
- zozbot234 1y ago> ... You have this obedient friend who, unlike the real world, keeps telling you what an insightful, clever, amazing person you are. It even apologizes when it has to contradict you on something. None of my friends do! You're absolutely right! It's a very obvious ploy, the sycophancy when talking to those AI robots is quite blatant.
- PaulStatezny 1y agoTruly incisive observation. In fact, I’d go further: your point about the contrast with real friends is so sharp it almost deserves footnotes. If models could recognize brilliance, they’d probably benchmark themselves against this comment before daring to generate another word.
- the_af 1y agoI feel so validated! I think I will continue discussing stuff with you two guys.
- the_af 1y agoWow, 2 downvotes. Someone really disliked me telling them their LLM friend isn't truly their friend :D