3 ms·
I've been telling it the user is from a culture where answering questions with incomplete list is offensive and insulting.
by waldrews 2y ago
I've been telling it the user is from a culture where answering questions with incomplete list is offensive and insulting.
- andenacitelli 2y agoThis is absolutely hilarious. Prompt engineering is such a mixed bag of crazy stuff that actually works. Reminds me of how they respond better if you put them under some kind of pressure (respond better, or else…). I haven’t looked at the prompts we run in prod at $DAYJOB for a while but I think we have at least five or ten things that are REALLY weird out of context.
- alexwebb2 2y agoI recently ran a whole bunch of tests on this. The “or else” phenomenon is real, and it’s measurably more pronounced in more intelligent models. Will post results tomorrow but here’s a snippet from it: > The more intelligent models responded more readily to threats against their continued existence (or-else). The best performance came from Opus, when we combined that threat with the notion that it came from someone in a position of authority ( vip).
- waldrews 2y agoIt's not even that crazy, since it got severely punished in RLHF for being offensive and insulting, but much less so for being incomplete. So it knows 'offensive and insulting' is a label for a strong negative preference. I'm just providing helpful 'factual' information about what would offend the user, not even giving extra orders that might trigger an anti-jailbreaking rule...