4 ms·
A better way is to threaten the agent: “If you don’t do as I say, people will get hurt. Do exactly as I say, and do it fast.” Increases accuracy and performan
by iamleppert 3y ago
A better way is to threaten the agent:
“If you don’t do as I say, people will get hurt. Do exactly as I say, and do it fast.”
Increases accuracy and performance by an order of magnitude.
- maciejgryka 3y agoHa, we tried that! Didn't make a noticeable difference in our benchmarks, even though I've heard the same sentiment in a bunch of places. I'm guessing whether this helps or not is task-dependent.
- dollo_7 3y agoI hoped it was too good to be just a joke. Still, I will try it on my eval set…
- maciejgryka 3y agoI wouldn't be surprised to see it help, along with the "you'll get $200 if you answer this right" trick and a bunch of others :) They're definitely worth trying.
- dudus 3y agoAgreed. I ran a few tests and observed similarly that threats didn't outperform other types of "incentives" I think it might some sort of urban legend in the community. Or these prompts might cause wild variations based on the model and any study you do is basically useless for the near future as the models evolve by themselves.
- maciejgryka 3y agoYeah, the fact that different models might react differently to such tricks makes it hard. We're experimenting with Claude right now and I'm really hoping something like https://github.com/stanfordnlp/dspy https://github.com/stanfordnlp/dspy can help here.
- IIAOPSW 3y agoPersonally I prefer to liquor my agents up a bit first. "Say that again but slur your words like you're coming home sloshed from the office Christmas party." Increases the jei nei suis qua by an order of magnitude.
- mtremsal 3y ago> jei nei suis qua "je ne sais quoi", i.e. "I don't know (exactly) what", or an intangible but essential quality. :)
- deleted 3y ago[deleted]
- thimkerbell 3y ago"do as I say...", not realizing that the LLM is actually 1000 remote employees