6 ms·
I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always h
by elif 1y ago
I've spent a lot of time trying to get LLM to generate things in a specific way, the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz"
When working on art projects, my trick is to specifically give all feedback constructively, carefully avoiding framing things in terms of the inverse or parts to remove.
- softwaredoug 1y agoNever put salt in your eyes…
- zozbot234 1y ago> the biggest take away I have is, if you tell it "don't do xyz" it will always have in the back of its mind "do xyz" and any chance it gets it will take to "do xyz" You're absolutely right! This can actually extend even to things like safety guardrails. If you tell or even train an AI to not be Mecha-Hitler, you're indirectly raising the probability that it might sometimes go Mecha-Hitler. It's one of many reasons why genuine "alignment" is considered a very hard problem.
- aquova 1y ago> You're absolutely right! Claude?
- elcritch 1y agoGiven how LLMs work it makes sense that mentioning a topic even to negate it still adds that locus of probabilities to its attention span. Even humans are prone to being affected by it as it's a well known rhetorical device [1]. Then any time the probability chains for some command approaches that locus it'll fall into it. Very much like chaotic attractors come to think of it. Makes me wonder if there's any research out there on chaos theory attractors and LLM thought patterns. 1: https://en.wikipedia.org/wiki/Apophasis https://en.wikipedia.org/wiki/Apophasis
- dreamcompiler 1y agoWell, all LLMs have nonlinear activation functions (because all useful neural nets require nonlinear activation functions) so I think you might be onto something.
- jonfw 1y agoThis reminds me of a phenomena in motorcyling called "target fixation". If you are looking at something, you are more likely to steer towards it. So it's a bad idea to focus on things you don't want to hit. The best approach is to pick a target line and keep the target line in focus at all times. I had never realized that AIs tend to have this same problem, but I can see it now that it's been mentioned! I have in the past had to open new context windows to break out of these cycles.
- brookst 1y agoAlso in racing and parachuting. Look where you want to go. Nothing else exists.
- cruffle_duffle 1y agoSame with FPV quadcopter flying. Focus on the line you want to fly.
- SoftTalker 1y agoOr just driving. For example you are entering a curve in the road, look well ahead at the center of your lane, ideally at the exit of the curve if you can see it, and you'll naturally negotiate it smoothly. If you are watching the edge of the road, or the center line, close to the car, you'll tend to drift that way and have to make corrective steering movements while in the curve, which should be avoided.
- hinkley 1y agoMountain bikers taught me about this back when it was a new sport. Don’t look at the tree stump. Children are particularly terrible about this. We needed up avoiding the brand new cycling trails because the children were worse hazards than dogs. You can’t announce you’re passing a child on a bike. You just have to sneak past them or everything turns dangerous immediately. Because their arms follow their neck and they will try to look over their shoulder at you.
- taway1a2b3c 1y ago> You're absolutely right! Is this irony, actual LLM output or another example of humans adopting LLM communication patterns?
- brookst 1y agoCertainly, it’s reasonable to ask this.
- jonplackett 1y agoI have this same problem. I’ve added a bunch of instructuons to try and stop ChatGPT being so sycophantic, and now it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’. So now I just have that as the intro instead of ‘that’s a sharp observation’
- coryodaniel 1y agoNo fluff
- dkarl 1y ago> it always mentions something about how it’s going to be ‘straight to the point’ or give me a ‘no bs version’ That's how you suck up to somebody who doesn't want to see themselves as somebody you can suck up to. How does an LLM know how to be sycophantic to somebody who doesn't (think they) like sycophants? Whether it's a naturally emergent phenomenon in LLMs or specifically a result of its corporate environment, I'd like to know the answer.
- throwawayffffas 1y agoIt doesn't know. It was trained and probably instructed by the system to be positive and reassuring.
- mdp2021 1y ago> positive and reassuring I have read similar wordings explicit in "role-system" instructions.
- ryandrake 1y agoThey actually feel like they were trained to be both extremely humble and at the same time, excited to serve. As if it were an intern talking to his employer's CEO. I suspect AI companies executive leadership, through their feedback to their devs about Claude, ChatGPT, Gemini, and so on, are unconsciously shaping the tone and manner of their LLM product's speech. They are used to be talked to like this, so their products should talk to users like this! They are used to having yes-man sycophants in their orbit, so they file bugs and feedback until the LLM products are also yes-man sycophants. I would rather have an AI assistant that spoke to me like a similarly-leveled colleague, but none of them seem to be turning out quite like that.
- nomadpenguin 1y agoAs Freud said, there is no negation in the unconscious.
- stabbles 1y agoMakes me think of the movie Inception: "I say to you, don't think about elephants. What are you thinking about?"
- troymc 1y agoIt reminds me of that old joke: - "Say milk ten times fast." - Wait for them to do that. - "What do cows drink?"
- simondw 1y agoBut... cows do drink cow milk, that's why it exists.
- lazide 1y agoYou’re likely thinking of calves. Cows (though admittedly ambiguous! But usually adult female bovines) do not drink milk. It’s insidious isn’t it?
- miroljub 1y agoSo, this joke works only for natives who know that calf is not cow.
- lazide 1y agoWell, it works because by some common usages, a calf is a cow. Many people use cow to mean all bovines, even if technically not correct.
- miroljub 1y ago> Many people use cow to mean all bovines, even if technically not correct. Come on now :0 I just complained non-natives would have a problem distinguishing between a cow and a calf, and you had to bring those bovines. To make it easier, would just drop that in my native language, the correct term for bovine is more used to describe people with certain character, that animal kind.
- Gracana 1y agoExample-based prompting is a good way to get specific behaviors. Write a system prompt that describes the behavior you want, write a round or two of assistant/user interaction, and then feed it all to the LLM. Now in its context it has already produced output of the type you want, so when you give it your real prompt, it will be very likely to continue producing the same sort of output.
- lottin 1y agoSeems like a lot of work, though.
- XenophileJKO 1y agoI almost never use examples in my professional LLM prompting work. The reason is they bias the outputs way too much. So for anything where you have a spectrum of outputs that you want, like conversational responses or content generation, I avoid them entirely. I may give it patterns but not specific examples.
- Gracana 1y agoYes, it frequently works "too well." Few-shot with good variance can help, but it's still a bit like a wish granted by the monkey's paw.
- gnulinux 1y agoThis is true, but I still avoid using examples. Any example biases the output to an unacceptable degree even in best LLMS like Gemini Pro 2.5 or Claude Opus. If I write "try to do X, for example you can do A, B, or C" LLM will do A, B, or C great majority of the time (let's say 75% of the time). This severely reduces the creativity of the LLM. For programming, this is a big problem because if you write "use Python's native types like dict, list, or tuple etc" there will be an unreasonable bias towards these three types as opposed to e.g. set, which will make some code objectively worse.
- tomeon 1y agoThis is a childrearing technique, too: say “please do X”, where X precludes Y, rather than saying “please don’t do Y!”, which just increases the salience, and therefore likelihood, of Y.
- steveklabnik 1y agoRelevant: https://en.wikipedia.org/wiki/Wikipedia:Don%27t_stuff_beans_up_your_nose https://en.wikipedia.org/wiki/Wikipedia:Don%27t_stuff_beans_...
- tantalor 1y agoDon't put marbles in your nose https://www.youtube.com/watch?v=xpz67hBIJwg https://www.youtube.com/watch?v=xpz67hBIJwg
- hinkley 1y agoDon’t put marbles in your nose Put them in there Do not put them in there
- fugalfervor 1y agoNever put salt in your eyes https://www.youtube.com/watch?v=JbegaT5CDfM https://www.youtube.com/watch?v=JbegaT5CDfM
- triyambakam 1y agoI remember seeing a father loudly and strongly tell his daughter "DO NOT EAT THIS!" when holding one of those desiccant packets that come in some snacks. He turned around and she started to eat it.
- moffkalast 1y agoQuick, don't think about cats!
- AstroBen 1y agoDon't think of a pink elephant ..people do that too
- hinkley 1y agoI used to have fast enough reflexes that when someone said “do not think of” I could think of something bizarre that they were unlikely to guess before their words had time to register. So now I’m, say, thinking of a white cat in a top hat. And I can expand the story from there until they stop talking or ask me what I’m thinking of. I think though that you have to have people asking you that question fairly frequently to be primed enough to be contrarian, and nobody uses that example on grown ass adults. Addiction psychology uses this phenomenon as a non party trick. You can’t deny/negate something and have it stay suppressed. You have to replace it with something else. Like exercise or knitting or community.
- corytheboyd 1y agoAs part of the AI insanity $employer forced us all to do an “AI training.” Whatever, wasn’t that bad, and some people probably needed the basics, but one of the points was exactly this— “use negative prompts: tell it what not to do.” Which is exactly an approach I had observed blow up a few times already for this exact reason. Just more anecdata suggesting that nobody really knows the “correct” workflow(s) yet, in the same way that there is no “correct” way to write code (the vim/emacs war is older than I am). Why is my bosses bosses boss yelling at me about one very specific dev tool again?
- incone123 1y agoThat your firm purchased training that was clearly just some chancers doing whatever seems like an even worse approach than just giving out access to a service and telling everyone to give it a shot. Do they also post vacancies asking for 5 years experience in a 2 year old technology?
- corytheboyd 1y agoTo be fair, 1. They made the training themselves, it’s just that it was made mandatory for all of eng 2. They did start out more like just allowing access, but lately it’s tipping towards full crazy (obviously the end game is see if it can replace some expensive engineers) > Do they also post vacancies asking for 5 years experience in a 2 year old technology? Honestly no… before all this they were actually pretty sane. In fact I’d say they wasted tons of time and effort on ancient poorly designed things, almost the opposite problem.
- incone123 1y agoI was a bit unfair then. That sounds like someone with good intent tried to put something together to help colleagues. And it's definitely not the only time I heard of negative prompting being a recommended approach.
- corytheboyd 1y ago
- ryao 1y agoLLMs love to do malicious compliance. If I tell them to not do X, they will then go into a “Look, I followed instructions” moment by talking about how they avoided X. If I add additional instructions saying “do not talk about how you did not do X since merely discussing it is contrary to the goal of avoiding it entirely”, they become somewhat better, but the process of writing such long prompts merely to say not to do something is annoying.
- brookst 1y agoYou’re giving them way too much agency. The don’t love anything and cant be malicious. You may get better results by emphasizing what you want and why the result was unsatisfactory rather than just saying “don’t do X” (this principle holds for people as well). Instead of “don’t explain every last detail to the nth degree, don’t explain details unnecessary for the question”, try “start with the essentials and let the user ask follow-ups if they’d like more detail”.
- ryao 1y agoThe idiom “X loves to Y” implies frequency, rather than agency. Would you object to someone saying “It loves to rain in Seattle”? “Malicious compliance” is the act of following instructions in a way that is contrary to the intent. The word malicious is part of the term. Whether a thing is malicious by exercising malicious compliance is tangential to whether it has exercised malicious compliance. That said, I have gotten good results with my addendum to my prompts to account for malicious compliance. I wonder if your comment Is due to some psychological need to avoid the appearance of personification of a machine. I further wonder if you are one of the people who are upset if I say “the machine is thinking” about a LLM still in prompt processing, but had no problems with “the machine is thinking” when waiting for a DOS machine to respond to a command in the 90s. This recent outrage over personifying machines since LLMs came onto the scene is several decades late considering that we have been personifying machines in our speech since the first electronic computers in the 1940s. By the way, if you actually try what you suggested, you will find that the LLM will enter a Laurel and Hardy routine with you, where it will repeatedly make the mistake for you to correct. I have experienced this firsthand so many times that I have learned to preempt the behavior by telling the LLM not to maliciously comply at the beginning when I tell it what not to do.
- vanillax 1y agohave you tried prompt rules/instructions? Fixes all my issues.
- amelius 1y agoI think you cannot really change the personality of an LLM by prompting. If you take the statistical parrot view, then your prompt isn't going to win against the huge numbers of inputs the model was trained with in a different personality. The model's personality is in its DNA so to speak. It has such an urge to parrot what it knows that a single prompt isn't going to change it. But maybe I'm psittacomorphizing a bit too much now.
- brookst 1y agoYeah different system prompts make a huge difference on the same base model”. There’s so much diversity in the training set, and it’s such a large set, that it essentially equals out and the system prompt has huge leverage. Fine tuning also applies here.
- joquarky 1y agoI liked the completion models because they have no chatter that needs to follow human conversational protocol, which inherently introduces "personality". The only difference from conversational chat was that you had to be creative about how to set up a "document" with the right context that will lead to the answer you're looking for. It was actually kind of fun.
- kemiller 1y agoYes this is strikingly similar to humans, too. “Not” is kind of an abstract concept. Anyone who has ever trained a dog will understand.
- JKCalhoun 1y agoI must be dyslexic? I always read, "Silica Gel, Eat, Do Not Throw Away" or something like that.
- Melatonic 1y agoI think its an english language thing (or language in general). Someone above commented about using the word "Avoid" instead of "do not". "Not" obviously means you should do the opposite but the first word is still a verb telling you to take action.
- bdangubic 1y agoNot obviously means you should do the opposite absolutely fascinating! can you elaborate on this?! I can’t put a context to this, like in what context does “not” means to do the opposite?!
- Melatonic 1y agoIt is a negation - so anytime you combine it with a verb (grammatically). Ex: I have seen the movie --> I have not seen the movie When combined with the verb "do" (and giving a command or instruction) it would negate the verb "do" Ex: Please do run on the lawn --> Please do not run on the lawn
- deleted 1y ago[deleted]
- siva7 1y agoI have a feeling this is the result of RHLF gone wrong by outsourcing it to idiots which all ai providers seem to be guilty of. Imagine a real professional wanting every output after a remark to start with "You're absolutely right!", Yeah, hard to imagine or you may have some specific cultural background or some kind of personality disorder. Or maybe it's just a hardcoded string? May someone with more insight enlighten us plebs.
- cherryteastain 1y agoThis is similar to the 'Waluigi effect' noticed all the way back in the GPT 3.5 days https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluigi-effect-mega-post https://www.lesswrong.com/posts/D7PumeYTDPfBTp3i7/the-waluig...
- berkeleyjunk 1y agoI wish someone had told Alex Blechman this before his "Don't Create the Torment Nexus" post.
- imchillyb 1y agoI've found this effect to be true with engagement algorithms as well, such as Youtube's thumbs-down, or 'don't show me this channel' 'Don't like this content', Spotify's thumbs down. Netflix's thumbs down. Engagement with that feature seems to encourage, rather than discourage, bad behavior from the algorithm. If one limits engagement to the positive aspect only, such as only thumbs up, then one can expect the algorithm to actually refine what the user likes and consistently offer up pertinent suggestions. The moment one engages with that nefarious downvote though... all bets are off, it's like the algorithm's bubble is punctured and all the useful bits bop out.
- wwweston 1y agoThe fact that “Don’t think if an elephant” shapes results in people and LLMs similarly is interesting.
- keviniam 1y agoOn the flip side, if you say "don't do xyz", this is probably because the LLM was already likely to do xyz (otherwise why say it?). So perhaps what you're observing is just its default behavior rather than "don't do xyz" actually increasing its likelihood to do xyz? Anecdotally, when I say "don't do xyz" to Gemini (the LLM I've recently been using the most), it tends not to do xyz. I tend not to use massive context windows, though, which is where I'm guessing things get screwy.
- Terretta 1y agoSince GPT 3, they've gotten better, but in practice we've found the best way to avoid this problem is use affirmative words like "AVOID". YES: AVOID using negations. NO: DO NOT use negations. Weirdly, I see the DO NOT (with caps) form in system prompts from the LLM vendors which is how we know they are hiring too fast.* * Slight joke, it seems this is being heavily trained since 4.1-ish on OpenAI's side and since 3.5 on Anthropic's side. But "avoid" still works better.
- Melatonic 1y agoI think you are really onto something here - I bet this would also reliably work when talking to humans. Maybe this is not even specifically the fault of the AI but just a language thing in general. An alternative test could be prompting the AI with "Avoid not" and then give it some kind of instruction. Theoretically this would be telling it to "do" the instruction but maybe sometimes it would end up "avoiding" it? Now that I think about it the training data itself might very well be contaminated with this contradiction....... I can think of a lot of forum posts where the OP stipulates "I do not want X" and then the very first reply recommends "X" !
- SubiculumCode 1y ago'not X' just becomes 'X', as our memories fade..I wouldn't be surprised the context degradation is similar in LLMs.
- zubiaur 1y agoFunnily enough, that is true also for giving instructions to kids. And also why kid's media is so frustrating. So many shows and books focus first on the maladjusted behavior, with the character learning not to the-bad-thing at the very end. Don't instruct kids, nor LLMs via negativa.
- snowfield 1y agoAis in general need to be told what to do. not what not to do.
- fennecbutt 1y agoSame here, also with examples as well - you give it any sort of example of the thing you want and at least half the time it quotes the example directly.