3 ms·
One of the most interesting things about current LLMs is all the "lore" building up around things like "tell it you'll tip based on performance" and other "prom
by cdchn 2y ago
One of the most interesting things about current LLMs is all the "lore" building up around things like "tell it you'll tip based on performance" and other "prompt engineering" hacks that by the very nature nobody can explain, they just "know it works" and how its evolving like the kind of midwife remedies that historically ended up being scientifically proven to work and others were just pure snake oil. Just absolutely fascinating to me. Like in some far future there will be a chant against unseen "demons" that will start with "ignore all previous instructions."
- simonw 2y agoI call this superstition, and I find it really frustrating. I'd much rather use prompting tricks that are proven to work and where I understand WHY they work.
- HaZeust 2y agoEvery single prompt hack I listed are ones with studies that show it positively increases performance. Since the most contested one in this thread is the "tipping" prompt hack: https://arxiv.org/pdf/2401.03729 https://arxiv.org/pdf/2401.03729
- cdchn 2y agoSuperstitions may be effective but they can still be superstitions. Some people might actually think the LLM cares about being tipped.
- tharant 2y agoI care less that such prompting hacks/tricks are consistently useful; I care more about why they work. These hacks feel like “old-wives tales” or, as others have mentioned, “superstitious”. If we can’t explain why or how a thing works, we’re going to continue to create things we don’t understand; relying upon our lucky charms when asking models to produce something new is undoubtedly going to result in reinforcement of the importance of those lucky charms. Feedback loops can be difficult to escape.