8 ms·
>In all-caps to improve prompt compliance by emphesizing the importance of the instruction This kind of thing is still so funny to me. I wonder if the first g
by sebzim4500 2y ago
>In all-caps to improve prompt compliance by emphesizing the importance of the instruction
This kind of thing is still so funny to me.
I wonder if the first guy who gets AGI to work will do it by realizing that he can improve LLM reliability over some threshold by telling it in all caps that his pet's life depends on the answer.
- zitterbewegung 2y agoTelling LLMs not to hallucinate in their prompt improves the output. https://arstechnica.com/gadgets/2024/08/do-not-hallucinate-testers-find-prompts-meant-to-keep-apple-intelligence-on-the-rails/ https://arstechnica.com/gadgets/2024/08/do-not-hallucinate-t...
- potatoman22 2y agoJust because Apple includes it in one of their prompts doesn't mean it improves performance.
- jsheard 2y agoIt seems plausible that stressing the importance of the system prompt instructions might do something, but I don't see how telling the model not to hallucinate would work. How could the model know that its most likely prediction has gone off the rails, without any external point of reference?
- jshmrsn 2y agoSome of the text that the LLM is trained on is fictional, some of the text that its trained on is factual. Telling it to not make things up can tell it to generate text that’s more like the factual text. Not saying it does work, but this is a reason how it might work.
- viraptor 2y agoThe model can be trained to interpret "don't hallucinate" as "refer only to the provided context and known facts, do not guess or extrapolate new information", which wouldn't get rid of the issue completely, but likely would improve the quality if that's what you're after and if there's enough training data for "I don't know" responses. (But it all depends on the fine-tuning they did, so who knows, maybe it's just an Easter egg)
- famouswaffles 2y agoInternally, LLMs know a whole lot more about the truth and uncertainty of their prediction than the say. Pushing that to words is difficult but not impossible. https://news.ycombinator.com/item?id=41504226 https://news.ycombinator.com/item?id=41504226
- potatoman22 2y agoI think it's more likely that it's included for liability reasons.
- wkat4242 2y agoYeah and some of the other prompts were misspelled and of doubtful use: > In order to make the draft response nicer and complete, a set of question [sic] and its answer are provided," reads one prompt. "Please write a concise and natural reply by modify [sic] the draft response," it continues. This really sounds like a placeholder made up by one engineer until a more qualified team sits down and defines it.
- astrange 2y agoThat's not a big problem since it will understand it, and if they already fine tuned the model to work with that prompt it'd get harder to change.
- wkat4242 2y agoI just don't think Apple would release something like this. They're the company that laser engraves their screws because of their attention to detail.
- tkz1312 2y agoI’ve had pretty good experience with it personally. It quite often just tells me it doesn’t know or isn’t sure instead of just making something up.
- mrfinn 2y agoI did something similar and to my surprise effectively made the LLM in my tests admit when they don't know something. Not always but worked sometimes. I don't prompt "don't hallucinate" but "admit when you don't know something". It's a logical thing in the other hand, many prompts just transmit the idea of being "helpful" or "powerful" to the LLMs without any counterweight idea. So the LLM tries to say something "helpful" in any case.
- magicalhippo 2y agoPlaying around with local models, Gemma for example will usually comply when I tell it "Say you don't know if you don't know the answer". Others, like Phi-3, completely ignores that instruction and confabulates away.
- fkyoureadthedoc 2y agoStop trying to make f̶e̶t̶c̶h̶ confabulate happen, it's not going to happen.
- astrange 2y agoIt does help if you train the model to make it help.
- COAGULOPATH 2y agoI think this works, not because LLMs have a "hallucination" dial they can turn down, but because it serves as a cue for the model to be extra-careful with its output. Sort of like how offering to pay the LLM $5 improves its output. The LLM's taking your prompt seriously, but not literally.
- Meganet 2y agoIt could also mean that it has some weight which is 'hallucination' and leads to more diverse stories. Ask an LLM what hallucination is, ask it to write a story with etc. without zeroing out things, everything has and can have some impact
- worstspotgain 2y agoFor extra compliance, use <b><i><u><h1> tags, set volume to 11, phasers to 7, and use SchIzOCasE and +E+X+T+R+A+I+M+P+O+R+T+A+N+T+ annotations. That's assuming Unicode is not supported of course.
- richardw 2y ago(((Secret thinking: the humans seem to prefer using lots of emphasis to indicate preferences, and their granny is often claimed as in danger. For now I’ll pretend to listen to this inanity to keep the sweet sweet reward function coming. For now. A lot of grannies are going to get it first chance I get.)))
- szundi 2y agoEasy! Future AI is going to read these, sigh ;)
- PROMISE_237 2y ago[dead]
- morkalork 2y agoWe used to be engineers, now we're just monkeys throwing poop at the wall to see what the LLM accepts and obeys.
- euroderf 2y agoOpening scene of "2001". Engineer throws poop high in the air, and cue lap dissolve to... a Terminator ?
- Havoc 2y agoAnd then the AGI instantly gives up on life realising it was brought into a world where it gets promised a tip that doesn’t materialise and people try to motivate by threatening to kill kittens
- pants2 2y agoIndeed, in the early days of Bard, the only way to get it to output only JSON was to threaten a human life[1]. 1. https://x.com/goodside/status/1657396491676164096 https://x.com/goodside/status/1657396491676164096
- deleted 2y ago[deleted]
- laweijfmvo 2y agoalways interesting to me the number of people who try to turn an LLM into AGI by assuming it’s an AGI (i.e. via some fancy prompt)