3 ms·

I’m curious, is adding “do not hallucinate” to prompts effective in preventing hallucinations? The author does this.

• krainboltgreene 2y ago
  It seems absurd, but I suppose it’s the same as misspelling with similar enough trigrams as to get the best autocorrect results.
• becquerel 2y ago
  I don't know about this specific technique, but I have found it useful to add a line like 'it's OK if you don't know or this isn't possible' at the end of queries. Otherwise LLMs have a tendency to tilt at whatever windmill you give them. Managing tone and expectations with them is a subtle but important art.
• simonw 2y ago
  Apple were using that in their Apple Intelligence system prompts last year, I don't know if they still have that in there. https://simonwillison.net/2024/Aug/6/apple-intelligence-prompts/ https://simonwillison.net/2024/Aug/6/apple-intelligence-prom... I have no idea if it works or not!
  • harper 2y ago
    I added it because of the apple prompts! I figured it is worth a try. The results are good, but i did not test it extensively
• watt 2y ago
  It will work - you can see it well with a Chain of Thought (CoT) model: it will keep asking itself: "am I hallucinating? let's double check" and then will self-reject thoughts if it can't find a proper grounding. In fact, this is the best part of CoT model, that you can see where it goes off rails and can add a message to fix it in the prompt. For example, there is this common challenge, "count how many r letters in strawberry", and you can see the issue is not counting, but that model does not know if "rr" should be treated as single "r" because it is not sure if you are counting r "letters" or r "sounds" and when you sound out the word, there is a single "r" sound where it is spelled with double "r". so if you tell the model, double "r" stands for 2 letters, it will get it right.