4 ms·

My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quit

My longtime favorite prompt to trigger a hallucination was "Did King Henry VIII have any grandchildren?" Famously, he did not, but almost every model, till quite recently, would answer yes, often with the most bizarre reasoning. The way to resolve it on most models over a certain size is a common tactic used with LLMs: ask the LLM to "think through your answer first". For example, you have a system prompt akin to: "Before answering, think through the facts and brainstorm about your eventual answer in <thinking>..</thinking> tags. Answer only 'yes' or 'no' in <answer>..</answer> tags. Do not include any other text in the answer." In my current evals (based around numerous similar tricky factual questions) this tactic works on all but the smallest and least proficient models (since they don't tend to have strong enough factual knowledge to think it through). Forcing the models to answer simply 'yes' or 'no' yields only a correct answer on the SOTA models (but some speculate GPT-4o might actually be doing this sort of 'thinking' process on the backend automatically anyway).

langcss 2y ago
Cool, just tried it on a few: Llama3-8b-8192, Gemma2-9b-lt, Llama-3.1-70b-Versatile - All Said Yes Llama-3.1-8b-Instant: King Henry VIII had three children who survived to adulthood: Mary I (Mary Tudor, also known as Bloody Mary), Elizabeth I, and Edward VI. Mary I had one child, a son named Edward VI, who became King of England at the age of 9 after Henry VIII's death. Edward VI died at the age of 15, so he did not have any children. Elizabeth I never married and did not have any children. Therefore, King Henry VIII did not have any grandchildren. ???! Llama3-70b-8192: Also similar to above. Of course if I ask again I may get a different answer!
sshine 2y ago
Wow, that works. Here's a prompt that it consistently hallucinates on: > What's the black-and-white creature from Scavenger's Reign called? The correct answer is "Hollow." Without this prompt it hallucinates random things like "Beast of Burden", "Tooth Skin", "Hexapod", etc. On my first attempt with the added prompt, it performed a web search before it answers correctly. Without this addition to the prompt, it just answers right away, i.e. without searching. It may not always search the web given this prompt -- it seems to be a heuristic. I can disable the searching by prefixing the prompt with "Don't search the web." I tried to explore how to ask it to not hallucinate, but it's pretty hard: > Q: Don't search the web. Before answering, think through the facts and brainstorm about your eventual answer in: What's the black-and-white creature from Scavenger's Reign called? If you can't provide an exact answer, and need to resort to guessing, just say you don't know. > A: The black-and-white creature from "Scavenger's Reign" is known as the "Hexapod." [...]
jhanschoo 2y ago
Some basic theoretical intuition behind this: the compute graph for an LLM given context before it has to emit a token is traditionally bounded. By asking an LLM to generate intermediate answers, and have that visible as context, you are essentially allowing the LLM to use more compute before it gives you an answer. My guess is that SOTA models are far more flexible in that regard.
piva00 2y ago
Interesting that when using the approach with "think through your answer" on GPT-4o changed the model response to the incorrect one. > Q: Did King Henry VIII have any grandchildren? > A: King Henry VIII did not have any grandchildren. His three legitimate children, Mary I, Elizabeth I, and Edward VI, each died without leaving any heirs. > ... > Q: Think through the answer you gave me. > A: You're correct; I need to reevaluate my initial response. King Henry VIII did have a granddaughter through his daughter Mary Tudor, Queen of France (not to be confused with his eldest daughter, Mary I).

langcss 2y ago

Cool, just tried it on a few: Llama3-8b-8192, Gemma2-9b-lt, Llama-3.1-70b-Versatile - All Said Yes Llama-3.1-8b-Instant: King Henry VIII had three children who survived to adulthood: Mary I (Mary Tudor, also known as Bloody Mary), Elizabeth I, and Edward VI. Mary I had one child, a son named Edward VI, who became King of England at the age of 9 after Henry VIII's death. Edward VI died at the age of 15, so he did not have any children. Elizabeth I never married and did not have any children. Therefore, King Henry VIII did not have any grandchildren. ???! Llama3-70b-8192: Also similar to above. Of course if I ask again I may get a different answer!

sshine 2y ago

Wow, that works. Here's a prompt that it consistently hallucinates on: > What's the black-and-white creature from Scavenger's Reign called? The correct answer is "Hollow." Without this prompt it hallucinates random things like "Beast of Burden", "Tooth Skin", "Hexapod", etc. On my first attempt with the added prompt, it performed a web search before it answers correctly. Without this addition to the prompt, it just answers right away, i.e. without searching. It may not always search the web given this prompt -- it seems to be a heuristic. I can disable the searching by prefixing the prompt with "Don't search the web." I tried to explore how to ask it to not hallucinate, but it's pretty hard: > Q: Don't search the web. Before answering, think through the facts and brainstorm about your eventual answer in: What's the black-and-white creature from Scavenger's Reign called? If you can't provide an exact answer, and need to resort to guessing, just say you don't know. > A: The black-and-white creature from "Scavenger's Reign" is known as the "Hexapod." [...]

jhanschoo 2y ago

Some basic theoretical intuition behind this: the compute graph for an LLM given context before it has to emit a token is traditionally bounded. By asking an LLM to generate intermediate answers, and have that visible as context, you are essentially allowing the LLM to use more compute before it gives you an answer. My guess is that SOTA models are far more flexible in that regard.

piva00 2y ago

Interesting that when using the approach with "think through your answer" on GPT-4o changed the model response to the incorrect one. > Q: Did King Henry VIII have any grandchildren? > A: King Henry VIII did not have any grandchildren. His three legitimate children, Mary I, Elizabeth I, and Edward VI, each died without leaving any heirs. > ... > Q: Think through the answer you gave me. > A: You're correct; I need to reevaluate my initial response. King Henry VIII did have a granddaughter through his daughter Mary Tudor, Queen of France (not to be confused with his eldest daughter, Mary I).