10 ms·

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. i

Notably, this prompt is making "hallucinations" an officially recognized phenomenon: > If Claude is asked about a very obscure person, object, or topic, i.e. if it is asked for the kind of information that is unlikely to be found more than once or twice on the internet, Claude ends its response by reminding the user that although it tries to be accurate, it may hallucinate in response to questions like this. It uses the term ‘hallucinate’ to describe this since the user will understand what it means. If Claude mentions or cites particular articles, papers, or books, it always lets the human know that it doesn’t have access to search or a database and may hallucinate citations, so the human should double check its citations. Probably for the best that users see the words "Sorry, I hallucinated" every now and then.

hotstickyballs 2y ago
“Hallucination” has been in the training data much earlier than even llms. The easiest way to control this phenomenon is using the “hallucination” tokens, hence the construction of this prompt. I wouldn’t say that this makes things official.
- creatonez 2y ago
  > The easiest way to control this phenomenon is using the “hallucination” tokens, hence the construction of this prompt. That's what I'm getting at. Hallucinations are well known about, but admitting that you "hallucinated" in a mundane conversation is a rare thing to happen in the training data, so a minimally prompted/pretrained LLM would be more likely to say "Sorry, I misinterpreted" and then not realize just how grave the original mistake was, leading to further errors. Add the word hallucinate and the chatbot is only going to humanize the mistake by saying "I hallucinated", which lets it recover from extreme errors gracefully. Other words, like "confabulation" or "lie", are likely more prone to causing it to have an existential crisis. It's mildly interesting that the same words everyone started using to describe strange LLM glitches also ended up being the best token to feed to make it characterize its own LLM glitches. This newer definition of the word is, of course, now being added to various human dictionaries (such as https://en.wiktionary.org/wiki/hallucinate#Verb https://en.wiktionary.org/wiki/hallucinate#Verb) which will probably strengthen the connection when the base model is trained on newer data.
axus 2y ago
I was thinking about LLMs hallucinating function names when writing programs, it's not a bad thing as long as it follows up and generates the code for each function name that isn't real yet. So hallucination is good for purely creative activities, and bad for analyzing the past.
- TeMPOraL 2y ago
  In a way, LLMs tend to follow a very reasonable practice of coding to the API you'd like to have, and only later reconcile it with the API you actually have. Reconciling may be as simple as fixing a function name, or as complex as implementing the "fake"/"hallucinated" functions, which work as glue code.
- rafaelmn 2y ago
  That's not hallucinating that's just missing parts of implementation. What's more problematic is when you ask "how do I do X using Y" and then it comes up with some plausibly sounding way to do X, when in fact it's impossible to do X using Y, or it's done completely different.
xienze 2y ago
> Probably for the best that users see the words "Sorry, I hallucinated" every now and then. Wouldn’t “sorry, I don’t know how to answer the question” be better?
- SatvikBeri 2y ago
  That requires more confidence. If there's a 50% chance something is true, I'd rather have Claude guess and give a warning than say it doesn't know how to answer.
- creatonez 2y ago
  Not necessarily. The LLM doesn't know what it can answer before it tries to. So in some cases it might be better to make an attempt and then later characterize it as a hallucination, so that the error doesn't spill over and produce even more incoherent nonsense. The chatbot admitting that it "hallucinated" is a strong indication to itself that part of the previous text is literal nonsense and cannot be trusted, and that it needs to take another approach.
- lemming 2y ago
  "Sorry, I just made that up" is more accurate.
  - throw310822 2y ago
    And it reveals how "hallucinations" are a quite common occurrence also for humans.
    - TeMPOraL 2y ago
      Specifically, "hallucinations" are very common in humans; we usually don't call it "making things up" (as in, intentionally), but rather we call it "talking faster than you think" or "talking at the speed of thought". Which is pretty much what LLMs do.
      - adastra22 2y ago
armchairhacker 2y ago
How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.
- brulard 2y ago
  I believe Claude is aware if information close to the one retrieved from the vector space is scarce. I'm no expert, but i imagine it makes a query to the vector database and get the data close enough to places pointed out by the prompt. And it may see that part of the space is quite empty. If this is far off, someone please explain.
  - th0ma5 2y ago
    I think good, true, but rare information would also fit that definition so it'd be a shame if it discovered something that could save humanity but then discounted it as probably not accurate.
    - deleted 2y ago
      [deleted]
  - rf15 2y ago
    I wonder if that's the case - the prompt text (like all text interaction with LLMs) is seen from "within" the vector space, while sparcity is only observable from the "outside"
  - deleted 2y ago
    [deleted]
- furyofantares 2y ago
  I think it could be fine tuned to give it an intuition, like how you or I have an intuition about what might be found on the internet. That said I've never seen it give the response suggested in this prompt and I've tried loads of prompts just like this in my own workflows and they never do anything.
samstave 2y ago
>...mentions or cites particular articles, papers, or books, it always lets the human know that it doesn’t have access to search or a database... I wonder if we can create a "reverse Google" -- which is a RAG/Human Reinforcement GPT-pedia == Where we dump "confirmed real" information into it that is always current - and all LLMs are free to harvest directly from it in discernment of crafting responses. For example - it could accept FireHose all current/active streams/podcasts of anything "live" and be like an AI-Tivo for any live streams and it can havea temporal windows that you cans search through "Show me every instance of [THING FROM ALL LIVE STREAMS WATCHED IN THE LAST 24 HOURS] - give me a markdown of the top channels, views, streams, comments - controversy, retweets regarding that topic. sort by time posted. (Recall that HNer posting the "if youtube had channels:") https://news.ycombinator.com/item?id=41247023 https://news.ycombinator.com/item?id=41247023 -- Remember when "Twitter give 'FireHose' directly to the Library of Congress! Why not firehose GPT-to tha Tap Data'sset https://www.forbes.com/sites/kalevleetaru/2017/12/28/the-library-of-congress-in-the-digital-age-from-archiving-twitter-to-innovation-lab/ https://www.forbes.com/sites/kalevleetaru/2017/12/28/the-lib...
fennecfoxy 2y ago
I don't know why large models are still trying to draw answers from their training info. Seems the quality of doing a search and parsing results is much more effective, less chance for hallucination if the model is specifically trained that drawing information from outside context = instant fail.

hotstickyballs 2y ago

“Hallucination” has been in the training data much earlier than even llms. The easiest way to control this phenomenon is using the “hallucination” tokens, hence the construction of this prompt. I wouldn’t say that this makes things official.

creatonez 2y ago
> The easiest way to control this phenomenon is using the “hallucination” tokens, hence the construction of this prompt. That's what I'm getting at. Hallucinations are well known about, but admitting that you "hallucinated" in a mundane conversation is a rare thing to happen in the training data, so a minimally prompted/pretrained LLM would be more likely to say "Sorry, I misinterpreted" and then not realize just how grave the original mistake was, leading to further errors. Add the word hallucinate and the chatbot is only going to humanize the mistake by saying "I hallucinated", which lets it recover from extreme errors gracefully. Other words, like "confabulation" or "lie", are likely more prone to causing it to have an existential crisis. It's mildly interesting that the same words everyone started using to describe strange LLM glitches also ended up being the best token to feed to make it characterize its own LLM glitches. This newer definition of the word is, of course, now being added to various human dictionaries (such as https://en.wiktionary.org/wiki/hallucinate#Verb https://en.wiktionary.org/wiki/hallucinate#Verb) which will probably strengthen the connection when the base model is trained on newer data.

axus 2y ago

I was thinking about LLMs hallucinating function names when writing programs, it's not a bad thing as long as it follows up and generates the code for each function name that isn't real yet. So hallucination is good for purely creative activities, and bad for analyzing the past.

TeMPOraL 2y ago
In a way, LLMs tend to follow a very reasonable practice of coding to the API you'd like to have, and only later reconcile it with the API you actually have. Reconciling may be as simple as fixing a function name, or as complex as implementing the "fake"/"hallucinated" functions, which work as glue code.
rafaelmn 2y ago
That's not hallucinating that's just missing parts of implementation. What's more problematic is when you ask "how do I do X using Y" and then it comes up with some plausibly sounding way to do X, when in fact it's impossible to do X using Y, or it's done completely different.

xienze 2y ago

> Probably for the best that users see the words "Sorry, I hallucinated" every now and then. Wouldn’t “sorry, I don’t know how to answer the question” be better?

SatvikBeri 2y ago
That requires more confidence. If there's a 50% chance something is true, I'd rather have Claude guess and give a warning than say it doesn't know how to answer.
creatonez 2y ago
Not necessarily. The LLM doesn't know what it can answer before it tries to. So in some cases it might be better to make an attempt and then later characterize it as a hallucination, so that the error doesn't spill over and produce even more incoherent nonsense. The chatbot admitting that it "hallucinated" is a strong indication to itself that part of the previous text is literal nonsense and cannot be trusted, and that it needs to take another approach.
lemming 2y ago
"Sorry, I just made that up" is more accurate.
- throw310822 2y ago
  And it reveals how "hallucinations" are a quite common occurrence also for humans.
  - TeMPOraL 2y ago
    Specifically, "hallucinations" are very common in humans; we usually don't call it "making things up" (as in, intentionally), but rather we call it "talking faster than you think" or "talking at the speed of thought". Which is pretty much what LLMs do.
    - adastra22 2y ago

armchairhacker 2y ago

How can Claude "know" whether something "is unlikely to be found more than once or twice on then internet"? Unless there are other sources that explicitly say "[that thing] is obscure". I don't think LLMs can report if something was encountered more/less often in their training data, there are too many weights and neither us nor them know exactly what each of them represents.

brulard 2y ago
I believe Claude is aware if information close to the one retrieved from the vector space is scarce. I'm no expert, but i imagine it makes a query to the vector database and get the data close enough to places pointed out by the prompt. And it may see that part of the space is quite empty. If this is far off, someone please explain.
- th0ma5 2y ago
  I think good, true, but rare information would also fit that definition so it'd be a shame if it discovered something that could save humanity but then discounted it as probably not accurate.
  - deleted 2y ago
    [deleted]
- rf15 2y ago
  I wonder if that's the case - the prompt text (like all text interaction with LLMs) is seen from "within" the vector space, while sparcity is only observable from the "outside"
- deleted 2y ago
  [deleted]
furyofantares 2y ago
I think it could be fine tuned to give it an intuition, like how you or I have an intuition about what might be found on the internet. That said I've never seen it give the response suggested in this prompt and I've tried loads of prompts just like this in my own workflows and they never do anything.

samstave 2y ago

>...mentions or cites particular articles, papers, or books, it always lets the human know that it doesn’t have access to search or a database... I wonder if we can create a "reverse Google" -- which is a RAG/Human Reinforcement GPT-pedia == Where we dump "confirmed real" information into it that is always current - and all LLMs are free to harvest directly from it in discernment of crafting responses. For example - it could accept FireHose all current/active streams/podcasts of anything "live" and be like an AI-Tivo for any live streams and it can havea temporal windows that you cans search through "Show me every instance of [THING FROM ALL LIVE STREAMS WATCHED IN THE LAST 24 HOURS] - give me a markdown of the top channels, views, streams, comments - controversy, retweets regarding that topic. sort by time posted. (Recall that HNer posting the "if youtube had channels:") https://news.ycombinator.com/item?id=41247023 https://news.ycombinator.com/item?id=41247023 -- Remember when "Twitter give 'FireHose' directly to the Library of Congress! Why not firehose GPT-to tha Tap Data'sset https://www.forbes.com/sites/kalevleetaru/2017/12/28/the-library-of-congress-in-the-digital-age-from-archiving-twitter-to-innovation-lab/ https://www.forbes.com/sites/kalevleetaru/2017/12/28/the-lib...

fennecfoxy 2y ago

I don't know why large models are still trying to draw answers from their training info. Seems the quality of doing a search and parsing results is much more effective, less chance for hallucination if the model is specifically trained that drawing information from outside context = instant fail.