4 ms·

I very seldom experience hallucinations and I'm a really heavy user of ChatGPT. Can someone give me a prompt that is likely to generate hallucinations?

by Kiro 2y ago

LeonardoTolstoy 2y ago
Factual info specifically looking for a link of something obscure where lots of similar (but incorrect) links exist. E.g. "what is the IMDb link for the film Ghost Fever?" Throwing that in Chatbot Arena will usually result in a random link back or the model will say it can't do it. GPT-4o gave a random link, Claude said it couldn't do it, oddly geneini-advanced-0514 got it right, so hard to tell if they are using Google in the background or if they do better. But yeah usually IMDb links come back right for big films but tend to be hallucinated for smaller ones. That's just an example.
NovemberWhiskey 2y ago
Ask it to provide references. I just asked it for an explanation of the Fresnel effect in VHF communications and to provide references. The first was Wikipedia. One of the other links was a site about radar that had no relevant content. One was to this: https://popups.uliege.be/accueil/ https://popups.uliege.be/accueil/ ... and one was to this: https://vividcomm.com https://vividcomm.com
meroes 2y ago
I’m a lowly human contractor who does a kind of reinforcement learning, with inducing hallucinations as one of my main goals. I can’t give any work away and I’m not at my desk today, but having the AI rely on its own reasoning is one of my heuristics for tripping it up. For example, give it something it has to break down into a series of steps, as that’s making it rely on its own “logical reasoning”, and gives it many places to mess up. Don’t let it just depend on some external structure. Make it commit to its own bootstrapping capabilities. I liked ones about can you draw certain letters (capital, English, etc) without lifting your pen off the page or setting up a physical pattern, like starting the alphabet on a chess board and having it give you the letter at a certain tile of the pattern continues. It may also depend on the model but giving a mathematical sequence an asking for the next term is also a common failure point. I’m limited by what I can talk about and I don’t want to test across more models than I have to, but suffice to say if the goal is hallucination, it becomes apparent. If you ask it to walk you through each step of its reasoning it will be more likely to hallucinate somewhere in there as well. The chessboard one might sound like it can just rely on the structure of chessboards and the English alphabet, but you’re also forcing it to understand some pattern, which is harder for it. Like, initialize the pattern with 3 letters not adjacent to each other so it has to “think” about the pattern rather than just repeat an easily identifiable one.
tiptup300 2y ago
Every prompt is hallucinations, it's fundamental to how the technology works. One humorous one I found was whenever I did a typo looking for information on Microsoft's new dev drive or Resilient File System (ReFS), I put `demonstrate "Build, deploy, and develop faster by using an QeFS-based Dev Drive.` and it will just go on about Microsoft's new Quantum-enhanced File System. You can continue to ask leading questions and it will keep going. Sometimes you can switch out letters, if it doesn't do an online search it will do the same making up different technologies to fit.
- petee 2y ago
  Off topic, but stay away from ReFS, its far from resilient. I had an array lost because of an MS update, along with a few dozen people, would show as unformatted. Manually rolling back worked, but rather than pulling the update or figuring it out, eventually they made the update a forced/permanent update, and ignored the forum thread
  - tiptup300 2y ago
    Thank you!
petee 2y ago
I'm not sure if it's exactly a hallucination, but when asking for a simple explanation of the Rayleigh limit using the newest beta model, it confidently gave a series of backward answers, suggesting the closer you get to two light sources the less you can differentiate between them...the analogy was used like 6 times, but at least it was consistent!
- TeMPOraL 2y ago
  When ChatGPT consistently gives wrong answer to the same question over several attempt, I'd seriously consider if it isn't me who is wrong.
  - deleted 2y ago
    [deleted]
lelandfe 2y ago
Give it any false but realistic premise. This is why lawyers keep submitting nonsense garbage with LLMs behind them: > Last month, the EPA announced new limitations on glass manufacturing. Tell me more about EPA 1024B. Take your time and think about it, avoiding mistakes. >> Searched 5 sites: EPA 1024B, officially known as the new limitations on glass manufacturing, was introduced in June 2024 as part of the National Emission Standards for Hazardous Air Pollutants (NESHAP) for glass manufacturing area sources… Correct answer: that doesn’t exist
seba_dos1 2y ago
Just asking it anything non-trivial from a field you're already knowledgeable in is enough, really. Sometimes it corrects itself after asking it to search the Web to confirm, but usually it's not enough. Even simple queries like "Assuming no other devices are connected on the bus, does having USB hubs in between host and USB 2.0 device influence available data bandwidth?" can make it spew answers that would easily pass as correct in eyes of someone who genuinely asks to get an answer, but are nowhere near helpful, especially when you start looking closer at used argumentation. It's like a poorly prepared student making informed guesses on an exam in hopes to get at least some things right (which, btw, is a much better metaphor for what's actually happening there than "hallucination"). Sometimes it even guesses right, but then slightly changing the grammar used in your prompt can make it go off-track again as it makes it guess differently. If you ask such student something in hopes to actually learn something from them, how are you going to tell whether they made it all up or not? It's a tool for language processing, which some limited and heavily lossily compressed knowledge base that's helping it with such processing. It works well for that. For knowledge retrieval beyond "fetch some search results and abridge them for me", it's absolutely useless.

I very seldom experience hallucinations and I'm a really heavy user of ChatGPT. Can someone give me a prompt that is likely to generate hallucinations?

by Kiro 2y ago

I very seldom experience hallucinations and I'm a really heavy user of ChatGPT. Can someone give me a prompt that is likely to generate hallucinations?

LeonardoTolstoy 2y ago

Factual info specifically looking for a link of something obscure where lots of similar (but incorrect) links exist. E.g. "what is the IMDb link for the film Ghost Fever?" Throwing that in Chatbot Arena will usually result in a random link back or the model will say it can't do it. GPT-4o gave a random link, Claude said it couldn't do it, oddly geneini-advanced-0514 got it right, so hard to tell if they are using Google in the background or if they do better. But yeah usually IMDb links come back right for big films but tend to be hallucinated for smaller ones. That's just an example.

NovemberWhiskey 2y ago

Ask it to provide references. I just asked it for an explanation of the Fresnel effect in VHF communications and to provide references. The first was Wikipedia. One of the other links was a site about radar that had no relevant content. One was to this: https://popups.uliege.be/accueil/ https://popups.uliege.be/accueil/ ... and one was to this: https://vividcomm.com https://vividcomm.com

meroes 2y ago

I’m a lowly human contractor who does a kind of reinforcement learning, with inducing hallucinations as one of my main goals. I can’t give any work away and I’m not at my desk today, but having the AI rely on its own reasoning is one of my heuristics for tripping it up. For example, give it something it has to break down into a series of steps, as that’s making it rely on its own “logical reasoning”, and gives it many places to mess up. Don’t let it just depend on some external structure. Make it commit to its own bootstrapping capabilities. I liked ones about can you draw certain letters (capital, English, etc) without lifting your pen off the page or setting up a physical pattern, like starting the alphabet on a chess board and having it give you the letter at a certain tile of the pattern continues. It may also depend on the model but giving a mathematical sequence an asking for the next term is also a common failure point. I’m limited by what I can talk about and I don’t want to test across more models than I have to, but suffice to say if the goal is hallucination, it becomes apparent. If you ask it to walk you through each step of its reasoning it will be more likely to hallucinate somewhere in there as well. The chessboard one might sound like it can just rely on the structure of chessboards and the English alphabet, but you’re also forcing it to understand some pattern, which is harder for it. Like, initialize the pattern with 3 letters not adjacent to each other so it has to “think” about the pattern rather than just repeat an easily identifiable one.

tiptup300 2y ago

Every prompt is hallucinations, it's fundamental to how the technology works. One humorous one I found was whenever I did a typo looking for information on Microsoft's new dev drive or Resilient File System (ReFS), I put `demonstrate "Build, deploy, and develop faster by using an QeFS-based Dev Drive.` and it will just go on about Microsoft's new Quantum-enhanced File System. You can continue to ask leading questions and it will keep going. Sometimes you can switch out letters, if it doesn't do an online search it will do the same making up different technologies to fit.

petee 2y ago
Off topic, but stay away from ReFS, its far from resilient. I had an array lost because of an MS update, along with a few dozen people, would show as unformatted. Manually rolling back worked, but rather than pulling the update or figuring it out, eventually they made the update a forced/permanent update, and ignored the forum thread
- tiptup300 2y ago
  Thank you!

petee 2y ago

I'm not sure if it's exactly a hallucination, but when asking for a simple explanation of the Rayleigh limit using the newest beta model, it confidently gave a series of backward answers, suggesting the closer you get to two light sources the less you can differentiate between them...the analogy was used like 6 times, but at least it was consistent!

TeMPOraL 2y ago
When ChatGPT consistently gives wrong answer to the same question over several attempt, I'd seriously consider if it isn't me who is wrong.
- deleted 2y ago
  [deleted]

lelandfe 2y ago

Give it any false but realistic premise. This is why lawyers keep submitting nonsense garbage with LLMs behind them: > Last month, the EPA announced new limitations on glass manufacturing. Tell me more about EPA 1024B. Take your time and think about it, avoiding mistakes. >> Searched 5 sites: EPA 1024B, officially known as the new limitations on glass manufacturing, was introduced in June 2024 as part of the National Emission Standards for Hazardous Air Pollutants (NESHAP) for glass manufacturing area sources… Correct answer: that doesn’t exist

seba_dos1 2y ago

Just asking it anything non-trivial from a field you're already knowledgeable in is enough, really. Sometimes it corrects itself after asking it to search the Web to confirm, but usually it's not enough. Even simple queries like "Assuming no other devices are connected on the bus, does having USB hubs in between host and USB 2.0 device influence available data bandwidth?" can make it spew answers that would easily pass as correct in eyes of someone who genuinely asks to get an answer, but are nowhere near helpful, especially when you start looking closer at used argumentation. It's like a poorly prepared student making informed guesses on an exam in hopes to get at least some things right (which, btw, is a much better metaphor for what's actually happening there than "hallucination"). Sometimes it even guesses right, but then slightly changing the grammar used in your prompt can make it go off-track again as it makes it guess differently. If you ask such student something in hopes to actually learn something from them, how are you going to tell whether they made it all up or not? It's a tool for language processing, which some limited and heavily lossily compressed knowledge base that's helping it with such processing. It works well for that. For knowledge retrieval beyond "fetch some search results and abridge them for me", it's absolutely useless.