6 ms·

Is this part of why ChatGPT hallucinates? Lower probability responses sometimes "win" which forces the model to take the wrong path?

albystein 4y ago
I think hallucination is akin to encountering a problem on a test to which you don’t fully remember the right concept to solve it. You might attempt the problem with whatever little knowledge you can recall, but the answer is ultimately incorrect. Likewise, these LLMs exhibit such behavior when they hallucinate. I think what distinguishes the LLM’s hallucinations from humans is the confidence level. A human who is unsure of their answer might tell you that they’re not certain or they don’t know. LLMs like chatgpt, however, will just confidently make stuff up.
geraneum 4y ago
Higher probably responses are not necessarily “truthful” or “logical” in any given context. Here* you can see an example of what happens when you always choose the highest probability. [*] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...
muzani 4y ago
It can halluciate at low temp too. e.g. mice don't necessarily eat cheese; it's just a common stereotype of what mice eat. High temp is more likely to lead it into ranting on an unrelated topic. You ask it a question on what the mouse eats. It answers the question and then starts saying what lizards eat, or goes on a rant about food chains.
recuter 4y ago
My favorite prompt to try so far to play with this effect is: Picture yourself in a boat on a river With tangerine trees and marmalade skies It might veer off into a scenario where you're on a romantic boat trip or talk about how it is its favorite Beatles song and how it remembers them going on Ed Sullivan. :) The temp affects the "color" of the next word which sends it on a tangent just like a person -- especially somebody with ADHD or playing an improve game as opposed to lip reading, for example.
wodenokoto 4y ago
No. I'd say everything the model does is a hallucination. Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this is what you are thinking when saying "Is this part of why ChatGPT hallucinates?", then no. The model never knows if what it is saying is correct. It has a probability about how likely the words are to follow eachother, and that is about it. The temperature changes how these probabilities are calculated, making otherwise less probable words more probable or vice versa. However, the model doesn't choose the most likely word every time. Instead it rolls a dice to see where it wants to go next. If Cheese is 80% and Cookie is 20% it won't necessarily choose Cheese. It rolls a die, and maybe the outcome lands on the less likely Cookie. The roll of the dice happen _after_ temperature has affected the calculation of probabilities. Maybe it is this roll you are thinking of as hallucination?
- logicallee 4y ago
  You say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.
- detrites 4y ago
  While it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction. Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary. The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.
  - jnwatson 4y ago
WithinReason 4y ago
I think it hallucinates in scenarios where it either doesn't "remember" the answer or there is no answer. For example: since the text generation is sequential, based on context the highest probability continuation in a situation might be: "A famous paper that discussed this phenomenon is" However, when the network gets to this point it doesn't know the name of such a paper. Maybe it already knew that it doesn't know the paper's name but the above sentence was generated despite that, since everything else was lower probability or it got sampled due to higher temperature. In any case, it can't go back and delete the sentence so it must make up something. So it ends up going with the most plausible sounding (highest sample probability) continuation, which ends up being a "hallucination". Basically hallucinations are produced when the network "writes itself into a corner" due to the sequential nature of text generation.
golol 4y ago
definetly not.
kzrdude 4y ago
Mice in nature don't eat cheese. So even the most favoured response is a hallucination.
- qwertox 4y ago
  In nature they also don't eat cookies then.
  - denton-scratch 4y ago
    They favour grains over cheese. Cookies and baguettes (even for non-French mice) work better than cheese in mousetraps. Peanut butter is widely recommended as the best mousetrap bait. I have had success with peanut butter.

albystein 4y ago

I think hallucination is akin to encountering a problem on a test to which you don’t fully remember the right concept to solve it. You might attempt the problem with whatever little knowledge you can recall, but the answer is ultimately incorrect. Likewise, these LLMs exhibit such behavior when they hallucinate. I think what distinguishes the LLM’s hallucinations from humans is the confidence level. A human who is unsure of their answer might tell you that they’re not certain or they don’t know. LLMs like chatgpt, however, will just confidently make stuff up.

geraneum 4y ago

Higher probably responses are not necessarily “truthful” or “logical” in any given context. Here* you can see an example of what happens when you always choose the highest probability. [*] https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-doing-and-why-does-it-work/ https://writings.stephenwolfram.com/2023/02/what-is-chatgpt-...

muzani 4y ago

It can halluciate at low temp too. e.g. mice don't necessarily eat cheese; it's just a common stereotype of what mice eat. High temp is more likely to lead it into ranting on an unrelated topic. You ask it a question on what the mouse eats. It answers the question and then starts saying what lizards eat, or goes on a rant about food chains.

recuter 4y ago

My favorite prompt to try so far to play with this effect is: Picture yourself in a boat on a river With tangerine trees and marmalade skies It might veer off into a scenario where you're on a romantic boat trip or talk about how it is its favorite Beatles song and how it remembers them going on Ed Sullivan. :) The temp affects the "color" of the next word which sends it on a tangent just like a person -- especially somebody with ADHD or playing an improve game as opposed to lip reading, for example.

wodenokoto 4y ago

No. I'd say everything the model does is a hallucination. Some writers tend to split ChatGPT oputput into "correct answers" and "Hallucinated answers". If this is what you are thinking when saying "Is this part of why ChatGPT hallucinates?", then no. The model never knows if what it is saying is correct. It has a probability about how likely the words are to follow eachother, and that is about it. The temperature changes how these probabilities are calculated, making otherwise less probable words more probable or vice versa. However, the model doesn't choose the most likely word every time. Instead it rolls a dice to see where it wants to go next. If Cheese is 80% and Cookie is 20% it won't necessarily choose Cheese. It rolls a die, and maybe the outcome lands on the less likely Cookie. The roll of the dice happen _after_ temperature has affected the calculation of probabilities. Maybe it is this roll you are thinking of as hallucination?

logicallee 4y ago
You say it never knows if what it is saying is correct, but in my experience it tells you when it does and doesn't know about a concept. For example in one thread I had it give me some random words from linear algebra, then in another thread I combined these into the nonsense phrase "decomposed orthogonal eigenvector division" and asked it if it was familiar with this, and if so to define it. It said it wasn't familiar with it and asked for more information. I'd say that shows it knows what it doesn't know.
detrites 4y ago
While it can be customary to trivialise this, the reality is that no "intelligence" is sure if what it's saying may be correct or constitute any kind of "knowledge". At this level a human can't claim to arbiter fact versus fiction. Fundamentally, no one can be sure what they experience is not some kind of simulation, or a dreamlike state beneath some more real "reality". As such, assigning hierarchy to such a foundation lacking in tangible proof is arbitrary. The point being, unless it's possible to conclusively and infallibly prove you are not hallucinating, you can't lay claim to having knowledge an AI is hallucinating, or not.
- jnwatson 4y ago

WithinReason 4y ago

I think it hallucinates in scenarios where it either doesn't "remember" the answer or there is no answer. For example: since the text generation is sequential, based on context the highest probability continuation in a situation might be: "A famous paper that discussed this phenomenon is" However, when the network gets to this point it doesn't know the name of such a paper. Maybe it already knew that it doesn't know the paper's name but the above sentence was generated despite that, since everything else was lower probability or it got sampled due to higher temperature. In any case, it can't go back and delete the sentence so it must make up something. So it ends up going with the most plausible sounding (highest sample probability) continuation, which ends up being a "hallucination". Basically hallucinations are produced when the network "writes itself into a corner" due to the sequential nature of text generation.

golol 4y ago

definetly not.

kzrdude 4y ago

Mice in nature don't eat cheese. So even the most favoured response is a hallucination.

qwertox 4y ago
In nature they also don't eat cookies then.
- denton-scratch 4y ago
  They favour grains over cheese. Cookies and baguettes (even for non-French mice) work better than cheese in mousetraps. Peanut butter is widely recommended as the best mousetrap bait. I have had success with peanut butter.