5 ms·

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is wi

Man it seems like the ship has sailed on "hallucination" but it's such a terrible name for the phenomenon we see. It is a major mistake to imply the issue is with perception rather than structural incompetence. Why not just say "incoherent output"? It's actually descriptive and doesn't require bastardizing a word we already find meaningful to mean something completely different.

jwuphysics 2y ago
> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.
- linguistbreaker 2y ago
  I appreciated a post on here recently that likened AI hallucination to 'bullshitting'. It's coherent, even plausible output without any regard for the truth.
  - wkat4242 2y ago
    While I have absolutely no issues with the word "shit" in popular terms, I'd normally like to reserve it for situations where there's actually intended malice like in "enshittification". Rather than just an imperfect technology as we have here. Many people object to the term enshittification for foul-mouthing reasons but I think it covers it very well because the principle it covers is itself so very nasty. But that's not at all the case here.
    - pessimizer 2y ago
      "Bullshitting" isn't a new piece of jargon, it's a common English word of many decades vintage, and is being used in its dictionary sense here.
  - MattPalmer1086 2y ago
    More true to say that all output is bullshitting, not just the ones we call hallucinations. Some of it is true, some isn't. The model doesn't know or care.
- nl 2y ago
  And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.
  - darby_nine 2y ago
    That was a misnomer—hallucination refers to perception, not generation. Completely misled an entire generation of people.
  - Raphaellll 2y ago
    "hallucination" was coined in the context of text generating RNNs. Specifically in this blog post by Karpathy in 2015: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/
exmadscientist 2y ago
"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't bad either.)
- kimixa 2y ago
  I don't really immediately link "Hallucinations" with "Unsound mind" - most people I know have experienced auditory hallucinations - often things like not sure if the doorbell went off, or if someone said their name. And I couldn't find a single one of my friends who hadn't experienced "phantom vibration syndrome". Both I'd say are "Hallucinations", without any real negative connotation.
- devjab 2y ago
  I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than that, but the essence of LLMs is that they are very good at being lucky. So good that they will often give you better results than random search engine clicks, but not good enough to be useful for anything important. I think what calling the times they get things wrong hallucinations is largely an advertising trick. So that they can sort of fit the LLMs into how all IT is sometimes “wonky” and sell their fundamentally flawed technology more easily. I also think it works extremely well.
  - Terr_ 2y ago
    To offer a satirical analogy: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: Hindenburg 2 will only contain only normal and unreactive hydrogen gas, and not the rare and unusual explosive kind, which is merely a temporary hurdle in this highly dynamic and growing field." Edit: It retrospect, perhaps a better analogy would involve gasoline, as its explosive nature is what's being actively being exploited in normal use.
wkat4242 2y ago
Hallucination is one single word. Even if it's not perfect it's great as a term. It's easy to remember and people new to the term already have an idea of what it entails. And the term will bend to cover what we take it to mean anyway. Language is flexible. Hallucination in an LLM context doesn't have to be the exact same as in a human context. All it matters is that we're aligned on what we're talking about. It's already achieved this purpose.
freilanzer 2y ago
Hallucination perfectly describes the phenomenon.
dgs_sgd 2y ago
I think calling it hallucination is because of our tendency to anthropomorphize things. Humans hallucinate. Programs have bugs.
- threeseed 2y ago
  The point is that this isn't a bug. It's inherent to how LLMs work and is expected although undesired behaviour.
TeaBrain 2y ago
The problem with "incoherent output" is that it isn't describing the phenomenon at all. There have been cases where LLM output has been incoherent, but modern LLM hallucinations are usually coherent and well-contructed, just completely fabricated.
- darby_nine 2y ago
  Do you have an example? Your statement seems trivially contradictory—how do you know it's fabricated without incoherence showing you this? Isn't fabrication the entire point of generative ai?
  - TeaBrain 2y ago
    It seems that your insistence upon using incoherence is based upon a misunderstanding of the word. Coherence does not mean to be factual, but to be logically ordered. Also, the word "fabricated" is generally used colloquially to describe made up information without factual basis.
breatheoften 2y ago
I think it's a pretty good name for the phenomenon -- maybe the only problem with the term is that what models are doing is 100% hallucination all the time -- it's just that when the hallucinations are useful we don't call them hallucinations -- so maybe that is a problem with the term (not sure if that's what you are getting at). But there's nothing at all different about what the model is doing between these cases -- the models are hallucinating all the time and have no ability to assess when they are hallucinating "right" or "wrong" or useful/non-useful output in any meaningful way.
- darby_nine 2y ago
  They aren't hallucinating in any way comparable to humans, which implies a delusion in perception. You're describing the quality of output by using a word used to describe the quality of input.
Slow_Hand 2y ago
I prefer “confabulate” to describe this phenomena. : to fill in gaps in memory by fabrication > In psychology, confabulation is a memory error consisting of the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It’s more about coming up with a plausible explanation in the absence of a readily-available one.
- skdotdan 2y ago
  This is not what hallucination means in the pre-LLM machine learning literature.
adrianmonk 2y ago
On a literal level, hallucinations are perceptual. But "hallucination" was already (before LLMs) being used in a figurative sense, i.e. for abstract ideas that are made up out of nothing. The same is also true of other words that were originally visual, like "illusion" and "mirage".
skdotdan 2y ago
It’s used incorrectly. Hallucination has (or used to have) a very specific meaning in machine learning. All hallucinations are errors but not all errors are hallucinations.
lopatin 2y ago
Why is "incoherent output" better? When an LLM hallucinates, it coherently and confidently lies to you. I think "hallucination" is the perfect word for this.

jwuphysics 2y ago

> Why not just say "incoherent output"? Because the biggest problem with hallucinations is that the output is usually coherent but factually incorrect. I agree that "hallucination" isn't the best word for it... perhaps something like "confabulation" is better.

linguistbreaker 2y ago
I appreciated a post on here recently that likened AI hallucination to 'bullshitting'. It's coherent, even plausible output without any regard for the truth.
- wkat4242 2y ago
  While I have absolutely no issues with the word "shit" in popular terms, I'd normally like to reserve it for situations where there's actually intended malice like in "enshittification". Rather than just an imperfect technology as we have here. Many people object to the term enshittification for foul-mouthing reasons but I think it covers it very well because the principle it covers is itself so very nasty. But that's not at all the case here.
  - pessimizer 2y ago
    "Bullshitting" isn't a new piece of jargon, it's a common English word of many decades vintage, and is being used in its dictionary sense here.
- MattPalmer1086 2y ago
  More true to say that all output is bullshitting, not just the ones we call hallucinations. Some of it is true, some isn't. The model doesn't know or care.
nl 2y ago
And we use "hallucination" because in the ancient times when generative AI meant image generation models would "hallucinate" extra fingers etc. The behavior of text models is similar enough that the wording stuck, and it's not all that bad.
- darby_nine 2y ago
  That was a misnomer—hallucination refers to perception, not generation. Completely misled an entire generation of people.
- Raphaellll 2y ago
  "hallucination" was coined in the context of text generating RNNs. Specifically in this blog post by Karpathy in 2015: https://karpathy.github.io/2015/05/21/rnn-effectiveness/ https://karpathy.github.io/2015/05/21/rnn-effectiveness/

exmadscientist 2y ago

"Hallucinations" implies that someone isn't of sound mental state. We can argue forever about what that means for a LLM and whether that's appropriate, but I think it's absolutely the right attitude and approach to be taking toward these things. They simply do not behave like humans of sound minds, and "hallucinations" conveys that in a way that "confabulations" or even "bullshit" does not. (Though "bullshit" isn't bad either.)

kimixa 2y ago
I don't really immediately link "Hallucinations" with "Unsound mind" - most people I know have experienced auditory hallucinations - often things like not sure if the doorbell went off, or if someone said their name. And I couldn't find a single one of my friends who hadn't experienced "phantom vibration syndrome". Both I'd say are "Hallucinations", without any real negative connotation.
devjab 2y ago
I disagree with this take because LLMs are, always, hallucinating. When they get things right it’s because they are lucky. Yes, yes, it’s more complicated than that, but the essence of LLMs is that they are very good at being lucky. So good that they will often give you better results than random search engine clicks, but not good enough to be useful for anything important. I think what calling the times they get things wrong hallucinations is largely an advertising trick. So that they can sort of fit the LLMs into how all IT is sometimes “wonky” and sell their fundamentally flawed technology more easily. I also think it works extremely well.
- Terr_ 2y ago
  To offer a satirical analogy: "Lastly, I want to reassure investors and members of the press that we take these concerns very seriously: Hindenburg 2 will only contain only normal and unreactive hydrogen gas, and not the rare and unusual explosive kind, which is merely a temporary hurdle in this highly dynamic and growing field." Edit: It retrospect, perhaps a better analogy would involve gasoline, as its explosive nature is what's being actively being exploited in normal use.

wkat4242 2y ago

Hallucination is one single word. Even if it's not perfect it's great as a term. It's easy to remember and people new to the term already have an idea of what it entails. And the term will bend to cover what we take it to mean anyway. Language is flexible. Hallucination in an LLM context doesn't have to be the exact same as in a human context. All it matters is that we're aligned on what we're talking about. It's already achieved this purpose.

freilanzer 2y ago

Hallucination perfectly describes the phenomenon.

dgs_sgd 2y ago

I think calling it hallucination is because of our tendency to anthropomorphize things. Humans hallucinate. Programs have bugs.

threeseed 2y ago
The point is that this isn't a bug. It's inherent to how LLMs work and is expected although undesired behaviour.

TeaBrain 2y ago

The problem with "incoherent output" is that it isn't describing the phenomenon at all. There have been cases where LLM output has been incoherent, but modern LLM hallucinations are usually coherent and well-contructed, just completely fabricated.

darby_nine 2y ago
Do you have an example? Your statement seems trivially contradictory—how do you know it's fabricated without incoherence showing you this? Isn't fabrication the entire point of generative ai?
- TeaBrain 2y ago
  It seems that your insistence upon using incoherence is based upon a misunderstanding of the word. Coherence does not mean to be factual, but to be logically ordered. Also, the word "fabricated" is generally used colloquially to describe made up information without factual basis.

breatheoften 2y ago

I think it's a pretty good name for the phenomenon -- maybe the only problem with the term is that what models are doing is 100% hallucination all the time -- it's just that when the hallucinations are useful we don't call them hallucinations -- so maybe that is a problem with the term (not sure if that's what you are getting at). But there's nothing at all different about what the model is doing between these cases -- the models are hallucinating all the time and have no ability to assess when they are hallucinating "right" or "wrong" or useful/non-useful output in any meaningful way.

darby_nine 2y ago
They aren't hallucinating in any way comparable to humans, which implies a delusion in perception. You're describing the quality of output by using a word used to describe the quality of input.

Slow_Hand 2y ago

I prefer “confabulate” to describe this phenomena. : to fill in gaps in memory by fabrication > In psychology, confabulation is a memory error consisting of the production of fabricated, distorted, or misinterpreted memories about oneself or the world. It’s more about coming up with a plausible explanation in the absence of a readily-available one.

skdotdan 2y ago
This is not what hallucination means in the pre-LLM machine learning literature.

adrianmonk 2y ago

On a literal level, hallucinations are perceptual. But "hallucination" was already (before LLMs) being used in a figurative sense, i.e. for abstract ideas that are made up out of nothing. The same is also true of other words that were originally visual, like "illusion" and "mirage".

skdotdan 2y ago

It’s used incorrectly. Hallucination has (or used to have) a very specific meaning in machine learning. All hallucinations are errors but not all errors are hallucinations.

lopatin 2y ago

Why is "incoherent output" better? When an LLM hallucinates, it coherently and confidently lies to you. I think "hallucination" is the perfect word for this.