4 ms·

Am I the only one who is bothered by calling this phenomenon "hallucinating"? It's marketing-speak and corporate buzzwords to cover for the fact that their LLM

Am I the only one who is bothered by calling this phenomenon "hallucinating"? It's marketing-speak and corporate buzzwords to cover for the fact that their LLMs often produced wrong information because they aren't capable of understanding your request, nuance, or the training data it used is wrong, or the model just plain sucks. Would we tolerate such doublespeak it were anything else? "Well, you ordered a side of fries with your burger but because our wait staff made a mistake...sorry, hallucinated, they brought you a peanut butter sandwich that's growing mold instead." It gets more concerning when the stakes are raised. When LLMs (inevitably) start getting used in more important contexts, like healthcare. "I know your file says you're allergic to penicillin and you repeated when talking to our ai-doctor but it hallucinated that you weren't."

samatman 3y ago
You're not the only one. I will continue to fight the losing battle for "confabulation" for as long as the problem remains current.
deleted 3y ago
[deleted]
altairprime 3y ago
Human beings regularly hallucinate details that aren’t real when asked to provide their memories of an event, and often don’t realize they’re doing it at all. So whole AI definitely is lacking in the “can assess fact versus fiction” department, that’s an overlapping problem with “invents things that aren’t actually real”. It can, today, hallucinate accurate and inaccurate information, but it can’t determine validity at all, so it’s sometimes wrong even when not hallucinating.
IshKebab 3y ago
Nonsense. It isn't marketing speak to cover for anything. It's a pretty good description of what is happening. The reason models hallucinate is because we train them to produce linguistically plausible output, which usually overlaps well with factually correct output (because it wouldn't be plausible to say e.g. "Barack Obama is white"). But when there isn't much data to show that something that is totally made up is implausible then there's no penalty to the model for it. It's nothing to do with not being able to understand your request, and it's rarely because the training data is wrong.
- dartos 3y ago
  "Hallucinate" is definitely marketing. it translates to "Creates text which contains incorrect or invalid information" The latter just doesn't sound as good in headlines/articles/tutorials (eg. marketing material).
  - ryandrake 3y ago
    We already have words for when a computer program produces unexpected/incorrect output: “defect” and “bug”
    - dartos 3y ago
      The weird thing is, it’s not a bug of software, it’s a limitation. The software is working as designed, statistics are just imperfect
  - ben_w 3y ago
    It's a term of art from the days of image recognition AI that would confidently report seeing a giraffe while looking at a picture of an ambulance. It doesn't feel right to me either, to use it in the context of generative AI, and I'd support renaming this behaviour in GenAI (text and images both) — though myself I'd call this behaviour "mis-remembering". Edit: apparently some have suggested "delusion". That also works for me.
  - lucianbr 3y ago
    So if I replied to your comment with "you are incorrect" I would be putting you in a worse light than saying "you are hallucinating"? The second is making it sound better? Doesn't feel that way to me.
    - JohnFen 3y ago
      My problem with "hallucination" isn't that it makes error sound better or worse, it's that it makes it sound like there's a consciousness involved when there isn't.
  - 3y ago
lucianbr 3y ago
To me it sounds pretty damning. "The tool hallucinates" makes me think it's completely out of touch with reality, spouting nonsense. While "It has made a mistake, it is factually incorrect" would apply to many of my comments if taken very literally. Webster definition: "a sensory perception (such as a visual image or a sound) that occurs in the absence of an actual external stimulus and usually arises from neurological disturbance (such as that associated with delirium tremens, schizophrenia, Parkinson's disease, or narcolepsy) or in response to drugs (such as LSD or phencyclidine)". I would fire with prejudice any marketing department that associated our product with "delirium tremens, schizophrenia, [...] LSD or phencyclidine".
ToValueFunfetti 3y ago
I don't get this at all. "Hallucinate" to me only can mean "produce false information". I've only ever seen it used perjoratively re: AI, and I don't understand what it covers up- how else are people interpreting it? I could see the point if you were saying that it implies sentience that isn't there, but your analogy to a restaurant implies that's not what you're getting at.
bcrosby95 3y ago
> Would we tolerate such doublespeak it were anything else? Yes: identity theft. My identity wasn't "stolen", what really happened was a company gave a bad loan. But calling it identity theft shifts the blame. Now it's my job to keep my data "safe", not their job to make sure they're giving the right person the loan.
oaktowner 3y ago
I can't stand it being called "hallucinating" because it anthropomorphizes the technology. This isn't a conciousness that is "seeing" things that don't exist: it's a word generator that is generating words that don't make sense (not in a syntactic sense, but in a semantic sense). Calling it "hallucination" implies that there are (other) moments when it is understanding the world correctly -- and that itself is not true. At those moments, it is a word generator that is generating words that DO make sense. At no point is this a conciousness, and anthropomorphizing it gives the impression that it is one.
- JohnFen 3y ago
  This. It's not "hallucination", it's "error".
  - krapp 3y ago
    It isn't an error, either. It's doing exactly what it's intended to, exactly as it's intended to do it. The error is in the human assumption that the ability to construct syntactically coherent language signals self-awareness or sentience. That it should be capable of understanding the semantics correctly, because humans obviously can. There really is no correct word to describe what's happening, because LLMs are effectively philosophical zombies. We have no metaphors for an entity that can appear to hold a coherent conversation, do useful work and respond to commands but not think. All we have is metaphors from human behavior which presume the connection between language and intellect, because that's all we know. Unfortunately we also have nearly a century of pop culture telling us "AI" is like Data from Star Trek, perfectly logical, superintelligent and always correct. And "hallucination" is good enough. It gets the point across, that these things can't be trusted. "Confabulation" would be better, but fewer people know it, and it's more important to communicate the untrustworthy nature of LLMs to the masses than it is to be technically precise.
    - JohnFen 3y ago
      > It isn't an error, either. It's doing exactly what it's intended to, exactly as it's intended to do it. If the output is incorrect, that's error. It may not be a bug, but it is still error.
programjames 3y ago
I think people are much more conservative with their health than text generation. If the text looks funky, you can just try regenerating it, or write it yourself and have only lost a few minutes. If your health starts looking funky, you're kind of screwed.

samatman 3y ago

You're not the only one. I will continue to fight the losing battle for "confabulation" for as long as the problem remains current.

deleted 3y ago

[deleted]

altairprime 3y ago

Human beings regularly hallucinate details that aren’t real when asked to provide their memories of an event, and often don’t realize they’re doing it at all. So whole AI definitely is lacking in the “can assess fact versus fiction” department, that’s an overlapping problem with “invents things that aren’t actually real”. It can, today, hallucinate accurate and inaccurate information, but it can’t determine validity at all, so it’s sometimes wrong even when not hallucinating.

IshKebab 3y ago

Nonsense. It isn't marketing speak to cover for anything. It's a pretty good description of what is happening. The reason models hallucinate is because we train them to produce linguistically plausible output, which usually overlaps well with factually correct output (because it wouldn't be plausible to say e.g. "Barack Obama is white"). But when there isn't much data to show that something that is totally made up is implausible then there's no penalty to the model for it. It's nothing to do with not being able to understand your request, and it's rarely because the training data is wrong.

dartos 3y ago
"Hallucinate" is definitely marketing. it translates to "Creates text which contains incorrect or invalid information" The latter just doesn't sound as good in headlines/articles/tutorials (eg. marketing material).
- ryandrake 3y ago
  We already have words for when a computer program produces unexpected/incorrect output: “defect” and “bug”
  - dartos 3y ago
    The weird thing is, it’s not a bug of software, it’s a limitation. The software is working as designed, statistics are just imperfect
- ben_w 3y ago
  It's a term of art from the days of image recognition AI that would confidently report seeing a giraffe while looking at a picture of an ambulance. It doesn't feel right to me either, to use it in the context of generative AI, and I'd support renaming this behaviour in GenAI (text and images both) — though myself I'd call this behaviour "mis-remembering". Edit: apparently some have suggested "delusion". That also works for me.
- lucianbr 3y ago
  So if I replied to your comment with "you are incorrect" I would be putting you in a worse light than saying "you are hallucinating"? The second is making it sound better? Doesn't feel that way to me.
  - JohnFen 3y ago
    My problem with "hallucination" isn't that it makes error sound better or worse, it's that it makes it sound like there's a consciousness involved when there isn't.
- 3y ago

lucianbr 3y ago

To me it sounds pretty damning. "The tool hallucinates" makes me think it's completely out of touch with reality, spouting nonsense. While "It has made a mistake, it is factually incorrect" would apply to many of my comments if taken very literally. Webster definition: "a sensory perception (such as a visual image or a sound) that occurs in the absence of an actual external stimulus and usually arises from neurological disturbance (such as that associated with delirium tremens, schizophrenia, Parkinson's disease, or narcolepsy) or in response to drugs (such as LSD or phencyclidine)". I would fire with prejudice any marketing department that associated our product with "delirium tremens, schizophrenia, [...] LSD or phencyclidine".

ToValueFunfetti 3y ago

I don't get this at all. "Hallucinate" to me only can mean "produce false information". I've only ever seen it used perjoratively re: AI, and I don't understand what it covers up- how else are people interpreting it? I could see the point if you were saying that it implies sentience that isn't there, but your analogy to a restaurant implies that's not what you're getting at.

bcrosby95 3y ago

> Would we tolerate such doublespeak it were anything else? Yes: identity theft. My identity wasn't "stolen", what really happened was a company gave a bad loan. But calling it identity theft shifts the blame. Now it's my job to keep my data "safe", not their job to make sure they're giving the right person the loan.

oaktowner 3y ago

I can't stand it being called "hallucinating" because it anthropomorphizes the technology. This isn't a conciousness that is "seeing" things that don't exist: it's a word generator that is generating words that don't make sense (not in a syntactic sense, but in a semantic sense). Calling it "hallucination" implies that there are (other) moments when it is understanding the world correctly -- and that itself is not true. At those moments, it is a word generator that is generating words that DO make sense. At no point is this a conciousness, and anthropomorphizing it gives the impression that it is one.

JohnFen 3y ago
This. It's not "hallucination", it's "error".
- krapp 3y ago
  It isn't an error, either. It's doing exactly what it's intended to, exactly as it's intended to do it. The error is in the human assumption that the ability to construct syntactically coherent language signals self-awareness or sentience. That it should be capable of understanding the semantics correctly, because humans obviously can. There really is no correct word to describe what's happening, because LLMs are effectively philosophical zombies. We have no metaphors for an entity that can appear to hold a coherent conversation, do useful work and respond to commands but not think. All we have is metaphors from human behavior which presume the connection between language and intellect, because that's all we know. Unfortunately we also have nearly a century of pop culture telling us "AI" is like Data from Star Trek, perfectly logical, superintelligent and always correct. And "hallucination" is good enough. It gets the point across, that these things can't be trusted. "Confabulation" would be better, but fewer people know it, and it's more important to communicate the untrustworthy nature of LLMs to the masses than it is to be technically precise.
  - JohnFen 3y ago
    > It isn't an error, either. It's doing exactly what it's intended to, exactly as it's intended to do it. If the output is incorrect, that's error. It may not be a bug, but it is still error.

programjames 3y ago

I think people are much more conservative with their health than text generation. If the text looks funky, you can just try regenerating it, or write it yourself and have only lost a few minutes. If your health starts looking funky, you're kind of screwed.