7 ms·

> the hallucination phenomena There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tell

by hbn 2y ago

> the hallucination phenomena There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. But sometimes we can tell either immediately or through fact checking it spat out a string of text that claims something incorrect. If you start asking an LLM for political takes, you'll get very different answers from humans about which ones are "hallucinations"

mortenjorck 2y ago
It is an unfortunately anthropomorphizing term for a transformer simply operating as designed, but the thing it's become a vernacular shorthand for, "outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. There's definitely room for a better label, though. "Empirical mismatch" doesn't quite have the same ring as "hallucination," but it's probably a more accurate place to start from.
- NovemberWhiskey 2y ago
  >"outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. Sure, but that would require semantic mechanisms rather than statistical ones.
  - riwsky 2y ago
    Statistics has a semantics all its own
- hbn 2y ago
  Regardless I don't think there's much to write papers on, other than maybe an anthropological look at how it's affected people putting too much trust into LLMs for research, decision-making, etc. If someone wants info to make their model to be more reliable for a specific domain, it's in the existing papers on model training.
  - deleted 2y ago
    [deleted]
emporas 2y ago
Chess engines, which are used for 25 years by the best human chess players daily, compute the best next move on the board. The total number of all possible chess positions is more than all the atoms in the universe. Is is possible for a chess engine to compute the next move and be absolutely sure it is the best one? It's not, it is a statistical approximation, but still very useful.
deleted 2y ago
[deleted]
raincole 2y ago
I don't know why the narrative became "don't call it hallucination". Grantly English isn't my mother tongue so I might miss some subtlty here. If you know how LLM works, call it "hallucination" doesn't make you know less. If you don't know how LLM works, using "hallucination" doesn't make you know less either. It's just a word meaning AI gives wrong[1] answer. People say it's "anthropomorphizing" but honestly I can't see it. The I in AI stands for intelligence, is this anthropomorphizing? L in ML? Reading and writing are clearly human activities, so is using read/write instead of input/output anthropomorphizing? How about "computer", a word once meant a human who does computing? Is there a word we can use safely without anthropomorphizing? [1]: And please don't argue what's "wrong".
- th0ma5 2y ago
  AI is a nebulous, undefined term, and many people specifically criticize the use of the word intelligent.
  - Affric 2y ago
    Always people who consider themselves intelligent
- Affric 2y ago
  The AI companies don’t want you “anthropomorphising” the models because it would put them at risk of increased liability. You will be told that linear algebra is just a model and the fact that epistemology has never turned up a decent result for what knowledge is will be ignored. We are meant to believe that we are somehow special magical creatures and that the behaviour of our minds cannot be modelled by linear algebra.
  - duggan 2y ago
    No, you’re just meant not to assert that linear algebra is equivalent to any process in the human brain, when the human brain is not understood well enough to draw that conclusion.
  - ben_w 2y ago
sandworm101 2y ago
Hallucination is emergent. It cannot be found as a thing inside the AI systems. It is a phenomena that only exists when the output is evaluated. That makes it an accurate description. A human who has hallucinated something is not lying when they speak of something that never actually happened, nor are they making any sort of mistake in their recollection. Similarly, an AI that is hallucinating isn't doing anything incorrect and doesn't have any motivation. The hallucinated data emerges just as any other output, only to evaluated by outsiders as incorrect.
- deleted 2y ago
  [deleted]
IanCal 2y ago
It's the new "serverless" and I would really like people to stop making the discussion between about the word. You know what it means, I know what it means, let's all move on. We won't, and we'll see this constant distraction.
- CaptainOfCoit 2y ago
  > It's the new "serverless" and I would really like people to stop making the discussion between about the word. You know what it means, I know what it means, let's all move on. Well, parent is lamenting the lack of lowerbound/upperbound for "hallucinations", something that cannot realistically exist as "hallucinations" don't exist. LLMs aren't fact-outputting machines, so when it outputs something a human would consider "wrong" like "the sky is purple", it isn't true/false/correct/incorrect/hallucination/fact, it's just the most probable character after the next. That's why it isn't useful to ask "but how much it hallucinates?" when in reality what you're out after is something more like "does it only output facts?". Which, if it did, LLMs would be a lot less useful.
  - elif 2y ago
    There is a huge gap between "facts" and "nonfacts" which compose the majority of human discourse. Statements, opinions, questions, when properly qualified, are not facts or nonfacts or hallucinations. LLM don't need to be perfect fact machines at all to be honest, and non-hallucinating. They simply need to ground statements in other grounded statements and identify the parts which are speculative or non-grounded.
    - kreyenborgi 2y ago
      If you simply want to ground statements in statements, you quickly get into GOFAI territory where you need to build up the full semantics of a sentence (in all supported languages) in order to prove that two sentences mean the same or have the same denotation or that one entails the other. Otherwise, how do you prove the grounding isn't "hallucinated"?
hatthew 2y ago
LLMs model their corpus, which for most models tends to be factually correct text (or subjective text with no factuality). Sure, there exist factually incorrect statements in the corpus, but for the vast majority of incorrect statements there exist many more equivalent but correct statements. If an LLM makes a statement that is not supported by the training data (either because it doesn't exist or because the equivalent correct statement is more strongly supported), I think that's an issue with the implementation of the model. I don't think it's an intrinsic feature/flaw in what the model is modeling. Hallucination might not be the best word, but I don't think it's a bad word. If a weather model predicted a storm when there isn't a cloud in the sky, I wouldn't have a problem with saying "the weather model had a hallucination." 50 years ago, weather models made incorrect predictions quite frequently. That's not because they weren't modeling correct weather, it's because we simply didn't yet have good models and clean data. Fundamentally, we could fix most LLM hallucinations with better model implementations and cleaner data. In the future we will probably be able to model factuality outside of the context of human language, and that will probably be the ultimate solution for correctness in AI, but I don't think that's a fundamental requirement.
- intended 2y ago
  I suspect you would fix first response accuracy. People still want it to be used for thinking. This isnt going to happen with better data. Better data means it will be better at predicting the next token. For questions or interactions where you need to process, consider, decompose a problem into multiple steps, solve those steps etc - you need to have a goal, tools, and the ability to split your thinking and govern the outcome. That isnt predicting the next token. I think it’s easier to think of LLMs as doing decompression. They take an initial set of tokens and decompress them into the most likely final set of tokens. What we want is processing. We would have to set up the reaction to somehow perfectly result in the next set of tokens to then set up the next set of tokens etc - till the system has an answer. Or in other words, we have to figure out how to phrase an initial set of tokens so that each subsequent set looks similar enough to “logic” in the training data, that the LLM expands correctly.
shrimp_emoji 2y ago
It should be "confabulation", since that's not carting along the notion of false sensory input. Humans also confabulate but not as a result of "hallucinations". They usually do it because that's actually what brains like to do, whether it's making up stories about how the world was created or, more infamously, in the case of neural disorders where the machinery's penchant for it becomes totally unmoderated and a person just spits out false information that they themselves can't realize is false. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation
nl 2y ago
> There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. This is a very "closed world" view of the phenomenon which looks at an LLM as a software component on its own. But "hallucination" is a user experience problem, and it describes the experience very well. If you are using a code assistant and it suggests using APIs that don't exist then the word "hallucination" is entirely appropriate. A vaguely similar analogy is the addition of the `let` and `const` keywords in JS ES6. While the behavior of `var` was "correct" as-per spec the user experience was horrible: bug prone and confusing.
sqeaky 2y ago
Yet for their value as tools the truth value of statements made by LLMs do matter.
wincy 2y ago
Well what the heck was Bing Chat doing when it wrote me a message all in emojis like it was the Zodiac killer telling me a hacker had taken it over then spitting out Python code to shutdown the system, and giving me nonsense secret messages like “PKCLDUBB”? What am I suppose to call that?

by hbn 2y ago

mortenjorck 2y ago

It is an unfortunately anthropomorphizing term for a transformer simply operating as designed, but the thing it's become a vernacular shorthand for, "outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. There's definitely room for a better label, though. "Empirical mismatch" doesn't quite have the same ring as "hallucination," but it's probably a more accurate place to start from.

NovemberWhiskey 2y ago
>"outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. Sure, but that would require semantic mechanisms rather than statistical ones.
- riwsky 2y ago
  Statistics has a semantics all its own
hbn 2y ago
Regardless I don't think there's much to write papers on, other than maybe an anthropological look at how it's affected people putting too much trust into LLMs for research, decision-making, etc. If someone wants info to make their model to be more reliable for a specific domain, it's in the existing papers on model training.
- deleted 2y ago
  [deleted]

emporas 2y ago

Chess engines, which are used for 25 years by the best human chess players daily, compute the best next move on the board. The total number of all possible chess positions is more than all the atoms in the universe. Is is possible for a chess engine to compute the next move and be absolutely sure it is the best one? It's not, it is a statistical approximation, but still very useful.

deleted 2y ago

[deleted]

raincole 2y ago

I don't know why the narrative became "don't call it hallucination". Grantly English isn't my mother tongue so I might miss some subtlty here. If you know how LLM works, call it "hallucination" doesn't make you know less. If you don't know how LLM works, using "hallucination" doesn't make you know less either. It's just a word meaning AI gives wrong[1] answer. People say it's "anthropomorphizing" but honestly I can't see it. The I in AI stands for intelligence, is this anthropomorphizing? L in ML? Reading and writing are clearly human activities, so is using read/write instead of input/output anthropomorphizing? How about "computer", a word once meant a human who does computing? Is there a word we can use safely without anthropomorphizing? [1]: And please don't argue what's "wrong".

th0ma5 2y ago
AI is a nebulous, undefined term, and many people specifically criticize the use of the word intelligent.
- Affric 2y ago
  Always people who consider themselves intelligent
Affric 2y ago
The AI companies don’t want you “anthropomorphising” the models because it would put them at risk of increased liability. You will be told that linear algebra is just a model and the fact that epistemology has never turned up a decent result for what knowledge is will be ignored. We are meant to believe that we are somehow special magical creatures and that the behaviour of our minds cannot be modelled by linear algebra.
- duggan 2y ago
  No, you’re just meant not to assert that linear algebra is equivalent to any process in the human brain, when the human brain is not understood well enough to draw that conclusion.
- ben_w 2y ago

sandworm101 2y ago

Hallucination is emergent. It cannot be found as a thing inside the AI systems. It is a phenomena that only exists when the output is evaluated. That makes it an accurate description. A human who has hallucinated something is not lying when they speak of something that never actually happened, nor are they making any sort of mistake in their recollection. Similarly, an AI that is hallucinating isn't doing anything incorrect and doesn't have any motivation. The hallucinated data emerges just as any other output, only to evaluated by outsiders as incorrect.

deleted 2y ago
[deleted]

IanCal 2y ago

It's the new "serverless" and I would really like people to stop making the discussion between about the word. You know what it means, I know what it means, let's all move on. We won't, and we'll see this constant distraction.

CaptainOfCoit 2y ago
> It's the new "serverless" and I would really like people to stop making the discussion between about the word. You know what it means, I know what it means, let's all move on. Well, parent is lamenting the lack of lowerbound/upperbound for "hallucinations", something that cannot realistically exist as "hallucinations" don't exist. LLMs aren't fact-outputting machines, so when it outputs something a human would consider "wrong" like "the sky is purple", it isn't true/false/correct/incorrect/hallucination/fact, it's just the most probable character after the next. That's why it isn't useful to ask "but how much it hallucinates?" when in reality what you're out after is something more like "does it only output facts?". Which, if it did, LLMs would be a lot less useful.
- elif 2y ago
  There is a huge gap between "facts" and "nonfacts" which compose the majority of human discourse. Statements, opinions, questions, when properly qualified, are not facts or nonfacts or hallucinations. LLM don't need to be perfect fact machines at all to be honest, and non-hallucinating. They simply need to ground statements in other grounded statements and identify the parts which are speculative or non-grounded.
  - kreyenborgi 2y ago
    If you simply want to ground statements in statements, you quickly get into GOFAI territory where you need to build up the full semantics of a sentence (in all supported languages) in order to prove that two sentences mean the same or have the same denotation or that one entails the other. Otherwise, how do you prove the grounding isn't "hallucinated"?

hatthew 2y ago

LLMs model their corpus, which for most models tends to be factually correct text (or subjective text with no factuality). Sure, there exist factually incorrect statements in the corpus, but for the vast majority of incorrect statements there exist many more equivalent but correct statements. If an LLM makes a statement that is not supported by the training data (either because it doesn't exist or because the equivalent correct statement is more strongly supported), I think that's an issue with the implementation of the model. I don't think it's an intrinsic feature/flaw in what the model is modeling. Hallucination might not be the best word, but I don't think it's a bad word. If a weather model predicted a storm when there isn't a cloud in the sky, I wouldn't have a problem with saying "the weather model had a hallucination." 50 years ago, weather models made incorrect predictions quite frequently. That's not because they weren't modeling correct weather, it's because we simply didn't yet have good models and clean data. Fundamentally, we could fix most LLM hallucinations with better model implementations and cleaner data. In the future we will probably be able to model factuality outside of the context of human language, and that will probably be the ultimate solution for correctness in AI, but I don't think that's a fundamental requirement.

intended 2y ago
I suspect you would fix first response accuracy. People still want it to be used for thinking. This isnt going to happen with better data. Better data means it will be better at predicting the next token. For questions or interactions where you need to process, consider, decompose a problem into multiple steps, solve those steps etc - you need to have a goal, tools, and the ability to split your thinking and govern the outcome. That isnt predicting the next token. I think it’s easier to think of LLMs as doing decompression. They take an initial set of tokens and decompress them into the most likely final set of tokens. What we want is processing. We would have to set up the reaction to somehow perfectly result in the next set of tokens to then set up the next set of tokens etc - till the system has an answer. Or in other words, we have to figure out how to phrase an initial set of tokens so that each subsequent set looks similar enough to “logic” in the training data, that the LLM expands correctly.

shrimp_emoji 2y ago

It should be "confabulation", since that's not carting along the notion of false sensory input. Humans also confabulate but not as a result of "hallucinations". They usually do it because that's actually what brains like to do, whether it's making up stories about how the world was created or, more infamously, in the case of neural disorders where the machinery's penchant for it becomes totally unmoderated and a person just spits out false information that they themselves can't realize is false. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation

nl 2y ago

> There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. This is a very "closed world" view of the phenomenon which looks at an LLM as a software component on its own. But "hallucination" is a user experience problem, and it describes the experience very well. If you are using a code assistant and it suggests using APIs that don't exist then the word "hallucination" is entirely appropriate. A vaguely similar analogy is the addition of the `let` and `const` keywords in JS ES6. While the behavior of `var` was "correct" as-per spec the user experience was horrible: bug prone and confusing.

sqeaky 2y ago

Yet for their value as tools the truth value of statements made by LLMs do matter.

wincy 2y ago

Well what the heck was Bing Chat doing when it wrote me a message all in emojis like it was the Zodiac killer telling me a hacker had taken it over then spitting out Python code to shutdown the system, and giving me nonsense secret messages like “PKCLDUBB”? What am I suppose to call that?