4 ms·

It is still a hallucination even if the words it hallucinates happen to line up with a factual sentence, in the same way that a broken clock happens to correctl

It is still a hallucination even if the words it hallucinates happen to line up with a factual sentence, in the same way that a broken clock happens to correctly display the time twice a day. The function of the clock does not suddenly begin working correctly for one minute and then stop working correctly the next. The function of a broken clock is always flawed. Those broken outputs, by pure coincidence, just happen to be correct sometimes. LLMs are broken in the same way. They are just predictive text generators, with no real knowledge of concepts or reasoning. As it happens most of the text it has been trained on is factual, so when it regurgitates that text it is only by happenstance, not function, that it produces facts. When it hallucinates a completely new sentence by mashing its learned texts together, it's pure chance whether the resulting sentence is truthful or not. Every generation is a hallucination. Some hallucinations happen to be sentences that reflect the truth. The LLM has no ability to tell the difference.

d-z-m 3y ago
You're using a different definition of "hallucination" than the one most people use when talking about LLMs. If you want to do that that's fine, but you're definitely in the minority.
- SrslyJosh 3y ago
  Most people anthropomorphize LLMs. That doesn't make them right. It's a bad term, and one that misunderstands what LLMs are doing. An LLM is doing the exact same thing when it generates output that you consider to be a "hallucination" that it's doing when it generates output that you consider "correct".
  - simonw 3y ago
    What's your alternative suggestion for a term we can use to describe instances where an LLM produces a statement that appears to be factual (the title and authors of a paper for example) but is in fact entirely made up and doesn't reflect the real world at all?
    - dleeftink 3y ago
      Similar to cache 'hits or misses', I always thought the idea of the underlying 'knowledge cache' being exhausted (i.e. its embedding space) would fit the bill nicely. Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches. These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, wayward branching or simply perplexion. Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.
    - diputsmonro 3y ago
      A coincidence. Like, it's a bit sarcastic, sure, but until factuality is explicitly built into the model, I don't think we should use any terminology that implies that the outputs are trustworthy in any way. Until then, every output is like a lucky guess. Similar to a student flipping a coin to answer a multiple choice test. Though they get the correct answer sometimes, it says nothing at all about what they know, or how much we can trust them when we ask a question that we don't already know the answer to. Every LLM user should keep that in mind.
    - chriskanan 3y ago
      The appropriate term from psychology is confabulation. Hallucinations are misinterpreting input data, but confabulations are plausible sounding fictions. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation
- haukurb 3y ago
  It's the same definition from a talk by one of the PPO developers and also used elsewhere, i.e. it being first and foremost whether the output is inferred by applying proper epistemology (justified correct belief) to its training data. It's a bit more nuanced than simply the negation of factualness (or 'correctness'). Yes, it means proper application of the term means you have to know what went into its training data (or current context), but you'd have to make those assumptions anyway to be able to put any credence at all to any of its outputs.
insanitybit 3y ago
That's like saying that I'm hallucinating right now by reading your post and interpreting the words, it just happens to be that I'm reading your post as it is written. Most people call that "thinking".
justanotherjoe 3y ago
Just because it inputs and outputs text embeddings, doesnt mean its all text in between. Inside, it doesnt work in units of texts. You wouldnt say a blind human is just a text pattern machine cause it inputs and outputs text. Theres nothing stopping the llm to learn a rich semantic model of the real world in its 100s of billions of params

d-z-m 3y ago

You're using a different definition of "hallucination" than the one most people use when talking about LLMs. If you want to do that that's fine, but you're definitely in the minority.

SrslyJosh 3y ago
Most people anthropomorphize LLMs. That doesn't make them right. It's a bad term, and one that misunderstands what LLMs are doing. An LLM is doing the exact same thing when it generates output that you consider to be a "hallucination" that it's doing when it generates output that you consider "correct".
- simonw 3y ago
  What's your alternative suggestion for a term we can use to describe instances where an LLM produces a statement that appears to be factual (the title and authors of a paper for example) but is in fact entirely made up and doesn't reflect the real world at all?
  - dleeftink 3y ago
    Similar to cache 'hits or misses', I always thought the idea of the underlying 'knowledge cache' being exhausted (i.e. its embedding space) would fit the bill nicely. Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches. These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, wayward branching or simply perplexion. Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.
  - diputsmonro 3y ago
    A coincidence. Like, it's a bit sarcastic, sure, but until factuality is explicitly built into the model, I don't think we should use any terminology that implies that the outputs are trustworthy in any way. Until then, every output is like a lucky guess. Similar to a student flipping a coin to answer a multiple choice test. Though they get the correct answer sometimes, it says nothing at all about what they know, or how much we can trust them when we ask a question that we don't already know the answer to. Every LLM user should keep that in mind.
  - chriskanan 3y ago
    The appropriate term from psychology is confabulation. Hallucinations are misinterpreting input data, but confabulations are plausible sounding fictions. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation
haukurb 3y ago
It's the same definition from a talk by one of the PPO developers and also used elsewhere, i.e. it being first and foremost whether the output is inferred by applying proper epistemology (justified correct belief) to its training data. It's a bit more nuanced than simply the negation of factualness (or 'correctness'). Yes, it means proper application of the term means you have to know what went into its training data (or current context), but you'd have to make those assumptions anyway to be able to put any credence at all to any of its outputs.

insanitybit 3y ago

That's like saying that I'm hallucinating right now by reading your post and interpreting the words, it just happens to be that I'm reading your post as it is written. Most people call that "thinking".

justanotherjoe 3y ago

Just because it inputs and outputs text embeddings, doesnt mean its all text in between. Inside, it doesnt work in units of texts. You wouldnt say a blind human is just a text pattern machine cause it inputs and outputs text. Theres nothing stopping the llm to learn a rich semantic model of the real world in its 100s of billions of params