4 ms·

Most people anthropomorphize LLMs. That doesn't make them right. It's a bad term, and one that misunderstands what LLMs are doing. An LLM is doing the exact sa

Most people anthropomorphize LLMs. That doesn't make them right. It's a bad term, and one that misunderstands what LLMs are doing. An LLM is doing the exact same thing when it generates output that you consider to be a "hallucination" that it's doing when it generates output that you consider "correct".

simonw 3y ago
What's your alternative suggestion for a term we can use to describe instances where an LLM produces a statement that appears to be factual (the title and authors of a paper for example) but is in fact entirely made up and doesn't reflect the real world at all?
- dleeftink 3y ago
  Similar to cache 'hits or misses', I always thought the idea of the underlying 'knowledge cache' being exhausted (i.e. its embedding space) would fit the bill nicely. Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches. These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, wayward branching or simply perplexion. Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.
- diputsmonro 3y ago
  A coincidence. Like, it's a bit sarcastic, sure, but until factuality is explicitly built into the model, I don't think we should use any terminology that implies that the outputs are trustworthy in any way. Until then, every output is like a lucky guess. Similar to a student flipping a coin to answer a multiple choice test. Though they get the correct answer sometimes, it says nothing at all about what they know, or how much we can trust them when we ask a question that we don't already know the answer to. Every LLM user should keep that in mind.
- chriskanan 3y ago
  The appropriate term from psychology is confabulation. Hallucinations are misinterpreting input data, but confabulations are plausible sounding fictions. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation

simonw 3y ago

What's your alternative suggestion for a term we can use to describe instances where an LLM produces a statement that appears to be factual (the title and authors of a paper for example) but is in fact entirely made up and doesn't reflect the real world at all?

dleeftink 3y ago
Similar to cache 'hits or misses', I always thought the idea of the underlying 'knowledge cache' being exhausted (i.e. its embedding space) would fit the bill nicely. Another way of framing it would be along the lines of 'catastrophic backtracking' but attenuated: a transformer attention head veering off the beaten path due to query/parameter mismatches. These are by no means exhaustive or complete, but I would suggest knowledge exhaustion, stochastic backtracking, wayward branching or simply perplexion. Verbiage along the lines of misconstrue, fabricate and confabulate have anecdotally been used to describe this state of perplexity.
diputsmonro 3y ago
A coincidence. Like, it's a bit sarcastic, sure, but until factuality is explicitly built into the model, I don't think we should use any terminology that implies that the outputs are trustworthy in any way. Until then, every output is like a lucky guess. Similar to a student flipping a coin to answer a multiple choice test. Though they get the correct answer sometimes, it says nothing at all about what they know, or how much we can trust them when we ask a question that we don't already know the answer to. Every LLM user should keep that in mind.
chriskanan 3y ago
The appropriate term from psychology is confabulation. Hallucinations are misinterpreting input data, but confabulations are plausible sounding fictions. https://en.m.wikipedia.org/wiki/Confabulation https://en.m.wikipedia.org/wiki/Confabulation