6 ms·

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof

> By establishing the mathematical certainty of hallucinations, we challenge the prevailing notion that they can be fully mitigated Having a mathematical proof is nice, but honestly this whole misunderstanding could have been avoided if we'd just picked a different name for the concept of "producing false information in the course of generating probabilistic text". "Hallucination" makes it sound like something is going awry in the normal functioning of the model, which subtly suggests that if we could just identify what went awry we could get rid of the problem and restore normal cognitive function to the LLM. The trouble is that the normal functioning of the model is simply to produce plausible-sounding text. A "hallucination" is not a malfunction of the model, it's a value judgement we assign to the resulting text. All it says is that the text produced is not fit for purpose. Seen through that lens it's obvious that mitigating hallucinations and creating "alignment" are actually identical problems, and we won't solve one without the other.

gwervc 2y ago
This comment should be pinned at the top of any LLM-related comment section.
- Moru 2y ago
  It should be part of every AI related story on the news. Just like they keep saying "X, formerly Twitter".
- elif 2y ago
  Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.
  - lolinder 2y ago
    Neologism undersells what this term is being used for. It's a technical term of art that's created its own semantic category in LLM research that separates "text generated that is factually inaccurate according to ${sources}" from "text generated that is morally repugnant to ${individuals}" or "text generated that ${governments} want to censor". These three categories are entirely identical at a technological level, so I think it's entirely reasonable to flag that serious LLM researchers are treating them as distinct categories of problems when they're fundamentally not at all distinct. This isn't just a case of linguistic pedantry, this is a case of the language actively impeding a proper understanding of the problem by the researchers who are working on that problem.
  - darby_nine 2y ago
    > This is the nature of language evolution. Only if it sticks. Hallucination is such an unnatural term for the phenomenon I would be surprised to see it stick. > Everyone knows what hallucination means with respect to AI This is false. It took me months to realize this just meant "output incoherent with reality" rather than an issue with training—the natural place for perceptual errors to occur.
  - AStonesThrow 2y ago
    "Hallucination" is derogatory and insulting when aimed at normal people who hear normal voices which don't necessarily belong to corporeal beings, or originate in the natural world. Labeling "hallucinations" and pathologizing, then medicating them, constitutes assault and bigotry. "Hallucination" applied to inanimate and non-sentient software is insulting and presumptive on a different level. I'm fine with "confabulation".
wrs 2y ago
Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does. If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whose objective truth is unknown or undefinable. I haven’t read the paper yet, but if they resolve this definition usefully, that would be a good contribution.
- diputsmonro 2y ago
  Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be truthful, but there is absolutely no guarantee or any function that even attempts to enforce that. All responses are hallucinations. Some hallucinations happen to overlap the truth.
  - fumeux_fume 2y ago
    Ok, but I think it would be more productive to educate people that LLMs have no concept of truth rather than insist they use the term "hallucinate" in an unintuitive way.
    - lolinder 2y ago
      I don't know about OP, but I'm suggesting that the term 'hallucinate' be abolished entirely as applies to LLMs, not redefined. It draws an arbitrary line in the middle of the set of problems that all amount to "how do we make sure that the output of an LLM is consistently acceptable" and will all be solved using the same techniques if at all.
    - aeternum 2y ago
      LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true. I think the problem is that humanity has a poor concept of truth. We think of most things as true or not true when much of our reality is uncertain due to fundamental limitations or because we often just don't know yet. During covid for example humanity collectively hallucinated the importance of disinfecting groceries for awhile.
Terr_ 2y ago
"So we built a blind-guessing machine, but how can we tweak it so that its blind guesses always happen to be good?"
durumu 2y ago
I think there's a useful distinction between plausible-seeming text that is wrong in some subtle way, vs text that is completely fabricated to match a superficial output format, and the latter is what I wish people used "hallucination" to mean. A clear example of this is when you ask an LLM for some sources, with ISBNs, and it just makes up random titles and ISBNs that it knows full well do not correspond with reality. If you ask "Did you just make that up?" the LLM will respond with something like "Sorry, yes, I made that up, I actually just remembered I can't cite direct sources." I wonder if this is because RLHF teaches the LLM that humans in practice prefer properly formatted fake output over truthful refusals?
- majormajor 2y ago
  How does a model "know full well" that it output a fake ISBN? It's been trained that sources look like plausible-titles + random numbers. It's been trained that when challenged it should say "oh sorry I can't do this." Are those things actually distinct?
- rerdavies 2y ago
  In fairness, they will also admit they were wrong even if they were right.
paulddraper 2y ago
It's an inaccuracy.
renjimen 2y ago
Maybe with vanilla LLMs, but new LLM training paradigms include post-training with the explicit goal of avoiding over-confident answers to questions the LLM should not be confident about answering. So hallucination is a malfunction, just like any overconfident incorrect prediction by a model.
- aprilthird2021 2y ago
  I still think OP has a point. The LLMs evolved after public use to be positioned as oracles which know so much knowledge. They were always probabilistic content generators, but people use them the way they use search engines, to retrieve info they know exists but don't exactly know. Since LLMs aren't designed for this there's a whole post process to try to make them amenable to this use case, but it will never plug that gap
  - ta8645 2y ago
    > but it will never plug that gap They don't have to be perfect, they just have to be better than humans. And that seems very likely to be achievable eventually.
    - AlexandrB 2y ago
      To be better than humans they have to able confidently say "I don't know" when the correct answer is not available[1]. To me this sounds like a totally different type of "knowledge" than stringing words together based on a training set. [1] LLMs are already better than humans in terms of breadth, and sometimes depth, of knowledge. So it's not a problem of the AI knowing more facts.
      - ta8645 2y ago
        > To me this sounds like a totally different type of "knowledge" than stringing words together based on a training set. We're desperate to keep seeing ourselves as unique with key distinguishing features that are unreproducible in silicon, but from my long experience with computer chess, every step along the way, people were explaining patiently how computers could never reproduce the next quality that set humans apart. And it was always just wishful thinking, because computers eventually stopped looking silly, stopped looking like they were playing by rote, and started to produce "beautiful" chess games. And it will happen again, after the next leap in AI, people will again latch on to whatever it is that AI systems still lack, and use it to explain how they'll "always" be lacking... only to eventually be disappointed again that silicon can in fact reach that height too. Humans aren't magic. Whatever we can do, silicon can do too. It's just a matter of time.
BurningFrog 2y ago
"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.
- leptons 2y ago
  Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is divorced from the reality of what an LLM does. Whoever called it "hallucinating" in the context of an LLM should have their nerd credentials revoked.
  - imchillyb 2y ago
    Human beings have a tendency to prefer comforting lies over uncomfortable truths. "The truth may set you free, but first it's really gonna piss you off." -G.S.
  - AnimalMuppet 2y ago
    Human readable, makes grammatical sense, and wrong. And no, that's often not subjective.
    - leptons 2y ago
      It's very subjective. An LLM could return statements like "Global warming is real and man-made", and it also could produce a result like "Global warming is a hoax", and it's definitely up to the reader as to whether the LLM is "hallucinating". It doesn't matter how readable or grammatically correct the LLM is, it's still up to the reader to call bullshit, or not.
      - BurningFrog 2y ago
        If you ask about opinions, sure. Because there are no "true" opinions. If you ask about the capital of France, any answer but "Paris" is objectively wrong, whether given by a human or LLM.
nativeit 2y ago
I don’t know who/how the term was initially coined in this context, but I’m concerned that the things that make it inaccurate are also, perhaps counterintuitively, things that serve the interests of those who would overstate the capabilities of LLMs, and seek to cloud their true nature (along with inherent limitations) to investors and potential buyers. As you already pointed out, the term implies that the problems that are represented are temporary “bugs”, rather than symptoms of the underlying nature of the technology itself.
rerdavies 2y ago
How different things would be if the phenomenon had been called "makin' stuff up" instead. Humans make stuff up all the time, and make up far more outrageous things than AIs make up. One has to ask whether humans are really intelligent /not entirely sarcasm.
- TeMPOraL 2y ago
  I'd prefer the phenomenon be called "saying the first thing that comes to your mind" instead, because humans do that a lot as well, and that happens to produce pretty much the same failures as LLMs do. IOW, humans "hallucinate" exactly the same way LLMs do - they just usually don't say those things out loud, but rather it's a part of the thinking process. See also: people who are somewhat drunk, or very excited, tend to lose inhibitions around speaking, and end up frequently just blurting whatever comes to their mind verbatim (including apologizing and backtracking and "it's not what I meant" when someone points out the nonsense).
pointlessone 2y ago
Confabulation is the term I’ve seen used a few times. I think it reflects what’s going on in LLMs better.
- dangsux 2y ago
  [dead]
- AceyMan 2y ago
  Yes, I thought this was already the agreed upon term for those in the know. I don't know what all this HN hullabaloo is about.
calf 2y ago
Your argument makes several mistakes. First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER attain a certain probabilistic distribution. This is a theoretical computer science problem and remains an open problem. So the second mistake is your probabilistic reductionism. It is true that LLMs, neural nets, and human brains alike are based on probabilistic computations. But the reasonable definition of a Hallucination is stronger than that - it needs to capture the notion that the probabilistic errors are way too extreme compared to the space of possible correct answers. An example of this is that Humans and LLMs get Right Answers and Wrong Answers in qualitatively very different ways. A concrete example of that is that Humans can demonstrate correctly the sequence of a power set (an EXP-TIME problem), but LLMs theoretically cannot ever do so. Yet both Humans and LLMs are probabilistic, we are made of chemicals and atoms. Thirdly, the authors' thesis is that mitigation is impossible. It is not some "lens" where mitigation is equal to alignment, in fact one should use their thesis to debunk the notion that Alignmnent is an attainable problem at all. It is formally unsolvable and should be rendered as a absurd as someone claiming prima facie that the Halting Problem is solvable. Finally, the meta issue is that the AI field is full of people who know zip about theoretical computer science. The vast majority of CS graduates have had maybe 1-2 weeks on Turing machines; an actual year-long course at the sophomore-senior level on theoretical computer science is Optional and for mathematically mature students who wish to concentrate in it. So the problem arises is a matter of a language and conceptual gap between two subdisciplines, the AI community and the TCS community. So you see lots of people believing in very simplistic arguments for or against some AI issue without a strong theoretical grounding that while CS itself has, but is not by default taught to undergraduates.
- deleted 2y ago
  [deleted]
- Terr_ 2y ago
  > You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). No they aren't: When you flip a coin, it landing to display heads or tails is "normal". That's no value judgement, it's just a way to characterize what is common in the mechanics. If it landed perfectly on its edge or was snatched out of the air by a hawk, that would not be "normal", but--to introduce a value judgement--it'd be pretty dang cool.
inglor_cz 2y ago
If the model starts concocting nonexistent sources, like "articles from serious newspapers that just never existed", it is definitely a malfunction for me. AFAIK this is what happened in the Jonathan Turley case.
wordofx 2y ago
[flagged]
mlindner 2y ago
I agreed with you until your last sentence. Solving alignment is not a necessity for solving hallucinations even though solving hallucinations is a necessity for solving alignment. Put another way, you can have a hypothetical model that doesn't have hallucinations and still has no alignment but you can't have alignment if you have hallucinations. Alignment is about skillful lying/refusing to answer questions and is a more complex task than simply telling no lies. (My personal opinion is that trying to solve alignment is a dystopian action and should not be attempted.)
- lolinder 2y ago
  My point is that eliminating hallucinations is just a special case of alignment: the case where we want to bound the possible text outputs to be constrained by the truth (for a value of truth defined by $SOMEONE). Other alignment issues have a problem statement that is effectively identical, but s/truth/morals/ or s/truth/politics/ or s/truth/safety/. It's all the same problem: how do we get probabilistic text to match our expectations of what should be outputted while still allowing it to be useful sometimes? As for whether we should be solving alignment, I'm inclined to agree that we shouldn't, but by extension I'd apply that to hallucinations. Truth, like morality, is much harder to define than we instinctively think it is, and any effort to eliminate hallucinations will run up against the problem of how we define truth.
sholladay 2y ago
It’s still a bug, even if it’s the only way the system can behave as currently designed. I agree that “hallucination” is a poor term for it, though. For medication, we call a bug a “side effect” even though it’s really just a chemical interaction which, given enough information, could be predicted. Ultimately the computer only does what we tell it to do. That has always been the case and probably always will be, just as we are the result of our inputs. As with most bugs, I think to solve hallucinations we will need to better understand the input and its interactions within the system.
- darby_nine 2y ago
  > It’s still a bug Only if it's actually possible to remove this behavior. I don't think there's any evidence of that at this point.
- lolinder 2y ago
  I find it helpful to distinguish between bugs and design flaws. A bug is caused by a poorly implemented version of the design (or a literal bug in the system). Fixing a bug requires identifying where the system varies from the design and bringing it into alignment with the design. A design flaw is a case where the idealized system as conceived by the engineers is incapable of fully solving the problem statement. Fixing a design flaw may require small tweaks, but it can also mean that the entire solution needs to be thrown out in favor of a new one. Importantly, what's a design flaw for one problem statement may be just fine or even beneficial for another problem statement. So, more objectively, we might refer to these as design characteristics. Hallucinations are a special case of two low-level design characteristics of LLMs: first, that they are trained on more data than can reasonably be filtered by a human (and therefore will be exposed to data that the humans wish it weren't) and second, that they produce their text by sampling a distribution of probabilities. These two characteristics mean that controlling the output of an LLM is very very difficult (or, as the article suggests, impossible), leading both to hallucinations and alignment concerns (which are actually the same concept framed slightly differently). If the problem statement for an LLM application requires more 99% factual accuracy or more than 99% "doesn't produce content that will make investors nervous" accuracy, these design characteristics count as a design flaw.
darby_nine 2y ago
Cannot agree enough. Hallucination is a terrible name for this behavior.
quantum_state 2y ago
Would second what you stated
FloatArtifact 2y ago
This is due to an entire field of AI /machine learning leaning into anthropomorphism shaping terminology reinforced by a narrative.
- noobermin 2y ago
  Which is totally on them. We had to suffer a decade of being told the human brain is a neural net (as if my brain was a numeric matrix, freaking ridiculous) they get to suffer when that dumb analogy comes back to bite them.
jiggawatts 2y ago
How sure are you that humans don’t also “simply produce plausible-sounding text”? I’m watching my toddler learning to speak and it’s remarkably like an early LLM that outputs gibberish with nuggets of sense. Conversely I’ve seen middle-aged professionals write formal design documents that are markedly inferior to what GPT-4 can produce. This is at every level: overall structure, semantics, and syntax. Diagrams with arrows the wrong way, labels with typos, labels on the wrong object, mixed up icons, tables with made-up content, and on and on. Aren’t we all doing this to some degree? Half way through a sentence do you known ahead of time exactly what you’re about to say three pages later? Or are you just “completing” the next word based on the context?
- lolinder 2y ago
  > How sure are you that humans don’t also “simply produce plausible-sounding text”? I'm not, and I didn't say I was. Nothing in my comment is downplaying the success and utility of LLMs. Nothing in my comment is suggesting that they won't get better or even that they won't get better than us humans. It is strictly an observation that these two things that we call different names—hallucinations and alignment—are actually equivalent and can each be expressed in terms of the other. A hallucination is a specific kind of misalignment, nothing more.
NoCoooode 2y ago
Agreed. Put simply, everything produced by an LLM today is an hallucination. Errors are simply poor hallucinations.
joe_the_user 2y ago
The term "hallucinate" may not be the best. However, if the context is a device generating a plausible sounding but non-existent scientific reference for a given claim it makes, some words more specific than "producing false information..." seem needed. These thing make assertions, some true, some false. Since they emulate human behavior, they say things that would tend to convince people of their assertions. So "anthropomorphism" is a complicated question.
xkcd1963 2y ago
Consequently we also shouldn't use intelligence or thinking when dealing with AI
drzzhan 2y ago
True. Actually, researchers know about it. In a sense, i feel "hallucination" is a way to keep the hype up, that we could fix it in the future, given enough data and compute and money. On the other hand, "generating false information by design" is too strong and negative. At least that's what I hear around in my research community.
cyanydeez 2y ago
I prefer the clearer term "bullshit"
more_corn 2y ago
It’s also problematic to suggest that the problem is unsolvable. It’s unsolvable inside the LLM. It’s 100% solvable within the product.

gwervc 2y ago

This comment should be pinned at the top of any LLM-related comment section.

Moru 2y ago
It should be part of every AI related story on the news. Just like they keep saying "X, formerly Twitter".
elif 2y ago
Nah it's quite pedantic to say that 'this neologism does not encapsulate the meaning it's meant to' This is the nature of language evolution. Everyone knows what hallucination means with respect to AI, without trying to confer to its definition the baggage of a term used for centuries as a human psychology term.
- lolinder 2y ago
  Neologism undersells what this term is being used for. It's a technical term of art that's created its own semantic category in LLM research that separates "text generated that is factually inaccurate according to ${sources}" from "text generated that is morally repugnant to ${individuals}" or "text generated that ${governments} want to censor". These three categories are entirely identical at a technological level, so I think it's entirely reasonable to flag that serious LLM researchers are treating them as distinct categories of problems when they're fundamentally not at all distinct. This isn't just a case of linguistic pedantry, this is a case of the language actively impeding a proper understanding of the problem by the researchers who are working on that problem.
- darby_nine 2y ago
  > This is the nature of language evolution. Only if it sticks. Hallucination is such an unnatural term for the phenomenon I would be surprised to see it stick. > Everyone knows what hallucination means with respect to AI This is false. It took me months to realize this just meant "output incoherent with reality" rather than an issue with training—the natural place for perceptual errors to occur.
- AStonesThrow 2y ago
  "Hallucination" is derogatory and insulting when aimed at normal people who hear normal voices which don't necessarily belong to corporeal beings, or originate in the natural world. Labeling "hallucinations" and pathologizing, then medicating them, constitutes assault and bigotry. "Hallucination" applied to inanimate and non-sentient software is insulting and presumptive on a different level. I'm fine with "confabulation".

wrs 2y ago

Yes, exactly, it’s a post-facto value judgment, not a precise term. If I understand the meaning of the word, “hallucination” is all the model does. If it happens to hallucinate something we think is objectively true, we just decide not to call that a “hallucination”. But there’s literally no functional difference between that case and the case of the model saying something that’s objectively false, or something whose objective truth is unknown or undefinable. I haven’t read the paper yet, but if they resolve this definition usefully, that would be a good contribution.

diputsmonro 2y ago
Exactly this, I've been saying this since the beginning. Every response is a hallucination - a probabilistic string of words divorced from any concept of truth or reality. By total coincidence, some hallucinations happen to reflect the truth, but only because the training data happened to generally be truthful sentences. Therefore, creating something that imitates a truthful sentence will often happen to also be truthful, but there is absolutely no guarantee or any function that even attempts to enforce that. All responses are hallucinations. Some hallucinations happen to overlap the truth.
- fumeux_fume 2y ago
  Ok, but I think it would be more productive to educate people that LLMs have no concept of truth rather than insist they use the term "hallucinate" in an unintuitive way.
  - lolinder 2y ago
    I don't know about OP, but I'm suggesting that the term 'hallucinate' be abolished entirely as applies to LLMs, not redefined. It draws an arbitrary line in the middle of the set of problems that all amount to "how do we make sure that the output of an LLM is consistently acceptable" and will all be solved using the same techniques if at all.
  - aeternum 2y ago
    LLMs do now have a concept of truth now since much of the RLHF is focused on making them more accurate and true. I think the problem is that humanity has a poor concept of truth. We think of most things as true or not true when much of our reality is uncertain due to fundamental limitations or because we often just don't know yet. During covid for example humanity collectively hallucinated the importance of disinfecting groceries for awhile.

Terr_ 2y ago

"So we built a blind-guessing machine, but how can we tweak it so that its blind guesses always happen to be good?"

durumu 2y ago

I think there's a useful distinction between plausible-seeming text that is wrong in some subtle way, vs text that is completely fabricated to match a superficial output format, and the latter is what I wish people used "hallucination" to mean. A clear example of this is when you ask an LLM for some sources, with ISBNs, and it just makes up random titles and ISBNs that it knows full well do not correspond with reality. If you ask "Did you just make that up?" the LLM will respond with something like "Sorry, yes, I made that up, I actually just remembered I can't cite direct sources." I wonder if this is because RLHF teaches the LLM that humans in practice prefer properly formatted fake output over truthful refusals?

majormajor 2y ago
How does a model "know full well" that it output a fake ISBN? It's been trained that sources look like plausible-titles + random numbers. It's been trained that when challenged it should say "oh sorry I can't do this." Are those things actually distinct?
rerdavies 2y ago
In fairness, they will also admit they were wrong even if they were right.

paulddraper 2y ago

It's an inaccuracy.

renjimen 2y ago

Maybe with vanilla LLMs, but new LLM training paradigms include post-training with the explicit goal of avoiding over-confident answers to questions the LLM should not be confident about answering. So hallucination is a malfunction, just like any overconfident incorrect prediction by a model.

aprilthird2021 2y ago
I still think OP has a point. The LLMs evolved after public use to be positioned as oracles which know so much knowledge. They were always probabilistic content generators, but people use them the way they use search engines, to retrieve info they know exists but don't exactly know. Since LLMs aren't designed for this there's a whole post process to try to make them amenable to this use case, but it will never plug that gap
- ta8645 2y ago
  > but it will never plug that gap They don't have to be perfect, they just have to be better than humans. And that seems very likely to be achievable eventually.
  - AlexandrB 2y ago
    To be better than humans they have to able confidently say "I don't know" when the correct answer is not available[1]. To me this sounds like a totally different type of "knowledge" than stringing words together based on a training set. [1] LLMs are already better than humans in terms of breadth, and sometimes depth, of knowledge. So it's not a problem of the AI knowing more facts.
    - ta8645 2y ago
      > To me this sounds like a totally different type of "knowledge" than stringing words together based on a training set. We're desperate to keep seeing ourselves as unique with key distinguishing features that are unreproducible in silicon, but from my long experience with computer chess, every step along the way, people were explaining patiently how computers could never reproduce the next quality that set humans apart. And it was always just wishful thinking, because computers eventually stopped looking silly, stopped looking like they were playing by rote, and started to produce "beautiful" chess games. And it will happen again, after the next leap in AI, people will again latch on to whatever it is that AI systems still lack, and use it to explain how they'll "always" be lacking... only to eventually be disappointed again that silicon can in fact reach that height too. Humans aren't magic. Whatever we can do, silicon can do too. It's just a matter of time.

BurningFrog 2y ago

"Hallucinations" just means that occasionally the LLM is wrong. The same is true of people, and I still find people extremely helpful.

leptons 2y ago
Except the LLM didn't deliver a "wrong" result, it delivered text that is human readable and makes grammatical sense. Whether or not the information contained in the text is "wrong" is subjective, and the reader gets to decide if it's factual or not. If the LLM delivered unreadable gibberish, then that could be considered "wrong", but there is no "hallicinating" going on with LLMs. That's an anthropomorphism that is divorced from the reality of what an LLM does. Whoever called it "hallucinating" in the context of an LLM should have their nerd credentials revoked.
- imchillyb 2y ago
  Human beings have a tendency to prefer comforting lies over uncomfortable truths. "The truth may set you free, but first it's really gonna piss you off." -G.S.
- AnimalMuppet 2y ago
  Human readable, makes grammatical sense, and wrong. And no, that's often not subjective.
  - leptons 2y ago
    It's very subjective. An LLM could return statements like "Global warming is real and man-made", and it also could produce a result like "Global warming is a hoax", and it's definitely up to the reader as to whether the LLM is "hallucinating". It doesn't matter how readable or grammatically correct the LLM is, it's still up to the reader to call bullshit, or not.
    - BurningFrog 2y ago
      If you ask about opinions, sure. Because there are no "true" opinions. If you ask about the capital of France, any answer but "Paris" is objectively wrong, whether given by a human or LLM.

nativeit 2y ago

I don’t know who/how the term was initially coined in this context, but I’m concerned that the things that make it inaccurate are also, perhaps counterintuitively, things that serve the interests of those who would overstate the capabilities of LLMs, and seek to cloud their true nature (along with inherent limitations) to investors and potential buyers. As you already pointed out, the term implies that the problems that are represented are temporary “bugs”, rather than symptoms of the underlying nature of the technology itself.

rerdavies 2y ago

How different things would be if the phenomenon had been called "makin' stuff up" instead. Humans make stuff up all the time, and make up far more outrageous things than AIs make up. One has to ask whether humans are really intelligent /not entirely sarcasm.

TeMPOraL 2y ago
I'd prefer the phenomenon be called "saying the first thing that comes to your mind" instead, because humans do that a lot as well, and that happens to produce pretty much the same failures as LLMs do. IOW, humans "hallucinate" exactly the same way LLMs do - they just usually don't say those things out loud, but rather it's a part of the thinking process. See also: people who are somewhat drunk, or very excited, tend to lose inhibitions around speaking, and end up frequently just blurting whatever comes to their mind verbatim (including apologizing and backtracking and "it's not what I meant" when someone points out the nonsense).

pointlessone 2y ago

Confabulation is the term I’ve seen used a few times. I think it reflects what’s going on in LLMs better.

dangsux 2y ago
[dead]
AceyMan 2y ago
Yes, I thought this was already the agreed upon term for those in the know. I don't know what all this HN hullabaloo is about.

calf 2y ago

Your argument makes several mistakes. First, you have just punted the validation problem of what a Normal LLM Model ought to be doing. You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). But we don't even know that - we would need theoretical proof that ALL theoretical LLMs (or neural networks as a more general argument) cannot EVER attain a certain probabilistic distribution. This is a theoretical computer science problem and remains an open problem. So the second mistake is your probabilistic reductionism. It is true that LLMs, neural nets, and human brains alike are based on probabilistic computations. But the reasonable definition of a Hallucination is stronger than that - it needs to capture the notion that the probabilistic errors are way too extreme compared to the space of possible correct answers. An example of this is that Humans and LLMs get Right Answers and Wrong Answers in qualitatively very different ways. A concrete example of that is that Humans can demonstrate correctly the sequence of a power set (an EXP-TIME problem), but LLMs theoretically cannot ever do so. Yet both Humans and LLMs are probabilistic, we are made of chemicals and atoms. Thirdly, the authors' thesis is that mitigation is impossible. It is not some "lens" where mitigation is equal to alignment, in fact one should use their thesis to debunk the notion that Alignmnent is an attainable problem at all. It is formally unsolvable and should be rendered as a absurd as someone claiming prima facie that the Halting Problem is solvable. Finally, the meta issue is that the AI field is full of people who know zip about theoretical computer science. The vast majority of CS graduates have had maybe 1-2 weeks on Turing machines; an actual year-long course at the sophomore-senior level on theoretical computer science is Optional and for mathematically mature students who wish to concentrate in it. So the problem arises is a matter of a language and conceptual gap between two subdisciplines, the AI community and the TCS community. So you see lots of people believing in very simplistic arguments for or against some AI issue without a strong theoretical grounding that while CS itself has, but is not by default taught to undergraduates.

deleted 2y ago
[deleted]
Terr_ 2y ago
> You rhetorically declared hallucinations to be part of the normal functioning (i.e., the word "Normal" is already a value judgement). No they aren't: When you flip a coin, it landing to display heads or tails is "normal". That's no value judgement, it's just a way to characterize what is common in the mechanics. If it landed perfectly on its edge or was snatched out of the air by a hawk, that would not be "normal", but--to introduce a value judgement--it'd be pretty dang cool.

inglor_cz 2y ago

If the model starts concocting nonexistent sources, like "articles from serious newspapers that just never existed", it is definitely a malfunction for me. AFAIK this is what happened in the Jonathan Turley case.

wordofx 2y ago

[flagged]

mlindner 2y ago

I agreed with you until your last sentence. Solving alignment is not a necessity for solving hallucinations even though solving hallucinations is a necessity for solving alignment. Put another way, you can have a hypothetical model that doesn't have hallucinations and still has no alignment but you can't have alignment if you have hallucinations. Alignment is about skillful lying/refusing to answer questions and is a more complex task than simply telling no lies. (My personal opinion is that trying to solve alignment is a dystopian action and should not be attempted.)

lolinder 2y ago
My point is that eliminating hallucinations is just a special case of alignment: the case where we want to bound the possible text outputs to be constrained by the truth (for a value of truth defined by $SOMEONE). Other alignment issues have a problem statement that is effectively identical, but s/truth/morals/ or s/truth/politics/ or s/truth/safety/. It's all the same problem: how do we get probabilistic text to match our expectations of what should be outputted while still allowing it to be useful sometimes? As for whether we should be solving alignment, I'm inclined to agree that we shouldn't, but by extension I'd apply that to hallucinations. Truth, like morality, is much harder to define than we instinctively think it is, and any effort to eliminate hallucinations will run up against the problem of how we define truth.

sholladay 2y ago

It’s still a bug, even if it’s the only way the system can behave as currently designed. I agree that “hallucination” is a poor term for it, though. For medication, we call a bug a “side effect” even though it’s really just a chemical interaction which, given enough information, could be predicted. Ultimately the computer only does what we tell it to do. That has always been the case and probably always will be, just as we are the result of our inputs. As with most bugs, I think to solve hallucinations we will need to better understand the input and its interactions within the system.

darby_nine 2y ago
> It’s still a bug Only if it's actually possible to remove this behavior. I don't think there's any evidence of that at this point.
lolinder 2y ago
I find it helpful to distinguish between bugs and design flaws. A bug is caused by a poorly implemented version of the design (or a literal bug in the system). Fixing a bug requires identifying where the system varies from the design and bringing it into alignment with the design. A design flaw is a case where the idealized system as conceived by the engineers is incapable of fully solving the problem statement. Fixing a design flaw may require small tweaks, but it can also mean that the entire solution needs to be thrown out in favor of a new one. Importantly, what's a design flaw for one problem statement may be just fine or even beneficial for another problem statement. So, more objectively, we might refer to these as design characteristics. Hallucinations are a special case of two low-level design characteristics of LLMs: first, that they are trained on more data than can reasonably be filtered by a human (and therefore will be exposed to data that the humans wish it weren't) and second, that they produce their text by sampling a distribution of probabilities. These two characteristics mean that controlling the output of an LLM is very very difficult (or, as the article suggests, impossible), leading both to hallucinations and alignment concerns (which are actually the same concept framed slightly differently). If the problem statement for an LLM application requires more 99% factual accuracy or more than 99% "doesn't produce content that will make investors nervous" accuracy, these design characteristics count as a design flaw.

darby_nine 2y ago

Cannot agree enough. Hallucination is a terrible name for this behavior.

quantum_state 2y ago

Would second what you stated

FloatArtifact 2y ago

This is due to an entire field of AI /machine learning leaning into anthropomorphism shaping terminology reinforced by a narrative.

noobermin 2y ago
Which is totally on them. We had to suffer a decade of being told the human brain is a neural net (as if my brain was a numeric matrix, freaking ridiculous) they get to suffer when that dumb analogy comes back to bite them.

jiggawatts 2y ago

How sure are you that humans don’t also “simply produce plausible-sounding text”? I’m watching my toddler learning to speak and it’s remarkably like an early LLM that outputs gibberish with nuggets of sense. Conversely I’ve seen middle-aged professionals write formal design documents that are markedly inferior to what GPT-4 can produce. This is at every level: overall structure, semantics, and syntax. Diagrams with arrows the wrong way, labels with typos, labels on the wrong object, mixed up icons, tables with made-up content, and on and on. Aren’t we all doing this to some degree? Half way through a sentence do you known ahead of time exactly what you’re about to say three pages later? Or are you just “completing” the next word based on the context?

lolinder 2y ago
> How sure are you that humans don’t also “simply produce plausible-sounding text”? I'm not, and I didn't say I was. Nothing in my comment is downplaying the success and utility of LLMs. Nothing in my comment is suggesting that they won't get better or even that they won't get better than us humans. It is strictly an observation that these two things that we call different names—hallucinations and alignment—are actually equivalent and can each be expressed in terms of the other. A hallucination is a specific kind of misalignment, nothing more.

NoCoooode 2y ago

Agreed. Put simply, everything produced by an LLM today is an hallucination. Errors are simply poor hallucinations.

joe_the_user 2y ago

The term "hallucinate" may not be the best. However, if the context is a device generating a plausible sounding but non-existent scientific reference for a given claim it makes, some words more specific than "producing false information..." seem needed. These thing make assertions, some true, some false. Since they emulate human behavior, they say things that would tend to convince people of their assertions. So "anthropomorphism" is a complicated question.

xkcd1963 2y ago

Consequently we also shouldn't use intelligence or thinking when dealing with AI

drzzhan 2y ago

True. Actually, researchers know about it. In a sense, i feel "hallucination" is a way to keep the hype up, that we could fix it in the future, given enough data and compute and money. On the other hand, "generating false information by design" is too strong and negative. At least that's what I hear around in my research community.

cyanydeez 2y ago

I prefer the clearer term "bullshit"

more_corn 2y ago

It’s also problematic to suggest that the problem is unsolvable. It’s unsolvable inside the LLM. It’s 100% solvable within the product.