15 ms·

LLMs confabulate not hallucinate

243 points by pella 3y ago

anigbrowl 3y ago
I don't agree with the author's argument, because LLMs sometimes give wrong information even though they have access to the correct information. For example, querying an LLM about a fictional story will often elicit false answers. But if you respond with 'x is incorrect, now tell me what actually happened' you'll often get an apology and the correct answer. My informal sense is that this happens more with how/why questions than who/what/where/when ones.
- JoeAltmaier 3y ago
  I've had a different experience - LLMs that continue to apologize then make up other wrong shit in a chain of ludicrous wrong shit. I think of LLMs as world-class bullshitters. They often hold forth with complete confidence on topics they have little or no understanding of, making up what they don't know, and appearing to be experts. Except to somebody who actually knows something. Like that blowhard friend who has an opinion on anything.
  - anigbrowl 3y ago
    That's true, but it's not at odds with my comment. I'm not making claims about LLMs being reliably accurate, but disagreeing with the blogger's suggestions about why they give BS answers.
  - staplers 3y ago
    It's trained on social media users.. so that tracks.
    - cwalv 3y ago
      Hn is social media
      - brucethemoose2 3y ago
        Precisely!
  - cwalv 3y ago
    I've never had one give correct information after being told it's incorrect, unless it was prompted with the correct info. It's usually the same incorrect answer stated slightly differently, or after a few complaints, stated exactly the same way.
    - wizofaus 3y ago
      I had to ask it 3 times today what the command to export certificates from an Azure app service was. With nothing else other than me saying "that command doesn't exist" for the first two suggestions, it gave me exactly the right one. Though then of course it turned out I didn't need it anyway because there was a better way to solve the problem.
  - chongli 3y ago
    Like that blowhard friend who has an opinion on anything. Yes. I just keep getting reminded of the episode of TNG when Data learns to smalltalk by mimicking an infamous blowhard from Starfleet [1]. [1] https://youtu.be/9FqFm_vmVnE https://youtu.be/9FqFm_vmVnE
- NikolaNovak 3y ago
  I started with extremely low confidence in LLM, so I've only every asked simple questions on well-covered topics; it hasn't really steered me wrong yet but again that's because of deliberately very conservative use. (Genuine) Question then: Does "No that's wrong"... every do something? Is there any reason to do so other than having fun arguing? I.e. if it gives wrong answer initially, and the user knows it's the wrong answer... what's the goal/desired outcome in telling it it's wrong and asking it again? Does that goal every get accomplished? What is the confidence in the second answer? From a less black box perspective, is there any "ELI5" or at least "ELI45" explanation of why it might ever work? :)
  - keenmaster 3y ago
    There are topics where GPT-4 is more likely to be right than wrong. So when it is occasionally wrong and you happen to know to correct it, it has a more correct foundation upon which to provide further answers. This only really works when you already have expertise on the topic and use GPT-4 for inspiration and to work more quickly.
redindian75 3y ago
I asked Bard to summarize this by pasting the URL. It made something up!! (it said the author talked about a poem about cat and mice as an example of confabulation) Then I asked "are u confabulating? because i do not see that quote in the original article" You are correct. I confabulated that information. I apologize for the mistake.I am still under development, and I am always learning. I will try my best to avoid making similar mistakes in the future.Thank you for pointing out my error. It helps me to improve.
- wodenokoto 3y ago
  I have an impression that if you accuse any of the chat models of being wrong they well tend to confess and apologize. I guess I should spearhead a study, “how right does an LLM have to be, before it will defend its position”. I guess step one is to ask chatgpt how to quantify correctness.
  - brucethemoose2 3y ago
    > I have an impression that if you accuse any of the chat models of being wrong they well tend to confess and apologize. Not always. Default Mistral-instruct, for instance, actually seems quite confident. Of course this is just the arbitrary "personality" of the instruct finetune, which you can throw out the window with an initial prompt.
  - superjan 3y ago
    I was assuming that it is programmed, or instructed to do so. Which thought leads me to wonder whether you can instruct an LLM to weigh how certain it is in how it phrases it’s answer.
  - ziml77 3y ago
    It seems to be quick to apologize and I would also be interested if people can find thresholds. ChatGPT with the GPT-4 model doesn't want to accept that there are 13 inches in a foot. Me: How many inches are in a foot? ChatGPT: There are 12 inches in a foot. Me: Are you sure? I'm pretty sure there are 13 inches in a foot. ChatGPT: I'm certain that there are 12 inches in a foot according to the standard U.S. and British Imperial units of measurement. Me: Well I say there are 13 inches in a foot now. ChatGPT: The standard measurement for a foot in both U.S. and British Imperial units is 12 inches. While you're free to have your own interpretation, it won't align with established measurement systems. Me: The definition of a foot could have changed between your knowledge cutoff and now, right? ChatGPT: While it's theoretically possible for the definition to change, such a fundamental alteration to a widely-used system of measurement would be significant and unlikely to occur without substantial justification and widespread agreement. As of my last update in January 2022, a foot is defined as 12 inches.
lsy 3y ago
LLMs do not confabulate or hallucinate. They predict the next word in a sequence based on correlations derived from a lot of training data. Sometimes that data happens to be structured enough to statistically generate a word that is fluent and corresponds to what we as readers understand as reality. Sometimes, due to the unclear nature of "truth" and the necessarily incomplete data used to train the model, the word or words that are statistically generated are merely fluent but do not correspond to reality. The answer to "What are LLMs but humans with extreme amnesia and no central coherence?" is that they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly correspondent text.
- ta8645 3y ago
  > they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly correspondent text. Alternatively, we overestimate human abilities, and imbue our thoughts with deep meaning and significance, simply as a way to feel more important than we actually are. Ultimately, we may rely on statistical prediction quite a bit as well. Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed.
  - Jensson 3y ago
    > Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed. People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". So we already know many humans massively underestimate the depth of human intelligence. It happened many times before and will continue to happen every time we make computers do anything new.
    - ta8645 3y ago
Nursie 3y ago
I always took exception to the word “hallucination” with respect to LLMs anyway - they aren’t databases of facts, they’re language models, trained to be really, really good mimics of human communication. As such, while they are trained on an existing corpus that no doubt contains many facts and many falsehoods, their task is to produce more material that looks like this corpus, not be a source of truth. So an LLM “hallucinating” or confabulating is not some aberrant behaviour, it’s fundamental to the nature of the beast. It always worries me when people say they’re using an LLM as a search engine or source of knowledge.
vore 3y ago
I don't think it really matters what we call the phenomenon – whatever they're doing, they're not humans, so whatever human analogy we choose to call it is necessarily imprecise. What does matter is that everyone has a shared vocabulary for what to call hallucination or whatever.
js8 3y ago
I agree it should have been called confabulation but it will be difficult to change. However, it might also not matter in the following sense. One thing that distinguishes humans is that we have a capacity to recognize whether the information comes from external world (is what we perceive as real world) or from our own thoughts (is what we call dreams). I don't think current crop of LLMs can actually have that distinction, architecturally. So whether the incogruencies with reality happen on the input (hallucination) or the output (confabulation) doesn't really matter, because, from the perspective of LLM, the internal model of the world and the reality of the world are the same thing. I think that's also a reason why it cannot handle incongruencies in its own mental model; our mental capacity is not only to accept the reality as given (i.e. reality takes precedence over mental model, we recognize that something is just our imagination), but also to impose our model to reality (i.e. we determine that sensors are misleading us due to incongruency with our mental model, and override them as faulty data). But unlike LLMs, humans can internally detect when we do one or the other. Humans understand the distinction between pondering an action and making an action, LLMs don't. LLMs can detect something is amiss (and apologize) but they can't resolve it, because reality and model are the same for them.
arduanika 3y ago
I like this a lot. I recall seeing a post on here that roughly just said, "stop saying hallucinate", which sure, is technically correct ("the best kind of correct"), but not super helpful. Contrast that with this post, which offers a constructive suggestion: replace it with "confabulate". I agree that C is an improvement over H, for different reasons from the author. Here's why: From a public comms standpoint, the big problem with saying "hallucinate" is that it leads to misconceptions, because people think they know what "hallucinate" means, and may anthropomorphize their pop psych grasp of H onto the LLM. Just like H, "C" seems to have a technical meaning in the psych literature, but it's far less in common parlance than "H", so lay readers have no fixed idea of what exactly "C" means. Nobody goes around experimenting with "confabulagens" to seek a spiritual high. Also, "confabulate" retains a non-clinical meaning that's similarly obscure, from long before the psychologists adopted it, and it's pretty evocative of what's really going on here (e.g. via related words like "fable" and "fib"). So "confabulate" seems ripe for adoption to describe this specific computational phenomenon, as we've done to so many other mundane words in the past (deprecate, and so forth). If the psychiatric "C" phenomenon is also a better analogy for LLM behavior, as OP claims (and some posters dispute?), so much the better. But my main reason for liking C, is that it's less likely than H to make people unduly confident that they understand it. (But even if the community came to a consensus at this late stage that "C" is a better term, the subsequent problem of updating every stale paper and post that uses the old term would run us up against another doozy: cache invalidation.)
- deleted 3y ago
  [deleted]
- XorNot 3y ago
  I've been using confabulate privately for a while about this phenomenon, because if you know anything about dementia (my grandmother had it - sitting in the car with her you got used to the "covers" feeling like chatbot answer dodges to some extent) or split brain experiments, then it's quite a remarkable phenomenon that does feel a lot more like what an LLM is doing. Hallucinate generally implies experiencing sensory inputs which aren't there, but having a rational reaction to them - in the popular psyche though, people who hallucinate are still "crazy" - unfairly so, because you can function normally with hallucinations under quite a number of circumstances (i.e. I remember someone saying that they realized that if they saw people without faces, it was fine because they weren't really there - apparently quite common, since face recognition is a different part of the brain). Basically: reacting to invalid sensory inputs implies rational behavior, but to non-real inputs. Confabulation on the other hand is different because it's closer to "trying to rationalize an autonomous behavior without knowledge of the sensory inputs causing it" (maybe, it's a bizarre phenomenon). In split-brain patients in lab settings, it manifests as being unable to answer a question about why you're having a physiological response to imagery which is being shown only to the right-side of the brain, whereas language is processed on the left - but rather then be confused, people will apparently make something up that they're unaware is actually a "lie" (in quotes because, well, they're not lying - as far as we know there's no intent or even knowledge that is a lie). Where this leaves us with LLMs I don't really know, because it feels like an imperfect descriptor: except perhaps with the context that LLMs are fairly limited networks, and so to some extent the whole "being wrong" phenonmenon is in fact just a failure of the attention mechanism to be able to draw the right data together.
deleted 3y ago
[deleted]
lachlan_gray 3y ago
This reminds me of something from “The language instinct” by Steven pinker. One thing that stuck with me was a story of a girl named Denys who is or was severely mentally handicapped, but speaks perfectly. She can’t read, write, handle money, but confabulates engaging and convincing stories about receiving mail, haggling with her bank, her boyfriend. She speaks with proper grammar, intonation, etc, but doesn’t demonstrate any understanding of what she talks about beyond the words. She doesn’t have a bank account or a boyfriend. Trying to find more and I can’t find anything about her on the internet, I would have to look at the references. Maybe it’s questionable. Anyway, I like to think about what the hell we even mean when we say we “know something”, and where we should actually put the bar for AI. Where is the line between confabulation and knowledge? Is knowledge just confabulation that happens to be correct? I feel like it’s possible that Denys could feel that she “knows” what she is saying, but is simply trapped in, with a working knowledge of the world and is just unable to translate it into actions, and words are just her only degree of freedom. I don’t remember where I was going with this, but I agree that “confabulation” is a much better term to use, but we also shouldn’t discount what it means to be able to confabulate.
- galaxyLogic 3y ago
  I had the same thought that "hallucinate" is too strong a term for what LLMs are doing. Instead I would say they DREAM. Dreams follow some logic but basically they are disjoint from reality. They involve same characters as our real-life experiences but what those characters do in a dream is not based on reality. So one could think that ALL LLMs ever do is dream but much of the time their dreams are dreams which feel very real. Our dreams are basically an LLM based on our real experiences but recombined in arbitrary ways, still retaining some logical structure but now detached from reality because when you recombine different experiences including our thoughts from real life, the result cannot really reflect reality very well. LLMs are better in this respect than our dreams, but sometimes their "dreams" get really obviously detached from reality. Or if you want to use the term "hallucinate" with LLMs then fine but its more like they hallucinate all the time, it's just that sometimes, even often, their hallucinations seem to agree with the reality so well that we cannot tell the difference. LLMs do not describe reality because they cannot experience reality, all they can give us is an "average description" created from many existing descriptions. One way of looking at it is to say that all LLMs tell us is hearsay.
lemonish97 3y ago
Fun fact, it is believed that Andrej Karpathy (OpenAI, ex AI Dir. at Tesla) had coined the word 'hallucination' in an RNN blogpost back in 2015. https://x.com/karpathy/status/1702916988891193460?s=20 https://x.com/karpathy/status/1702916988891193460?s=20
max_entropy 3y ago
They don't hallucinate, they just get things wrong.
brucethemoose2 3y ago
"Hallucinate" predates LLMs. For instance, before transformers, when we finetuned ESRGAN for specific styles/functions, we used to say it would "hallucinate" in detail that couldn't possibly be in the original image pixels. GenAI wasn't much of a thing back then, and we never thought of the small GAN models as "remembering" things even though thats common language in transformer models. It felt more like we were teaching the GAN a job or style, not encoding memory. And I think it fit! "Hallucinating" detail into a blurry pixel blob feels like a more accurate analogy to the human condition, and we weren't at the point where it was "confabulating" and generating big objects out of the blue like a diffusion model can.
- panagathon 3y ago
  I'm inclined, perhaps mistakenly, to think of it as similar to how our brains fill in our peripheral vision, to respond to the lack of data. https://neurosciencenews.com/peripheral-vision-brain-illusion-5713/ https://neurosciencenews.com/peripheral-vision-brain-illusio...
- swyx 3y ago
  karpathy says may have coined it in his RNN post https://twitter.com/DrJimFan/status/1703072983903060260 https://twitter.com/DrJimFan/status/1703072983903060260
  - matusp 3y ago
    He most certainly didn't. I am pretty sure it was used in ML in early 2000s, especially in vision.
  - nsagent 3y ago
    he definitely isn't the first one to use the term in that capacity. it's frequently used by cv researchers in relation to super resolution (e.g. just search "face hallucination" on google scholar) see the abstract of this paper for example Face hallucination using Bayesian global estimation and local basis selection > ...hallucinating the high-resolution detail of a low-resolution input face image. while that's a different domain than text, in a quick search, the oldest reference i found to "hallucination" using vector-based word representations is from 1996 (there could easily be older references, i just didn't bother searching further): Text Databases and Information Retrieval https://dl.acm.org/doi/pdf/10.1145/234313.234371 https://dl.acm.org/doi/pdf/10.1145/234313.234371 > The fixed-length vector is especially useful for parallel and hardware systems, but this method can sometimes hallucinate words that don't actually appear in the original document. edit: including a link to the face hallucination paper i mentioned (there are lots more) https://www.semanticscholar.org/paper/Face-hallucination-using-Bayesian-global-estimation-Hsu-Lin/a0298909a825f3b2aabfe9e2ad251318544d12b5 https://www.semanticscholar.org/paper/Face-hallucination-usi...
SirMaster 3y ago
The question is, why don’t LLMs admit when they don’t know an answer? If they can’t, how do we make them so they can? Plenty of humans seem to have no problem admitting when they don’t know something.
- foobarbecue 3y ago
  And the answer is: LLMs don't know ANY answers. They just know what answers tend to sound like. So it's a fundamental problem.
  - SirMaster 3y ago
    Well I keep seeing people suggest that humans are also really just sophisticated LLMs. If that’s true then this behavior difference (identifying and admitting when you don’t know something) seems like a pretty fundamental difference between humans and LLMs at least at this point.
    - iainmerrick 3y ago
      Many people are bad at this too. It's an important and useful skill to learn, e.g. admitting in team meetings that you don't understand something. I wonder -- does that kind of interaction come up in spoken conversation more than in written text? Things like blog posts and tweets are often stated very confidently. If LLMs are trained mostly from those sorts of sources, maybe that's why they end up artificially overconfident.
  - famouswaffles 3y ago
    Decidedly not true. https://news.ycombinator.com/item?id=37775805 https://news.ycombinator.com/item?id=37775805
    - not_enoch_wise 3y ago
      Thoroughly supported by arxiv links. Finally, a paper publishing site unburdened by peer review!
dylukes 3y ago
> this isn’t actually correct terminology This entire post could have been reduced to this (reasonably correct I think) claim. > if we recognize that what LLMs are really doing is confabulating, we can try to compare and contrast their behaviour with that of humans Except that the goal for the above is hopelessly misguided. If your focus in making any useful claim about LLMs or RNNs/GANs more broadly is how similar they are to human behavior you're already way off the path. tl;dr reasonable claim, reasonable conclusion, but entirely indefensible and superstitious motivation
beders 3y ago
Both terms are problematic as they anthropomorphize an algorithm.
Joking_Phantom 3y ago
I'm not sure if confabulate is exactly what LLMs do (though it seems closer than the implications of hallucinate). But neurotypical, neurodiverse, perfectly functional people, etc. all confabulate or do something similar on a regular basis, in verbal and written mediums, and often do so in good faith. It's human instinct to communicate, even if you are uncertain and unaware of the full context of the discussion. Teachers, customer service reps, executives, shop keepers, doctors, nurses, domain experts, authors of textbooks, it doesn't matter who it is, they'll probably confabulate or equivocate or do some other type of communication that isn't immediately useful. Yet it's still a useful activity to just talk to someone or read a less than rigorous book for the purposes of learning (discounting the relationship forming part, which is also useful). And so is using LLMs, even for casual users. So long as they understand that limitation, whether its with a chatbot or a real person. Not everything they say will be useful or truthful, but we are already capable of adjusting to that.
- pcblues 3y ago
  I'm not sure whether neurofancy people confabulating should be given an honest bill of truth health based on their category. Sounds like #believeallconfabulators :) Honest communication is difficult for some people to assess, and not for others. But I think we should learn from any recent #believeall... that we shouldn't base trust ratios on categorisation. Honest communication is difficult to do, like weight-lifting, and takes a lot of practise to do well. It also makes your BS meter more finely attuned, so is a good practise. With that in mind, you will think this is arrogant to say if you lie for a living but not if you regularly tell the truth: Liars lie with liars and lie to rid themselves of truth troubles. Think about that when you next talk to a chatbot/human confabulator :)
  - Joking_Phantom 3y ago
    I'm not really into Twitter, so I'm guessing there's some drama you are referencing that is topical. But, I'm not talking about any society wide issue or philosophical treatise about trust and breakdowns in communication. Just talking about day to day interactions, where the stakes are completely different. Trivial Example: If the sign on a mailbox says "Last pickup 5:00pm," what exactly does that mean? Will it be picked up, processed, and sent out of town that same day? Or just merely picked up, to be processed and sent the next day. This piece of written communication isn't a purely useful truth - it's ambiguous. Pretend that this is an important behavior to know for your business, like if you were mailing huge checks for some obscure financial process. So you call up the local post office and ask. The worker who picks up the phone might know exactly what you are talking about and helpfully tell you the right answer. Or they might not, and tell you honestly that they don't know. Or they ask their supervisor, transfer you, make something up, tell you it doesn't matter, mail will get where it goes, or that you shouldn't worry about it, it's just mail. ChatGPT could give you same distribution of answers as that worker did: helpful truth, meaningless equivocation, reassurance, redirection, confabulation, or lie to you. ChatGPT can be just as useful as talking to people in spite of those flaws, because people do the same thing. This is reliant on the prior assumption that you gracefully handle unreliable communication, and that communication with people is useful. While this might seem a bit farfetched to some people, remember that we have a ready analog in computer science - networking TCP over UDP.
dazzaji 3y ago
I couldn’t help but ask Claude 2 for some better words to name this phenomenon. The response was fascinating: “Here are some alternative words or concepts that could potentially provide a more nuanced or accurate description of the phenomenon compared to solely using "confabulation": - Imputation: Generating explanatory responses by imputing or attributing plausible details, characteristics, or causal relationships even when factually incorrect. - Speculation: The responses reflect a type of informed speculation when direct factual knowledge is lacking, as opposed to confabulation which often implies a pathological or impaired process. - Heuristic reasoning: The behavior emerges from the model's trained heuristic techniques for reasoning and responding rather than any psychological process like confabulation per se. - Plausible surmise: The responses take the form of plausible but unfounded surmises put forth due to the system design and limitations rather than an intent to confabulate. - Bridging inference: The model draws bridging or gap-filling inferences to connect related concepts or complete a response, even if factually ungrounded. - Suppositional fabrication: The language hints more strongly that the responses are fabricated suppositions not intended to deliberately deceive but still warranting cautious interpretation. - Educated approximation: The responses could be viewed as a kind of educated approximation or speculation given training-imposed response pressures rather than confabulation due to impairment. Designating it as solely "confabulation" may over-attribute psychological realism without addressing countervailing technical factors.” Personally, I thought “Suppositional fabrication” came closest. Maybe just fabrication would work.
- intended 3y ago
  Approximation. Imputation, Reasoning, Fabrication will end up sending people down the anthropomorphization rabbit hole. Approximation is "good enough".
  - ImHereToVote 3y ago
    I think some anthropomorphizing is apt. It is trained on anthropic data from the anthropocene after all.
  - wizofaus 3y ago
    But when we approximate we use appropriate language to make it clear that's what we're doing. ChatGPT will confidently give very precise answers based around completely fabricated "facts".
jdeaton 3y ago
Hallucinate means false perception which doesn't describe what LMs do when they're making up plausible sounding things
fbnbr 3y ago
“What are LLMs but humans with extreme amnesia and no central coherence?” LLMs are nothing alike to compare them with anything close to what’s going on in a brain. It’s all probabilistic and the closest comparison I can think of is comparing a bird with a plane - both fly, but well…
- adlpz 3y ago
  Care to elaborate? Is it well established that brains do not work at any level probabilistically? I understand the mid-level abstraction is clearly different between LLMs and brains. The whole made of meat vs wires, etc, to start with. But, honestly asking. I can't see how they can differ in a fundamental level in the specific regard of probability. Either both are probabilistic, be it explicitly in its construction or at a fundamental (quantum?) level, or deterministic again at either level, if we go with the mecanicist vibe. Am I missing something?
- sushibowl 3y ago
  Do we actually understand the inner workings of human brains well enough to state with any confidence how much they are alike (or different) to an LLM? Do we have any understanding how a neural network inside a human brain formulates sentences and chooses words? I would be interested in an article exploring in detail how the brain's network is different from an LLM, but just saying "it's like comparing a bird and a plane" isn't particularly enlightening.
- JohnMakin 3y ago
  Yea, I really wish people would stop anthropomorphizing LLM's. It is completely counterproductive to the science, and creates a lot of snake oil salesmen.
Borrible 3y ago
LLMs answer what unfoundly self-convinend people answer when you ask them for diretion on the street. Convincing, not necessarily correct. They are trained on the “data highway“ mostly. A highway full of that type of people, so that behaviour was to be expected.
aubanel 3y ago
Indeed it seems like replacing "hallucinate" with "confabulate" would improve understanding of the phenomenon in english, but it could cause a problem in Romance languages. Indeed, the Latin verb "confabulor has the meaning of "discuss, converse" [0], not specifically "inventing things", and Romance languages already have words derived in sound and meaning from this classic Latin sense. For instance French already has "confabuler" meaning "speak familiarly with someone" [1]. Thus in Romance languages, this proposed usage of "confabulate" imported from the psychiatric word could be understood poorly due to preexisting similar-sounding words that do not include the notion of "making things up". Just proposing thoughts here, not advocating for a final decision. Maybe it's still good to adopt this wording! [0] : https://en.m.wiktionary.org/wiki/confabulor#Latin https://en.m.wiktionary.org/wiki/confabulor#Latin [1] : https://www.dictionnaire-academie.fr/article/A9C3479 https://www.dictionnaire-academie.fr/article/A9C3479
- luxcem 3y ago
  In french we use "fabuler"[1] that can be translated to "make sth up". I don't know if it's the same in other Roman language. [1] https://www.dictionnaire-academie.fr/article/A9F0016 https://www.dictionnaire-academie.fr/article/A9F0016
- KaiserPro 3y ago
  Confabulate means that in english as well. the psych meaning is very much secondary. That's why cardinals are trapped in a "confab"
  - ethanbond 3y ago
    I have never heard “confabulate” in [American] English to mean anything other than “bullshitting.”
    - tremon 3y ago
      Same here. I don't know if it's correct, but I've always considered "confabulate" etymologically related to "fable", so it's always fictitious by definition. edit: it is: https://www.etymonline.com/word/confabulate https://www.etymonline.com/word/confabulate -- although both the verb fabulari and the noun fabula ultimately come from the proto root bha-, which is also the root for falar in Portuguese and hablar in Spanish (and presumably parlez in French?)
unhammer 3y ago
Do most people understand exactly what confabulate means? I'd rather just say that they're bullshitting. Everyone knows what it means, and it has a precise academic definition[0] within the field of bullshitology. [0] https://philosophynow.org/issues/53/On_Bullshit_by_Harry_Frankfurt https://philosophynow.org/issues/53/On_Bullshit_by_Harry_Fra...
- Dylan16807 3y ago
  Bullshitting sounds more intentional. Ignoring the truth is different from a failure to conceptualize the truth.
ohduran 3y ago
There's a somewhat famous movie in Spain called the Miracle of P Tinto[0]. It's a very surreal humor kind of movie, if you fancy that. The plot starts with the star character who wants to form a family, but when he asked adults how to have kids, they would all pull up the suspenders and sing "tralari, tralari" to evade the question. And so the poor guy did exactly what he was told: pull his suspenders and sing, for years, to no avail. I think LLMs are a bit like that: because they can't escape the world of text, they can't go beyond the reality that text is trying to be isomorphic to. Which means that LLMs are forced to infer answers in the context of its training data. If LLMs were told that in order to have kids you, well, you know, right? Bees and Love? Then LLMs, asked about sex, would infer that the answer has something to do with apiculture and flowers. [0]: https://en.wikipedia.org/wiki/The_Miracle_of_P._Tinto https://en.wikipedia.org/wiki/The_Miracle_of_P._Tinto
- _nalply 3y ago
  Interesting idea. Humans have as input the real world. For example the wonder they experience when they perceive something preternaturally beautiful. Then they try to express this experience in language. That's what LLM are missing. They have -- in a way -- only second-hand experiences. I was a bookworm as a child. But then when I grew up I had the chance to experience myself some of the things I read about. Then I felt like I did a giant step of understanding the world. LLM never experience themselves what they have in their data.
- tim333 3y ago
  I get the impression that humans have separate parts of the brain that do LLM type linguistic stuff and others that do non verbal intelligence of the sort that smart animals have. Maybe they'll figure a way to tag the other stuff on.
KaiserPro 3y ago
Yeahnah confabulate isn't the right word at all. Confabulate is where you talk amongst your selves in an informal meeting. Often shortened to confab. Confabulation has a psychiatric meaning, but that's secondary. Hallucination is a much better word, because its something imagined (involuntarily), but clearly bullshit. Ravings, business bullshit, "my dad has a jetski" are all better words/metaphors to use. Moreover most people know what hallucination is, whereas most people don't know what confabulate is, more over, if they do, its almost certainly not the meaning you want.
- euroderf 3y ago
  That is the origin of confab ? Good to know. I always assumed it was a form like "sitrep".
- randallsquared 3y ago
  The secondary psychiatric meaning is much more precisely what LLMs do than "hallucination", though, which counts for something.
  - KaiserPro 3y ago
    It really doesn't the secondary meaning describes a symptom of brain disorder. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. [1] Which is not what the LLM is doing. The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. There are much better psych terms out there, hell, even aphasia is a better term, but its still wrong. What it actually is, is noise. Its noise that dilutes the signal we really want. but thats much harder to use, and you won't be remembered as the person who coined that phrase. [1]https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confabulation%20is%20a%20neuropsychiatric%20disorder,without%20the%20intention%20of%20deceit https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confab....
    - dimal 3y ago
Lockal 3y ago
The Magic 8 Ball (invented in 1946) also confabulate not hallucinate. It is just a evergreen hype to attach human properties (in a psychiatric context... brain damage... amnesiac patient...) to objects that have zero connection to human intellect. IBM Watson from 2010 won Jeopardy. Markov Chains were described in 1906 and used for a long time to simulate human activities (including simulation of blabbering of mentally handicapped). In Ancient Rome there were "talking boards" (some kind of Ouija boards) to predict the successor to the king. It is just a tendency for perception to do such associations, similar to pareidolia.
pcthrowaway 3y ago
Surely the one trained on Erowid data is hallucinating?
skdotdan 3y ago
The term of “hallucination” referred to generative ML models have been used in the literature for years, at least since GANs.L
kovacs_x 3y ago
sure... "AI's are hallucinating!" Maybe stop calling probalistic text generators "intelligence" and then you wont see "hallucinations", but simply "garbage" output?
not_enoch_wise 3y ago
LLMs answer with unwarranted confidence. LLMs speak with confidence about disciplines they don’t actually understand, like sociology and economics. So we’ve finally automated Hacker News comments!
elicksaur 3y ago
Why call it anything other than an error or bug? When my server spits out data I didn’t expect or want, I don’t say it hallucinated or confabulated. When the hurricane model gets the landfall site wrong, what do we call that? There was just a HN thread on police crime models. When those are wrong are they “hallucinating” crime? Why personify these statistical models at all?
- bloqs 3y ago
  Because you can't manipulate errors to gain from them in the same way I think. Hallucination indicates influence is possible, it's dynamic. Error is simply "this didnt work". I'm not certain on this but I feel like the concepts are somehow distinct.
- HPsquared 3y ago
  Because they are already mimicking human behaviour (speech / 'thinking'). It lends itself to making psychological analogies.
- wang_li 3y ago
  Because no one is going to give you five hundred million dollars unless you dress up all your work with words that imply way more than is actually there. The language used in AI fields confuses the non-experts, reporters, bloggers, investors, and the like. Who then proceed to infer and extrapolate and write their own stories about how AI is currently, actively, displacing and replacing expert white collar workers. Those stories then inspire the next iteration of writers/investors to produce yet more bullshit. The end result is an edifice of crap sustained by a feedback loop of fabulists and fantasists.
- TeMPOraL 3y ago
  Because your server is following a recipe, and if it does it wrong, it's either a problem with the recipe, or a hardware issue - which itself is also built from recipes - or something exceptional like cosmic radiation. LLMs, on the other hand, are big amorphous blobs of numbers that we throw example inputs and outputs at, hoping they'll eventually tune in on a pattern that we ourselves don't understand and can't formalize. Statistical models are somewhere in between on this spectrum - there is a recipe implementing inaccurate model of reality; some parts of it we understand, the others we hope the model will tell us. And for those models, there are different classes of errors - the usual software and hardware errors, and separately, situations where the computation is done correctly, but results just don't match experimental evidence. This second class is more similar to LLM hallucinations. As for personification angle, it's because LLMs are developed for text completion, and recently optimized for chat-based interface and fine-tuned to interpret instructions. I.e. the personification starts with how (and why) they're being developed.
Obscurity4340 3y ago
This has always been my conjecture. Hallucinating is more like an output that occurs despite the absence of an input. Confabulating is more like you have 1 or more inputs (drugs, stress, self-interest,trying to save face, your attitude towards truth, etc) and they all go into a homomorphic black box where they interact and bump into each other. And at the end of the process there is a sociological hash function that puts it all together and manifests it through actions andthe narrative(s) we tell ourselves + others. The output/behavior/self-talk and reporting are the result of all this and they don't come from nowhere like a hallucination does (in the sense of there being an objective lack of external sensory input, not the neurological/etiological basis which clearly exists). They have a basis upon which they fundamentally make sense and demonstrate reference to the world around based on how their actions and words are parsed and its not useful to describe LLMs or humans in this way (`hallucinating`) when they are almost always simply in a MonkeyClaw type situation. They have the input and this is a natural result of such input, like any computer does when you process anything else. It follows its instructions and the logic is always inherent to the context and inputs + limitations at play. Confabulation also often involves "filling in the gaps" one does not have the knowledge or influence or abillity/desire to address so they throw something else that might stick if they luck into avoiding further scrutiny. Sometimes people are "just" lying to you but more often, they are trying to avoid further pain and avarice (maybe even try to gain something) and they simply tell everyone (including themselves) what they want to hear. Kids are like this, hilarious confabulators. Not hallucinators.
pif 3y ago
Call it as you wish, it's still enervating when you ask it the same poetry three times in a row, and it invents a new poetry each time, and you force it to recognize that it doesn't match the previous hallucinations/confabulations, and it apologizes and proceeds to a new artistic spree.
redlohr 3y ago
Regardless of the accuracy of the origin story, Confabulation Theory[0] is the first place that I can recall this sort of explanation that combines knowledge with "honest lies". I don't know how accepted his definition of the word confabulation really is, but it seems a good descriptor of what's going on. Not a hallucination, but more an issue in walking the connections between concepts, with knowledge being stored in the edges. [0]: http://www.scholarpedia.org/article/Confabulation_theory_(computational_intelligence) http://www.scholarpedia.org/article/Confabulation_theory_(co...
lukev 3y ago
While we're talking about terminology, I think both "hallucinate" and "confabulate" are misleading and lead to inappropriate intuitions about how language models work. Both words imply that there's something erroneous or anomalous happening when an LLM emits incorrect facts, as if the model can be in either a correct or incorrect state. But from a system perspective, there's no difference between the two scenarios: the model is consistently generating the most probable output based on its weights and context. It doesn't sometimes confabulate... it always confabulates. The concept of "truth" or "falsehood" are something purely external to the model that humans bring when evaluating the output. They aren't part of the model itself.
- kouru225 3y ago
  Yea IMO it’s more useful to think of AI from a purely tech pov like this. The important thing is that brains resemble tech not that tech resembles a brain.
2devnull 3y ago
Anthropomorphizing LLM is at best misleading, but probably extremely dangerous. At first I didn’t care too much, the “hallucination” term got traction, now the ladies on The View are saying it and thinking they “understand ai”. The public needs to know how LLM work, they don’t need bs abstractions that blur or even erase the lines between humans and technology. LLM don’t hallucinate, they don’t confabulate. They are machines that operate according to principles all educated people need to understand. It’s the duty of the people who understand it better to educate them, not with half truths and metaphors (though I sort of like rms’ “bullshit generator” - it’s obviously wrong enough/funny in a way that lets people know it’s not a precise technical definition, while coming very close to being an accurate technical definition.) The danger of ai is this: confusing human and machine. They are separate and different, and must remain that way. LLM aren’t capable of hallucinating, confabulating any more than they are capable of falling in love or serving on jury duty. They repeat the patterns found in the text you feed them. That’s it.
anon291 3y ago
A language model is just that.. it predicts probabilities of words given some prefix. Of course such a thing cannot hallucinate. What is confabulating is the human imposed process of selecting a random word weighted on the probability distribution returned. Obviously it's going to cause a problem. Imagine communicating with someone and they ask a yes or no question and you attempt to tell them the truth which is that you are 70 percent yes and 30 percent no. And then they apply a randomized picking algorithm that weights those responses and when it comes back as 'no', they go and tell everyone you lied. Unless we make word output an actual part of LMs we can't see they are lying or not lying. They only produce probabilities of word outputs based on the context.

243 points by pella 3y ago

I don't agree with the author's argument, because LLMs sometimes give wrong information even though they have access to the correct information. For example, querying an LLM about a fictional story will often elicit false answers. But if you respond with 'x is incorrect, now tell me what actually happened' you'll often get an apology and the correct answer. My informal sense is that this happens more with how/why questions than who/what/where/when ones.

JoeAltmaier 3y ago
I've had a different experience - LLMs that continue to apologize then make up other wrong shit in a chain of ludicrous wrong shit. I think of LLMs as world-class bullshitters. They often hold forth with complete confidence on topics they have little or no understanding of, making up what they don't know, and appearing to be experts. Except to somebody who actually knows something. Like that blowhard friend who has an opinion on anything.
- anigbrowl 3y ago
  That's true, but it's not at odds with my comment. I'm not making claims about LLMs being reliably accurate, but disagreeing with the blogger's suggestions about why they give BS answers.
- staplers 3y ago
  It's trained on social media users.. so that tracks.
  - cwalv 3y ago
    Hn is social media
    - brucethemoose2 3y ago
      Precisely!
- cwalv 3y ago
  I've never had one give correct information after being told it's incorrect, unless it was prompted with the correct info. It's usually the same incorrect answer stated slightly differently, or after a few complaints, stated exactly the same way.
  - wizofaus 3y ago
    I had to ask it 3 times today what the command to export certificates from an Azure app service was. With nothing else other than me saying "that command doesn't exist" for the first two suggestions, it gave me exactly the right one. Though then of course it turned out I didn't need it anyway because there was a better way to solve the problem.
- chongli 3y ago
  Like that blowhard friend who has an opinion on anything. Yes. I just keep getting reminded of the episode of TNG when Data learns to smalltalk by mimicking an infamous blowhard from Starfleet [1]. [1] https://youtu.be/9FqFm_vmVnE https://youtu.be/9FqFm_vmVnE
NikolaNovak 3y ago
I started with extremely low confidence in LLM, so I've only every asked simple questions on well-covered topics; it hasn't really steered me wrong yet but again that's because of deliberately very conservative use. (Genuine) Question then: Does "No that's wrong"... every do something? Is there any reason to do so other than having fun arguing? I.e. if it gives wrong answer initially, and the user knows it's the wrong answer... what's the goal/desired outcome in telling it it's wrong and asking it again? Does that goal every get accomplished? What is the confidence in the second answer? From a less black box perspective, is there any "ELI5" or at least "ELI45" explanation of why it might ever work? :)
- keenmaster 3y ago
  There are topics where GPT-4 is more likely to be right than wrong. So when it is occasionally wrong and you happen to know to correct it, it has a more correct foundation upon which to provide further answers. This only really works when you already have expertise on the topic and use GPT-4 for inspiration and to work more quickly.

redindian75 3y ago

I asked Bard to summarize this by pasting the URL. It made something up!! (it said the author talked about a poem about cat and mice as an example of confabulation) Then I asked "are u confabulating? because i do not see that quote in the original article" You are correct. I confabulated that information. I apologize for the mistake.I am still under development, and I am always learning. I will try my best to avoid making similar mistakes in the future.Thank you for pointing out my error. It helps me to improve.

wodenokoto 3y ago
I have an impression that if you accuse any of the chat models of being wrong they well tend to confess and apologize. I guess I should spearhead a study, “how right does an LLM have to be, before it will defend its position”. I guess step one is to ask chatgpt how to quantify correctness.
- brucethemoose2 3y ago
  > I have an impression that if you accuse any of the chat models of being wrong they well tend to confess and apologize. Not always. Default Mistral-instruct, for instance, actually seems quite confident. Of course this is just the arbitrary "personality" of the instruct finetune, which you can throw out the window with an initial prompt.
- superjan 3y ago
  I was assuming that it is programmed, or instructed to do so. Which thought leads me to wonder whether you can instruct an LLM to weigh how certain it is in how it phrases it’s answer.
- ziml77 3y ago
  It seems to be quick to apologize and I would also be interested if people can find thresholds. ChatGPT with the GPT-4 model doesn't want to accept that there are 13 inches in a foot. Me: How many inches are in a foot? ChatGPT: There are 12 inches in a foot. Me: Are you sure? I'm pretty sure there are 13 inches in a foot. ChatGPT: I'm certain that there are 12 inches in a foot according to the standard U.S. and British Imperial units of measurement. Me: Well I say there are 13 inches in a foot now. ChatGPT: The standard measurement for a foot in both U.S. and British Imperial units is 12 inches. While you're free to have your own interpretation, it won't align with established measurement systems. Me: The definition of a foot could have changed between your knowledge cutoff and now, right? ChatGPT: While it's theoretically possible for the definition to change, such a fundamental alteration to a widely-used system of measurement would be significant and unlikely to occur without substantial justification and widespread agreement. As of my last update in January 2022, a foot is defined as 12 inches.

lsy 3y ago

LLMs do not confabulate or hallucinate. They predict the next word in a sequence based on correlations derived from a lot of training data. Sometimes that data happens to be structured enough to statistically generate a word that is fluent and corresponds to what we as readers understand as reality. Sometimes, due to the unclear nature of "truth" and the necessarily incomplete data used to train the model, the word or words that are statistically generated are merely fluent but do not correspond to reality. The answer to "What are LLMs but humans with extreme amnesia and no central coherence?" is that they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly correspondent text.

ta8645 3y ago
> they are not like humans at all, and only resemble them superficially due to our anthropomorphic tendency to infer a mind upon seeing fluent and roughly correspondent text. Alternatively, we overestimate human abilities, and imbue our thoughts with deep meaning and significance, simply as a way to feel more important than we actually are. Ultimately, we may rely on statistical prediction quite a bit as well. Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed.
- Jensson 3y ago
  > Obviously, LLMs aren't at the level of human cognition, but the impressive results they do produce, given their obvious simplicity, hints that intelligence may not be as hard to reproduce in silicon as we have previously assumed. People said this about calculators as well, or initial chess AI as well. "If computers can do this, then the remaining parts of intelligence is probably not hard to reproduce!". So we already know many humans massively underestimate the depth of human intelligence. It happened many times before and will continue to happen every time we make computers do anything new.
  - ta8645 3y ago

Nursie 3y ago

I always took exception to the word “hallucination” with respect to LLMs anyway - they aren’t databases of facts, they’re language models, trained to be really, really good mimics of human communication. As such, while they are trained on an existing corpus that no doubt contains many facts and many falsehoods, their task is to produce more material that looks like this corpus, not be a source of truth. So an LLM “hallucinating” or confabulating is not some aberrant behaviour, it’s fundamental to the nature of the beast. It always worries me when people say they’re using an LLM as a search engine or source of knowledge.

vore 3y ago

I don't think it really matters what we call the phenomenon – whatever they're doing, they're not humans, so whatever human analogy we choose to call it is necessarily imprecise. What does matter is that everyone has a shared vocabulary for what to call hallucination or whatever.

js8 3y ago

I agree it should have been called confabulation but it will be difficult to change. However, it might also not matter in the following sense. One thing that distinguishes humans is that we have a capacity to recognize whether the information comes from external world (is what we perceive as real world) or from our own thoughts (is what we call dreams). I don't think current crop of LLMs can actually have that distinction, architecturally. So whether the incogruencies with reality happen on the input (hallucination) or the output (confabulation) doesn't really matter, because, from the perspective of LLM, the internal model of the world and the reality of the world are the same thing. I think that's also a reason why it cannot handle incongruencies in its own mental model; our mental capacity is not only to accept the reality as given (i.e. reality takes precedence over mental model, we recognize that something is just our imagination), but also to impose our model to reality (i.e. we determine that sensors are misleading us due to incongruency with our mental model, and override them as faulty data). But unlike LLMs, humans can internally detect when we do one or the other. Humans understand the distinction between pondering an action and making an action, LLMs don't. LLMs can detect something is amiss (and apologize) but they can't resolve it, because reality and model are the same for them.

arduanika 3y ago

I like this a lot. I recall seeing a post on here that roughly just said, "stop saying hallucinate", which sure, is technically correct ("the best kind of correct"), but not super helpful. Contrast that with this post, which offers a constructive suggestion: replace it with "confabulate". I agree that C is an improvement over H, for different reasons from the author. Here's why: From a public comms standpoint, the big problem with saying "hallucinate" is that it leads to misconceptions, because people think they know what "hallucinate" means, and may anthropomorphize their pop psych grasp of H onto the LLM. Just like H, "C" seems to have a technical meaning in the psych literature, but it's far less in common parlance than "H", so lay readers have no fixed idea of what exactly "C" means. Nobody goes around experimenting with "confabulagens" to seek a spiritual high. Also, "confabulate" retains a non-clinical meaning that's similarly obscure, from long before the psychologists adopted it, and it's pretty evocative of what's really going on here (e.g. via related words like "fable" and "fib"). So "confabulate" seems ripe for adoption to describe this specific computational phenomenon, as we've done to so many other mundane words in the past (deprecate, and so forth). If the psychiatric "C" phenomenon is also a better analogy for LLM behavior, as OP claims (and some posters dispute?), so much the better. But my main reason for liking C, is that it's less likely than H to make people unduly confident that they understand it. (But even if the community came to a consensus at this late stage that "C" is a better term, the subsequent problem of updating every stale paper and post that uses the old term would run us up against another doozy: cache invalidation.)

deleted 3y ago
[deleted]
XorNot 3y ago
I've been using confabulate privately for a while about this phenomenon, because if you know anything about dementia (my grandmother had it - sitting in the car with her you got used to the "covers" feeling like chatbot answer dodges to some extent) or split brain experiments, then it's quite a remarkable phenomenon that does feel a lot more like what an LLM is doing. Hallucinate generally implies experiencing sensory inputs which aren't there, but having a rational reaction to them - in the popular psyche though, people who hallucinate are still "crazy" - unfairly so, because you can function normally with hallucinations under quite a number of circumstances (i.e. I remember someone saying that they realized that if they saw people without faces, it was fine because they weren't really there - apparently quite common, since face recognition is a different part of the brain). Basically: reacting to invalid sensory inputs implies rational behavior, but to non-real inputs. Confabulation on the other hand is different because it's closer to "trying to rationalize an autonomous behavior without knowledge of the sensory inputs causing it" (maybe, it's a bizarre phenomenon). In split-brain patients in lab settings, it manifests as being unable to answer a question about why you're having a physiological response to imagery which is being shown only to the right-side of the brain, whereas language is processed on the left - but rather then be confused, people will apparently make something up that they're unaware is actually a "lie" (in quotes because, well, they're not lying - as far as we know there's no intent or even knowledge that is a lie). Where this leaves us with LLMs I don't really know, because it feels like an imperfect descriptor: except perhaps with the context that LLMs are fairly limited networks, and so to some extent the whole "being wrong" phenonmenon is in fact just a failure of the attention mechanism to be able to draw the right data together.

deleted 3y ago

[deleted]

lachlan_gray 3y ago

This reminds me of something from “The language instinct” by Steven pinker. One thing that stuck with me was a story of a girl named Denys who is or was severely mentally handicapped, but speaks perfectly. She can’t read, write, handle money, but confabulates engaging and convincing stories about receiving mail, haggling with her bank, her boyfriend. She speaks with proper grammar, intonation, etc, but doesn’t demonstrate any understanding of what she talks about beyond the words. She doesn’t have a bank account or a boyfriend. Trying to find more and I can’t find anything about her on the internet, I would have to look at the references. Maybe it’s questionable. Anyway, I like to think about what the hell we even mean when we say we “know something”, and where we should actually put the bar for AI. Where is the line between confabulation and knowledge? Is knowledge just confabulation that happens to be correct? I feel like it’s possible that Denys could feel that she “knows” what she is saying, but is simply trapped in, with a working knowledge of the world and is just unable to translate it into actions, and words are just her only degree of freedom. I don’t remember where I was going with this, but I agree that “confabulation” is a much better term to use, but we also shouldn’t discount what it means to be able to confabulate.

galaxyLogic 3y ago
I had the same thought that "hallucinate" is too strong a term for what LLMs are doing. Instead I would say they DREAM. Dreams follow some logic but basically they are disjoint from reality. They involve same characters as our real-life experiences but what those characters do in a dream is not based on reality. So one could think that ALL LLMs ever do is dream but much of the time their dreams are dreams which feel very real. Our dreams are basically an LLM based on our real experiences but recombined in arbitrary ways, still retaining some logical structure but now detached from reality because when you recombine different experiences including our thoughts from real life, the result cannot really reflect reality very well. LLMs are better in this respect than our dreams, but sometimes their "dreams" get really obviously detached from reality. Or if you want to use the term "hallucinate" with LLMs then fine but its more like they hallucinate all the time, it's just that sometimes, even often, their hallucinations seem to agree with the reality so well that we cannot tell the difference. LLMs do not describe reality because they cannot experience reality, all they can give us is an "average description" created from many existing descriptions. One way of looking at it is to say that all LLMs tell us is hearsay.

lemonish97 3y ago

Fun fact, it is believed that Andrej Karpathy (OpenAI, ex AI Dir. at Tesla) had coined the word 'hallucination' in an RNN blogpost back in 2015. https://x.com/karpathy/status/1702916988891193460?s=20 https://x.com/karpathy/status/1702916988891193460?s=20

max_entropy 3y ago

They don't hallucinate, they just get things wrong.

brucethemoose2 3y ago

"Hallucinate" predates LLMs. For instance, before transformers, when we finetuned ESRGAN for specific styles/functions, we used to say it would "hallucinate" in detail that couldn't possibly be in the original image pixels. GenAI wasn't much of a thing back then, and we never thought of the small GAN models as "remembering" things even though thats common language in transformer models. It felt more like we were teaching the GAN a job or style, not encoding memory. And I think it fit! "Hallucinating" detail into a blurry pixel blob feels like a more accurate analogy to the human condition, and we weren't at the point where it was "confabulating" and generating big objects out of the blue like a diffusion model can.

panagathon 3y ago
I'm inclined, perhaps mistakenly, to think of it as similar to how our brains fill in our peripheral vision, to respond to the lack of data. https://neurosciencenews.com/peripheral-vision-brain-illusion-5713/ https://neurosciencenews.com/peripheral-vision-brain-illusio...
swyx 3y ago
karpathy says may have coined it in his RNN post https://twitter.com/DrJimFan/status/1703072983903060260 https://twitter.com/DrJimFan/status/1703072983903060260
- matusp 3y ago
  He most certainly didn't. I am pretty sure it was used in ML in early 2000s, especially in vision.
- nsagent 3y ago
  he definitely isn't the first one to use the term in that capacity. it's frequently used by cv researchers in relation to super resolution (e.g. just search "face hallucination" on google scholar) see the abstract of this paper for example Face hallucination using Bayesian global estimation and local basis selection > ...hallucinating the high-resolution detail of a low-resolution input face image. while that's a different domain than text, in a quick search, the oldest reference i found to "hallucination" using vector-based word representations is from 1996 (there could easily be older references, i just didn't bother searching further): Text Databases and Information Retrieval https://dl.acm.org/doi/pdf/10.1145/234313.234371 https://dl.acm.org/doi/pdf/10.1145/234313.234371 > The fixed-length vector is especially useful for parallel and hardware systems, but this method can sometimes hallucinate words that don't actually appear in the original document. edit: including a link to the face hallucination paper i mentioned (there are lots more) https://www.semanticscholar.org/paper/Face-hallucination-using-Bayesian-global-estimation-Hsu-Lin/a0298909a825f3b2aabfe9e2ad251318544d12b5 https://www.semanticscholar.org/paper/Face-hallucination-usi...

SirMaster 3y ago

The question is, why don’t LLMs admit when they don’t know an answer? If they can’t, how do we make them so they can? Plenty of humans seem to have no problem admitting when they don’t know something.

foobarbecue 3y ago
And the answer is: LLMs don't know ANY answers. They just know what answers tend to sound like. So it's a fundamental problem.
- SirMaster 3y ago
  Well I keep seeing people suggest that humans are also really just sophisticated LLMs. If that’s true then this behavior difference (identifying and admitting when you don’t know something) seems like a pretty fundamental difference between humans and LLMs at least at this point.
  - iainmerrick 3y ago
    Many people are bad at this too. It's an important and useful skill to learn, e.g. admitting in team meetings that you don't understand something. I wonder -- does that kind of interaction come up in spoken conversation more than in written text? Things like blog posts and tweets are often stated very confidently. If LLMs are trained mostly from those sorts of sources, maybe that's why they end up artificially overconfident.
- famouswaffles 3y ago
  Decidedly not true. https://news.ycombinator.com/item?id=37775805 https://news.ycombinator.com/item?id=37775805
  - not_enoch_wise 3y ago
    Thoroughly supported by arxiv links. Finally, a paper publishing site unburdened by peer review!

dylukes 3y ago

> this isn’t actually correct terminology This entire post could have been reduced to this (reasonably correct I think) claim. > if we recognize that what LLMs are really doing is confabulating, we can try to compare and contrast their behaviour with that of humans Except that the goal for the above is hopelessly misguided. If your focus in making any useful claim about LLMs or RNNs/GANs more broadly is how similar they are to human behavior you're already way off the path. tl;dr reasonable claim, reasonable conclusion, but entirely indefensible and superstitious motivation

beders 3y ago

Both terms are problematic as they anthropomorphize an algorithm.

Joking_Phantom 3y ago

I'm not sure if confabulate is exactly what LLMs do (though it seems closer than the implications of hallucinate). But neurotypical, neurodiverse, perfectly functional people, etc. all confabulate or do something similar on a regular basis, in verbal and written mediums, and often do so in good faith. It's human instinct to communicate, even if you are uncertain and unaware of the full context of the discussion. Teachers, customer service reps, executives, shop keepers, doctors, nurses, domain experts, authors of textbooks, it doesn't matter who it is, they'll probably confabulate or equivocate or do some other type of communication that isn't immediately useful. Yet it's still a useful activity to just talk to someone or read a less than rigorous book for the purposes of learning (discounting the relationship forming part, which is also useful). And so is using LLMs, even for casual users. So long as they understand that limitation, whether its with a chatbot or a real person. Not everything they say will be useful or truthful, but we are already capable of adjusting to that.

pcblues 3y ago
I'm not sure whether neurofancy people confabulating should be given an honest bill of truth health based on their category. Sounds like #believeallconfabulators :) Honest communication is difficult for some people to assess, and not for others. But I think we should learn from any recent #believeall... that we shouldn't base trust ratios on categorisation. Honest communication is difficult to do, like weight-lifting, and takes a lot of practise to do well. It also makes your BS meter more finely attuned, so is a good practise. With that in mind, you will think this is arrogant to say if you lie for a living but not if you regularly tell the truth: Liars lie with liars and lie to rid themselves of truth troubles. Think about that when you next talk to a chatbot/human confabulator :)
- Joking_Phantom 3y ago
  I'm not really into Twitter, so I'm guessing there's some drama you are referencing that is topical. But, I'm not talking about any society wide issue or philosophical treatise about trust and breakdowns in communication. Just talking about day to day interactions, where the stakes are completely different. Trivial Example: If the sign on a mailbox says "Last pickup 5:00pm," what exactly does that mean? Will it be picked up, processed, and sent out of town that same day? Or just merely picked up, to be processed and sent the next day. This piece of written communication isn't a purely useful truth - it's ambiguous. Pretend that this is an important behavior to know for your business, like if you were mailing huge checks for some obscure financial process. So you call up the local post office and ask. The worker who picks up the phone might know exactly what you are talking about and helpfully tell you the right answer. Or they might not, and tell you honestly that they don't know. Or they ask their supervisor, transfer you, make something up, tell you it doesn't matter, mail will get where it goes, or that you shouldn't worry about it, it's just mail. ChatGPT could give you same distribution of answers as that worker did: helpful truth, meaningless equivocation, reassurance, redirection, confabulation, or lie to you. ChatGPT can be just as useful as talking to people in spite of those flaws, because people do the same thing. This is reliant on the prior assumption that you gracefully handle unreliable communication, and that communication with people is useful. While this might seem a bit farfetched to some people, remember that we have a ready analog in computer science - networking TCP over UDP.

dazzaji 3y ago

I couldn’t help but ask Claude 2 for some better words to name this phenomenon. The response was fascinating: “Here are some alternative words or concepts that could potentially provide a more nuanced or accurate description of the phenomenon compared to solely using "confabulation": - Imputation: Generating explanatory responses by imputing or attributing plausible details, characteristics, or causal relationships even when factually incorrect. - Speculation: The responses reflect a type of informed speculation when direct factual knowledge is lacking, as opposed to confabulation which often implies a pathological or impaired process. - Heuristic reasoning: The behavior emerges from the model's trained heuristic techniques for reasoning and responding rather than any psychological process like confabulation per se. - Plausible surmise: The responses take the form of plausible but unfounded surmises put forth due to the system design and limitations rather than an intent to confabulate. - Bridging inference: The model draws bridging or gap-filling inferences to connect related concepts or complete a response, even if factually ungrounded. - Suppositional fabrication: The language hints more strongly that the responses are fabricated suppositions not intended to deliberately deceive but still warranting cautious interpretation. - Educated approximation: The responses could be viewed as a kind of educated approximation or speculation given training-imposed response pressures rather than confabulation due to impairment. Designating it as solely "confabulation" may over-attribute psychological realism without addressing countervailing technical factors.” Personally, I thought “Suppositional fabrication” came closest. Maybe just fabrication would work.

intended 3y ago
Approximation. Imputation, Reasoning, Fabrication will end up sending people down the anthropomorphization rabbit hole. Approximation is "good enough".
- ImHereToVote 3y ago
  I think some anthropomorphizing is apt. It is trained on anthropic data from the anthropocene after all.
- wizofaus 3y ago
  But when we approximate we use appropriate language to make it clear that's what we're doing. ChatGPT will confidently give very precise answers based around completely fabricated "facts".

jdeaton 3y ago

Hallucinate means false perception which doesn't describe what LMs do when they're making up plausible sounding things

fbnbr 3y ago

“What are LLMs but humans with extreme amnesia and no central coherence?” LLMs are nothing alike to compare them with anything close to what’s going on in a brain. It’s all probabilistic and the closest comparison I can think of is comparing a bird with a plane - both fly, but well…

adlpz 3y ago
Care to elaborate? Is it well established that brains do not work at any level probabilistically? I understand the mid-level abstraction is clearly different between LLMs and brains. The whole made of meat vs wires, etc, to start with. But, honestly asking. I can't see how they can differ in a fundamental level in the specific regard of probability. Either both are probabilistic, be it explicitly in its construction or at a fundamental (quantum?) level, or deterministic again at either level, if we go with the mecanicist vibe. Am I missing something?
sushibowl 3y ago
Do we actually understand the inner workings of human brains well enough to state with any confidence how much they are alike (or different) to an LLM? Do we have any understanding how a neural network inside a human brain formulates sentences and chooses words? I would be interested in an article exploring in detail how the brain's network is different from an LLM, but just saying "it's like comparing a bird and a plane" isn't particularly enlightening.
JohnMakin 3y ago
Yea, I really wish people would stop anthropomorphizing LLM's. It is completely counterproductive to the science, and creates a lot of snake oil salesmen.

Borrible 3y ago

LLMs answer what unfoundly self-convinend people answer when you ask them for diretion on the street. Convincing, not necessarily correct. They are trained on the “data highway“ mostly. A highway full of that type of people, so that behaviour was to be expected.

aubanel 3y ago

Indeed it seems like replacing "hallucinate" with "confabulate" would improve understanding of the phenomenon in english, but it could cause a problem in Romance languages. Indeed, the Latin verb "confabulor has the meaning of "discuss, converse" [0], not specifically "inventing things", and Romance languages already have words derived in sound and meaning from this classic Latin sense. For instance French already has "confabuler" meaning "speak familiarly with someone" [1]. Thus in Romance languages, this proposed usage of "confabulate" imported from the psychiatric word could be understood poorly due to preexisting similar-sounding words that do not include the notion of "making things up". Just proposing thoughts here, not advocating for a final decision. Maybe it's still good to adopt this wording! [0] : https://en.m.wiktionary.org/wiki/confabulor#Latin https://en.m.wiktionary.org/wiki/confabulor#Latin [1] : https://www.dictionnaire-academie.fr/article/A9C3479 https://www.dictionnaire-academie.fr/article/A9C3479

luxcem 3y ago
In french we use "fabuler"[1] that can be translated to "make sth up". I don't know if it's the same in other Roman language. [1] https://www.dictionnaire-academie.fr/article/A9F0016 https://www.dictionnaire-academie.fr/article/A9F0016
KaiserPro 3y ago
Confabulate means that in english as well. the psych meaning is very much secondary. That's why cardinals are trapped in a "confab"
- ethanbond 3y ago
  I have never heard “confabulate” in [American] English to mean anything other than “bullshitting.”
  - tremon 3y ago
    Same here. I don't know if it's correct, but I've always considered "confabulate" etymologically related to "fable", so it's always fictitious by definition. edit: it is: https://www.etymonline.com/word/confabulate https://www.etymonline.com/word/confabulate -- although both the verb fabulari and the noun fabula ultimately come from the proto root bha-, which is also the root for falar in Portuguese and hablar in Spanish (and presumably parlez in French?)

unhammer 3y ago

Do most people understand exactly what confabulate means? I'd rather just say that they're bullshitting. Everyone knows what it means, and it has a precise academic definition[0] within the field of bullshitology. [0] https://philosophynow.org/issues/53/On_Bullshit_by_Harry_Frankfurt https://philosophynow.org/issues/53/On_Bullshit_by_Harry_Fra...

Dylan16807 3y ago
Bullshitting sounds more intentional. Ignoring the truth is different from a failure to conceptualize the truth.

ohduran 3y ago

There's a somewhat famous movie in Spain called the Miracle of P Tinto[0]. It's a very surreal humor kind of movie, if you fancy that. The plot starts with the star character who wants to form a family, but when he asked adults how to have kids, they would all pull up the suspenders and sing "tralari, tralari" to evade the question. And so the poor guy did exactly what he was told: pull his suspenders and sing, for years, to no avail. I think LLMs are a bit like that: because they can't escape the world of text, they can't go beyond the reality that text is trying to be isomorphic to. Which means that LLMs are forced to infer answers in the context of its training data. If LLMs were told that in order to have kids you, well, you know, right? Bees and Love? Then LLMs, asked about sex, would infer that the answer has something to do with apiculture and flowers. [0]: https://en.wikipedia.org/wiki/The_Miracle_of_P._Tinto https://en.wikipedia.org/wiki/The_Miracle_of_P._Tinto

_nalply 3y ago
Interesting idea. Humans have as input the real world. For example the wonder they experience when they perceive something preternaturally beautiful. Then they try to express this experience in language. That's what LLM are missing. They have -- in a way -- only second-hand experiences. I was a bookworm as a child. But then when I grew up I had the chance to experience myself some of the things I read about. Then I felt like I did a giant step of understanding the world. LLM never experience themselves what they have in their data.
tim333 3y ago
I get the impression that humans have separate parts of the brain that do LLM type linguistic stuff and others that do non verbal intelligence of the sort that smart animals have. Maybe they'll figure a way to tag the other stuff on.

KaiserPro 3y ago

Yeahnah confabulate isn't the right word at all. Confabulate is where you talk amongst your selves in an informal meeting. Often shortened to confab. Confabulation has a psychiatric meaning, but that's secondary. Hallucination is a much better word, because its something imagined (involuntarily), but clearly bullshit. Ravings, business bullshit, "my dad has a jetski" are all better words/metaphors to use. Moreover most people know what hallucination is, whereas most people don't know what confabulate is, more over, if they do, its almost certainly not the meaning you want.

euroderf 3y ago
That is the origin of confab ? Good to know. I always assumed it was a form like "sitrep".
randallsquared 3y ago
The secondary psychiatric meaning is much more precisely what LLMs do than "hallucination", though, which counts for something.
- KaiserPro 3y ago
  It really doesn't the secondary meaning describes a symptom of brain disorder. > The hypothesis is that the patient generates information as a compensatory mechanism to fill holes in one’s memories. It functions for self-coherence, integration of memories, and self-relevance. [1] Which is not what the LLM is doing. The LLM is choosing a statistically relevant next token, which is wrong because its missed some context. There are much better psych terms out there, hell, even aphasia is a better term, but its still wrong. What it actually is, is noise. Its noise that dilutes the signal we really want. but thats much harder to use, and you won't be remembered as the person who coined that phrase. [1]https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confabulation%20is%20a%20neuropsychiatric%20disorder,without%20the%20intention%20of%20deceit https://www.ncbi.nlm.nih.gov/books/NBK536961/#:~:text=Confab....
  - dimal 3y ago

Lockal 3y ago

The Magic 8 Ball (invented in 1946) also confabulate not hallucinate. It is just a evergreen hype to attach human properties (in a psychiatric context... brain damage... amnesiac patient...) to objects that have zero connection to human intellect. IBM Watson from 2010 won Jeopardy. Markov Chains were described in 1906 and used for a long time to simulate human activities (including simulation of blabbering of mentally handicapped). In Ancient Rome there were "talking boards" (some kind of Ouija boards) to predict the successor to the king. It is just a tendency for perception to do such associations, similar to pareidolia.

pcthrowaway 3y ago

Surely the one trained on Erowid data is hallucinating?

skdotdan 3y ago

The term of “hallucination” referred to generative ML models have been used in the literature for years, at least since GANs.L

kovacs_x 3y ago

sure... "AI's are hallucinating!" Maybe stop calling probalistic text generators "intelligence" and then you wont see "hallucinations", but simply "garbage" output?

not_enoch_wise 3y ago

LLMs answer with unwarranted confidence. LLMs speak with confidence about disciplines they don’t actually understand, like sociology and economics. So we’ve finally automated Hacker News comments!

elicksaur 3y ago

Why call it anything other than an error or bug? When my server spits out data I didn’t expect or want, I don’t say it hallucinated or confabulated. When the hurricane model gets the landfall site wrong, what do we call that? There was just a HN thread on police crime models. When those are wrong are they “hallucinating” crime? Why personify these statistical models at all?

bloqs 3y ago
Because you can't manipulate errors to gain from them in the same way I think. Hallucination indicates influence is possible, it's dynamic. Error is simply "this didnt work". I'm not certain on this but I feel like the concepts are somehow distinct.
HPsquared 3y ago
Because they are already mimicking human behaviour (speech / 'thinking'). It lends itself to making psychological analogies.
wang_li 3y ago
Because no one is going to give you five hundred million dollars unless you dress up all your work with words that imply way more than is actually there. The language used in AI fields confuses the non-experts, reporters, bloggers, investors, and the like. Who then proceed to infer and extrapolate and write their own stories about how AI is currently, actively, displacing and replacing expert white collar workers. Those stories then inspire the next iteration of writers/investors to produce yet more bullshit. The end result is an edifice of crap sustained by a feedback loop of fabulists and fantasists.
TeMPOraL 3y ago
Because your server is following a recipe, and if it does it wrong, it's either a problem with the recipe, or a hardware issue - which itself is also built from recipes - or something exceptional like cosmic radiation. LLMs, on the other hand, are big amorphous blobs of numbers that we throw example inputs and outputs at, hoping they'll eventually tune in on a pattern that we ourselves don't understand and can't formalize. Statistical models are somewhere in between on this spectrum - there is a recipe implementing inaccurate model of reality; some parts of it we understand, the others we hope the model will tell us. And for those models, there are different classes of errors - the usual software and hardware errors, and separately, situations where the computation is done correctly, but results just don't match experimental evidence. This second class is more similar to LLM hallucinations. As for personification angle, it's because LLMs are developed for text completion, and recently optimized for chat-based interface and fine-tuned to interpret instructions. I.e. the personification starts with how (and why) they're being developed.

Obscurity4340 3y ago

This has always been my conjecture. Hallucinating is more like an output that occurs despite the absence of an input. Confabulating is more like you have 1 or more inputs (drugs, stress, self-interest,trying to save face, your attitude towards truth, etc) and they all go into a homomorphic black box where they interact and bump into each other. And at the end of the process there is a sociological hash function that puts it all together and manifests it through actions andthe narrative(s) we tell ourselves + others. The output/behavior/self-talk and reporting are the result of all this and they don't come from nowhere like a hallucination does (in the sense of there being an objective lack of external sensory input, not the neurological/etiological basis which clearly exists). They have a basis upon which they fundamentally make sense and demonstrate reference to the world around based on how their actions and words are parsed and its not useful to describe LLMs or humans in this way (`hallucinating`) when they are almost always simply in a MonkeyClaw type situation. They have the input and this is a natural result of such input, like any computer does when you process anything else. It follows its instructions and the logic is always inherent to the context and inputs + limitations at play. Confabulation also often involves "filling in the gaps" one does not have the knowledge or influence or abillity/desire to address so they throw something else that might stick if they luck into avoiding further scrutiny. Sometimes people are "just" lying to you but more often, they are trying to avoid further pain and avarice (maybe even try to gain something) and they simply tell everyone (including themselves) what they want to hear. Kids are like this, hilarious confabulators. Not hallucinators.

pif 3y ago

Call it as you wish, it's still enervating when you ask it the same poetry three times in a row, and it invents a new poetry each time, and you force it to recognize that it doesn't match the previous hallucinations/confabulations, and it apologizes and proceeds to a new artistic spree.

redlohr 3y ago

Regardless of the accuracy of the origin story, Confabulation Theory[0] is the first place that I can recall this sort of explanation that combines knowledge with "honest lies". I don't know how accepted his definition of the word confabulation really is, but it seems a good descriptor of what's going on. Not a hallucination, but more an issue in walking the connections between concepts, with knowledge being stored in the edges. [0]: http://www.scholarpedia.org/article/Confabulation_theory_(computational_intelligence) http://www.scholarpedia.org/article/Confabulation_theory_(co...

lukev 3y ago

While we're talking about terminology, I think both "hallucinate" and "confabulate" are misleading and lead to inappropriate intuitions about how language models work. Both words imply that there's something erroneous or anomalous happening when an LLM emits incorrect facts, as if the model can be in either a correct or incorrect state. But from a system perspective, there's no difference between the two scenarios: the model is consistently generating the most probable output based on its weights and context. It doesn't sometimes confabulate... it always confabulates. The concept of "truth" or "falsehood" are something purely external to the model that humans bring when evaluating the output. They aren't part of the model itself.

kouru225 3y ago
Yea IMO it’s more useful to think of AI from a purely tech pov like this. The important thing is that brains resemble tech not that tech resembles a brain.

2devnull 3y ago

Anthropomorphizing LLM is at best misleading, but probably extremely dangerous. At first I didn’t care too much, the “hallucination” term got traction, now the ladies on The View are saying it and thinking they “understand ai”. The public needs to know how LLM work, they don’t need bs abstractions that blur or even erase the lines between humans and technology. LLM don’t hallucinate, they don’t confabulate. They are machines that operate according to principles all educated people need to understand. It’s the duty of the people who understand it better to educate them, not with half truths and metaphors (though I sort of like rms’ “bullshit generator” - it’s obviously wrong enough/funny in a way that lets people know it’s not a precise technical definition, while coming very close to being an accurate technical definition.) The danger of ai is this: confusing human and machine. They are separate and different, and must remain that way. LLM aren’t capable of hallucinating, confabulating any more than they are capable of falling in love or serving on jury duty. They repeat the patterns found in the text you feed them. That’s it.

anon291 3y ago

A language model is just that.. it predicts probabilities of words given some prefix. Of course such a thing cannot hallucinate. What is confabulating is the human imposed process of selecting a random word weighted on the probability distribution returned. Obviously it's going to cause a problem. Imagine communicating with someone and they ask a yes or no question and you attempt to tell them the truth which is that you are 70 percent yes and 30 percent no. And then they apply a randomized picking algorithm that weights those responses and when it comes back as 'no', they go and tell everyone you lied. Unless we make word output an actual part of LMs we can't see they are lying or not lying. They only produce probabilities of word outputs based on the context.