4 ms·

> Like LLMs, we confabulate. Thank you for describing LLM behaviour that way. It's a much better description than the more popular but less apt "hallucination"

> Like LLMs, we confabulate. Thank you for describing LLM behaviour that way. It's a much better description than the more popular but less apt "hallucination".

Nevermark 2y ago
I agree, hallucination is a completely different phenomena from inadvertently filling in knowledge gaps. Hallucinating to me, is not a one off effect, but a dynamic phenomena. When our sensory processing/interpreting continues iterating, but wanders away from, or separates from, actual sensory input. Dreams being a functional example. Drugs that cause our sensory systems to be disrupted or overwhelmed by unusual internal signals, being another example.
- Retric 2y ago
  LLM’s operate in discrete steps. So that “interpretation continues iterating” is a very good description of what’s actually happening. There’s a little uncertainty in the process, so sometimes it will pick the wrong symbol at the wrong time and the entire process just spirals into nonsense. But it’s semi lucid nonsense like a dream or hallucination not line noise. The confidently stating the wrong thing bit is arguably a different though related problem. There’s making up a citations that don’t exist and there’s inserting song lyrics where a citation should be.
  - Nevermark 2y ago
    Confabulation: Inadvertent pseudo-memory fill in, within a generally reasonable stable context representation. Hallucination: Context representation becomes unstable, resulting in open ended drifting and morphing into incoherence. The scale of error and recursion of error in the latter make the effects quite different. You could be right, perhaps sometimes the initial causes might not be so different. But I would be surprised if that was true, given there doesn't seem to be much of a middle ground between the wide differences in scale and downstream effect.
    - Retric 2y ago
      You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token. I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.
      - Nevermark 2y ago
        > I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. That's a really good point. Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor. With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space! But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories. We must confabulate to recall. Statistically, sometimes we over confabulate.
deleted 2y ago
[deleted]
bunderbunder 2y ago
You're in good company with that thought. Many deep learning researchers and practitioners have suggested the same thing. Heck, even ChatGPT itself agrees: https://community.openai.com/t/hallucination-vs-confabulation/172639/3 https://community.openai.com/t/hallucination-vs-confabulatio...
- dahart 2y ago
  Hallucinate seems like a good word for image-producing neural nets (which is probably where it’s use originated?). Confabulate might be a better word for talkative LLMs, and less good for images. I wonder if any neuroscientists or psychologists agree about humans and neural nets being similar this way. It’s less than unlikely that the mechanisms are the same between people and LLMs saying not true things. Aside from there being a wide variety of reasons people fabricate untrue things, we already know the mechanisms for neural net hallucinations and confabulations. It’s a non-self-aware machine designed to output tokens or pixels or whatever, it will turn the crank and spit out something with no concept of whether it’s true or not at all times. People, on the other hand, are often using emotion to drive what they say (and often without knowing it). People will sometimes rationalize their confabulations, sometimes say untrue things based on belief, sometimes say things driven by fear or embarrassment, sometimes lie because they have ulterior motives. None of these things apply to neural nets, so the similarity between human and NN confabulations seems at best limited to superficial summaries, no?

> Like LLMs, we confabulate. Thank you for describing LLM behaviour that way. It's a much better description than the more popular but less apt "hallucination".

Nevermark 2y ago

I agree, hallucination is a completely different phenomena from inadvertently filling in knowledge gaps. Hallucinating to me, is not a one off effect, but a dynamic phenomena. When our sensory processing/interpreting continues iterating, but wanders away from, or separates from, actual sensory input. Dreams being a functional example. Drugs that cause our sensory systems to be disrupted or overwhelmed by unusual internal signals, being another example.

Retric 2y ago
LLM’s operate in discrete steps. So that “interpretation continues iterating” is a very good description of what’s actually happening. There’s a little uncertainty in the process, so sometimes it will pick the wrong symbol at the wrong time and the entire process just spirals into nonsense. But it’s semi lucid nonsense like a dream or hallucination not line noise. The confidently stating the wrong thing bit is arguably a different though related problem. There’s making up a citations that don’t exist and there’s inserting song lyrics where a citation should be.
- Nevermark 2y ago
  Confabulation: Inadvertent pseudo-memory fill in, within a generally reasonable stable context representation. Hallucination: Context representation becomes unstable, resulting in open ended drifting and morphing into incoherence. The scale of error and recursion of error in the latter make the effects quite different. You could be right, perhaps sometimes the initial causes might not be so different. But I would be surprised if that was true, given there doesn't seem to be much of a middle ground between the wide differences in scale and downstream effect.
  - Retric 2y ago
    You could be right that it’s a fundamental difference in what’s going on, but each word is a new iteration/token and most errors are more than a single token. I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. Where the boundary between the two states is arbitrarily small, but OpenAI etc has fine tuned the system so you generally see the converging / confabulation types of errors even if the difference is just a slight change in the scale of error.
    - Nevermark 2y ago
      > I’m assuming it’s like a converging vs diverging series in how well the LLM recovers from problems. That's a really good point. Similar causes diverging in effect, a phase change or not, depending on whether a critical feedback threshold is hit or not. That could certainly be a factor. With increasing doses of some drugs, humans do progress from being just a bit wonky, to full on lost in space! But I think confabulation might be on a different continuum too. We fill in recalled memories with made up details all the time. Our memory storage is lossy, associative and overlapping. Our recall is always an imperfect combination of actual incidence memory filled out by similar memories. We must confabulate to recall. Statistically, sometimes we over confabulate.

deleted 2y ago

[deleted]

bunderbunder 2y ago

You're in good company with that thought. Many deep learning researchers and practitioners have suggested the same thing. Heck, even ChatGPT itself agrees: https://community.openai.com/t/hallucination-vs-confabulation/172639/3 https://community.openai.com/t/hallucination-vs-confabulatio...

dahart 2y ago
Hallucinate seems like a good word for image-producing neural nets (which is probably where it’s use originated?). Confabulate might be a better word for talkative LLMs, and less good for images. I wonder if any neuroscientists or psychologists agree about humans and neural nets being similar this way. It’s less than unlikely that the mechanisms are the same between people and LLMs saying not true things. Aside from there being a wide variety of reasons people fabricate untrue things, we already know the mechanisms for neural net hallucinations and confabulations. It’s a non-self-aware machine designed to output tokens or pixels or whatever, it will turn the crank and spit out something with no concept of whether it’s true or not at all times. People, on the other hand, are often using emotion to drive what they say (and often without knowing it). People will sometimes rationalize their confabulations, sometimes say untrue things based on belief, sometimes say things driven by fear or embarrassment, sometimes lie because they have ulterior motives. None of these things apply to neural nets, so the similarity between human and NN confabulations seems at best limited to superficial summaries, no?