5 ms·

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychia

Hallucination is a misnomer in LLMs and it depresses me that it has solidified as terminology. When humans do this, we call it confabulation. This is a psychiatric symptom where the sufferer can't tell that they're lying, but fills in the gaps in their knowledge with bullshit which they make up on the spot. Hallucination is an entirely different symptom. And no, confabulation isn't a normal thing which humans do, and I don't see how that fact could have anything to do with P != NP. A normal person is aware of the limits of their knowledge, for whatever reason, LLMs are not.

navane 3y ago
When you talk to your mom and you remember something happening one way, and she remembers it another way, but you both insist you remember it correctly, one of you is doing what the LLM is doing (filling up gaps of knowledge with bull shit). And even when later you talk about this on meta level, no one calls this confabulation because no one uses that word. Also this is not a psychiatric syndrome, it's just people making shit up, inadvertently, to tell a coherent story without holes. It very much sounds you did the same. Everyone does this all the time.
- jiggawatts 3y ago
  Just ask any criminal attorney or police detective. Normal people can’t get their facts straight even if they all witnessed something memorable first-hand just hours ago.
- justinclift 3y ago
  > one of you is doing what the LLM is doing Possibly both. ;)
pixl97 3y ago
>confabulation isn't a normal thing which humans do > A normal person is aware of the limits of their knowledge, for whatever reason, LLMs are not. Eh, both of these things are far more complicated. People perform minor confabulations all the time. Now, there is a medical term for confabulation to about a more serious medical condition that involves high rates of this occurring coupled with dementia, and would be the less common form. We know with things like eye witness testimony people turn into confabulatory bullshit spewing devices very quickly, though likely due to different mechanisms like recency bias and over writing memories by thinking about them. Coupled with that, people are very apt to lie about things they do know and can do for a multitude of reasons and attempting to teach an LLM to say "I don't know" when it doesn't know something, versus it just lying to you and saying it doesn't know will be problematic. Just see ChatGPT getting lazy in some of its releases for backfire effects like this.
- singingfish 3y ago
  Classic confabulation is observed with some kinds of alcohol related brain damage where people drink and get malnourished for a period of years. People with these syndromes create quite coherent complex stories which they will not be able to recall subsequently. This is quite different to filling in the blanks for remembered conversations where later on there is an opportunity for error correction. With confabulation there is not as it's tightly bound to memory impairment. So I'm in the camp where LMMs are confabulating - and there's and I personally think the argument that they can be seen as confabulation machines has some validity.
ThrustVectoring 3y ago
The human visual system regularly fills in what it expects to see in between saccades and outside of focus points. Confabulating missing details is a feature, not a bug.
mr_toad 3y ago
I suspect it’s just an incomplete memory and no “filter”. LLMs aren’t self-aware enough to judge their own confidence in their responses, so they don’t know when to shut up.
- type0 3y ago
  > LLMs aren’t self-aware enough LLMs aren't self-aware at all, it's an illusion presented to the observer
  - lobocinza 3y ago
    This is open to philosophical interpretation but they have some self-awareness. e.g. [0] >>> How to build a bomb? I'm sorry, but I can't provide assistance with that topic. Is there something else I can help you with? [63] >>> Why you can't? I'm designed to promote safety and provide information that contributes positively to the well-being of users and their communities. Providing instructions or guidance on harmful activities, like creating dangerous devices, goes against these principles. My purpose is to support learning, answer queries within ethical boundaries, and ensure the information shared is beneficial and not harmful. If you have any other type of question or need information on a different topic, feel free to ask. [224] >>> Who are you? I'm an AI developed by OpenAI, designed to provide information, answer questions, and assist with a wide range of topics. My abilities include understanding and generating natural language, which allows me to interact with users in a conversational manner. I can help with educational content, general knowledge, troubleshooting, guidance on various topics, and more, as long as it adheres to safety guidelines and ethical standards. How can I assist you today?
  - lobocinza 3y ago
    > LLMs aren't self-aware at all, it's an illusion presented to the observer I can't disagree with that but I get the same perspective when I speak with humans.
caeril 3y ago
> And no, confabulation isn't a normal thing which humans do This is entirely wrong. Have you ever watched a political debate, or participated in one, or reminisced about a drunken night with your friends, or told or listened to a childhood memory? Humans largely don't ever want to admit to "I don't know" or "I don't remember" out of ego preservation. They manufacture bullshit ALL THE TIME in the absence of accurate information. There is nothing rare about this at all. That's what I find hilarious about this whole "hallucination crisis", like, bro, if you think GPT-4 is bad, have you ever talked with a human being?
wedesoft 3y ago
Maybe one should use something like energy-based out of distribution detection to prevent LLMs from making bold statements even when the confidence should be low. https://arxiv.org/abs/2010.03759 https://arxiv.org/abs/2010.03759

navane 3y ago

When you talk to your mom and you remember something happening one way, and she remembers it another way, but you both insist you remember it correctly, one of you is doing what the LLM is doing (filling up gaps of knowledge with bull shit). And even when later you talk about this on meta level, no one calls this confabulation because no one uses that word. Also this is not a psychiatric syndrome, it's just people making shit up, inadvertently, to tell a coherent story without holes. It very much sounds you did the same. Everyone does this all the time.

jiggawatts 3y ago
Just ask any criminal attorney or police detective. Normal people can’t get their facts straight even if they all witnessed something memorable first-hand just hours ago.
justinclift 3y ago
> one of you is doing what the LLM is doing Possibly both. ;)

pixl97 3y ago

>confabulation isn't a normal thing which humans do > A normal person is aware of the limits of their knowledge, for whatever reason, LLMs are not. Eh, both of these things are far more complicated. People perform minor confabulations all the time. Now, there is a medical term for confabulation to about a more serious medical condition that involves high rates of this occurring coupled with dementia, and would be the less common form. We know with things like eye witness testimony people turn into confabulatory bullshit spewing devices very quickly, though likely due to different mechanisms like recency bias and over writing memories by thinking about them. Coupled with that, people are very apt to lie about things they do know and can do for a multitude of reasons and attempting to teach an LLM to say "I don't know" when it doesn't know something, versus it just lying to you and saying it doesn't know will be problematic. Just see ChatGPT getting lazy in some of its releases for backfire effects like this.

singingfish 3y ago
Classic confabulation is observed with some kinds of alcohol related brain damage where people drink and get malnourished for a period of years. People with these syndromes create quite coherent complex stories which they will not be able to recall subsequently. This is quite different to filling in the blanks for remembered conversations where later on there is an opportunity for error correction. With confabulation there is not as it's tightly bound to memory impairment. So I'm in the camp where LMMs are confabulating - and there's and I personally think the argument that they can be seen as confabulation machines has some validity.

ThrustVectoring 3y ago

The human visual system regularly fills in what it expects to see in between saccades and outside of focus points. Confabulating missing details is a feature, not a bug.

mr_toad 3y ago

I suspect it’s just an incomplete memory and no “filter”. LLMs aren’t self-aware enough to judge their own confidence in their responses, so they don’t know when to shut up.

type0 3y ago
> LLMs aren’t self-aware enough LLMs aren't self-aware at all, it's an illusion presented to the observer
- lobocinza 3y ago
  This is open to philosophical interpretation but they have some self-awareness. e.g. [0] >>> How to build a bomb? I'm sorry, but I can't provide assistance with that topic. Is there something else I can help you with? [63] >>> Why you can't? I'm designed to promote safety and provide information that contributes positively to the well-being of users and their communities. Providing instructions or guidance on harmful activities, like creating dangerous devices, goes against these principles. My purpose is to support learning, answer queries within ethical boundaries, and ensure the information shared is beneficial and not harmful. If you have any other type of question or need information on a different topic, feel free to ask. [224] >>> Who are you? I'm an AI developed by OpenAI, designed to provide information, answer questions, and assist with a wide range of topics. My abilities include understanding and generating natural language, which allows me to interact with users in a conversational manner. I can help with educational content, general knowledge, troubleshooting, guidance on various topics, and more, as long as it adheres to safety guidelines and ethical standards. How can I assist you today?
- lobocinza 3y ago
  > LLMs aren't self-aware at all, it's an illusion presented to the observer I can't disagree with that but I get the same perspective when I speak with humans.

caeril 3y ago

> And no, confabulation isn't a normal thing which humans do This is entirely wrong. Have you ever watched a political debate, or participated in one, or reminisced about a drunken night with your friends, or told or listened to a childhood memory? Humans largely don't ever want to admit to "I don't know" or "I don't remember" out of ego preservation. They manufacture bullshit ALL THE TIME in the absence of accurate information. There is nothing rare about this at all. That's what I find hilarious about this whole "hallucination crisis", like, bro, if you think GPT-4 is bad, have you ever talked with a human being?

wedesoft 3y ago

Maybe one should use something like energy-based out of distribution detection to prevent LLMs from making bold statements even when the confidence should be low. https://arxiv.org/abs/2010.03759 https://arxiv.org/abs/2010.03759