8 ms·

Language is power. Using the word hallucinate to describe what is happening when language models invent facts is inappropriate. They lie.

scrollaway 4y ago
Lying requires intent and knowledge.
- DoItToMe81 4y ago
  And hallucination requires consciousness and sentience. There really needs to be better terminology for this.
  - grrdotcloud 4y ago
    Statistical fallout of unexpected words droppings. We humans call it "misspeaking". Speak quickly, often enough, and without sufficient training and yule misspeak.
  - abecedarius 4y ago
    Consciousness? I dunno about that. Does an ant on the right drugs not hallucinate? I agree though that it's a bad term for what current LLMs do. "Confabulate" seems closer.
    - JohnFen 4y ago
      > Does an ant on the right drugs not hallucinate? I think we have no way of knowing, just like we don't know if an ant has consciousness or not.
      - abecedarius 4y ago
        There's neuroscience research on hallucinations, for example https://www.math.utah.edu/~bresslof/publications/01-3.pdf https://www.math.utah.edu/~bresslof/publications/01-3.pdf "What Geometric Visual Hallucinations Tell Us about the Visual Cortex". The word "conscious" does not appear. > The results are sensitive to the detailed specification of the lateral connectivity and suggest that the cortical mechanisms that generate geometric visual hallucinations are closely related to those used to process edges, contours, surfaces, and textures. Now this example is only one theory, but it shows that serious people can talk in these terms about hallucination and expect to be understood and not have to argue past reviewer #2, without this whole question vitiating their discussion. Also, btw, we don't fully know that GPT-4 is not conscious, if you really think it matters. When the question comes up, people who are certain it's not usually point to the fact that it's not running continuously, as if that had any bearing on the question.
  - hammyhavoc 4y ago
    "Wrong", "a fail-state", "I'm sorry, Dave". Any of these do anything for ya?
  - gumballindie 4y ago
    Non consistent databases is how i’d call these chat toys. We made software to get accurate results but now they want to sell us the idea that a system that consistently produces inaccurate results is good … because it “hallucinates” just like hoomans. As if that’s what we needed from machines in the first place, to mimics our limitations.
  - boygobbo 4y ago
    confabulate: to replace the gaps left by a disorder of the memory with imaginary remembered experiences consistently believed to be true (Collins)
burnished 4y ago
That seems overly emotional, to lie requires intent right? It implies the will to deceive. We don't accuse people of deception when they are simply wrong after all, so it feels inappropriate to accuse a machine that is predicting the next most likely word of lieing.
- 13years 4y ago
  There is actually some basis in concept. Emergent deception ... https://bounded-regret.ghost.io/emergent-deception-optimization https://bounded-regret.ghost.io/emergent-deception-optimizat...
  - burnished 4y ago
    Much obliged! That article is off with a strong start
- dweinberger 3y ago
  Yes. Lying requires being able to discern truth from falsity. But LLMs don't know what's true even when they say true things. That's why "hallucinate" is a better word ... better but not perfect because for an LLM it's all hallucinations all the time. Some of those hallucinations (most of them) happen to turn out to be true.
norvdamage 4y ago
[dead]
hammyhavoc 4y ago
They neither lie nor hallucinate. They get it wrong. It's a fail. Is the guy who adds an edit whilst citing a seemingly credible source also a liar? No, they're wrong. Intent is everything. AI does not have intent. Don't feed the animals, don't personify the LLM.
- memen 4y ago
  Well, in some cases, negligence is sufficient for a lie or deception. E.g. forgetting to properly cite results is plagiarism, whether you intended to lie or not. Maybe it makes sense to hold the owner of the hardware that produces these <hallucinations/lies/failures/deceptions/confabulations> accountable for it. In many places with freedom of speech, this will be difficult probably.
petemir 4y ago
Words can take new meanings when they are used in other, sometimes technical, contexts.
jl2718 4y ago
I think this is more accurate for non-emotional reasons. The GPT models have actually been trained to lie in the sense that they will tell you something convincing not just about the topic, but also how they are getting the information, e.g. “I checked my sources and found…”. You can also see jailbreaks where it has been trained to reverse belief on sensitive topics. This is consistently deceptive bias rather than random variance. Imagine if Apple had a generative image model that produces fake metadata and watermarks showing the data cam from a real iPhone, and then signs it with their key. That would show obvious intent to deceive rather than attempt to accurately represent reality.
chpatrick 4y ago
I think "they bullshit" is a better way to put it: https://en.wikipedia.org/wiki/On_Bullshit https://en.wikipedia.org/wiki/On_Bullshit

scrollaway 4y ago

Lying requires intent and knowledge.

DoItToMe81 4y ago
And hallucination requires consciousness and sentience. There really needs to be better terminology for this.
- grrdotcloud 4y ago
  Statistical fallout of unexpected words droppings. We humans call it "misspeaking". Speak quickly, often enough, and without sufficient training and yule misspeak.
- abecedarius 4y ago
  Consciousness? I dunno about that. Does an ant on the right drugs not hallucinate? I agree though that it's a bad term for what current LLMs do. "Confabulate" seems closer.
  - JohnFen 4y ago
    > Does an ant on the right drugs not hallucinate? I think we have no way of knowing, just like we don't know if an ant has consciousness or not.
    - abecedarius 4y ago
      There's neuroscience research on hallucinations, for example https://www.math.utah.edu/~bresslof/publications/01-3.pdf https://www.math.utah.edu/~bresslof/publications/01-3.pdf "What Geometric Visual Hallucinations Tell Us about the Visual Cortex". The word "conscious" does not appear. > The results are sensitive to the detailed specification of the lateral connectivity and suggest that the cortical mechanisms that generate geometric visual hallucinations are closely related to those used to process edges, contours, surfaces, and textures. Now this example is only one theory, but it shows that serious people can talk in these terms about hallucination and expect to be understood and not have to argue past reviewer #2, without this whole question vitiating their discussion. Also, btw, we don't fully know that GPT-4 is not conscious, if you really think it matters. When the question comes up, people who are certain it's not usually point to the fact that it's not running continuously, as if that had any bearing on the question.
- hammyhavoc 4y ago
  "Wrong", "a fail-state", "I'm sorry, Dave". Any of these do anything for ya?
- gumballindie 4y ago
  Non consistent databases is how i’d call these chat toys. We made software to get accurate results but now they want to sell us the idea that a system that consistently produces inaccurate results is good … because it “hallucinates” just like hoomans. As if that’s what we needed from machines in the first place, to mimics our limitations.
- boygobbo 4y ago
  confabulate: to replace the gaps left by a disorder of the memory with imaginary remembered experiences consistently believed to be true (Collins)

burnished 4y ago

That seems overly emotional, to lie requires intent right? It implies the will to deceive. We don't accuse people of deception when they are simply wrong after all, so it feels inappropriate to accuse a machine that is predicting the next most likely word of lieing.

13years 4y ago
There is actually some basis in concept. Emergent deception ... https://bounded-regret.ghost.io/emergent-deception-optimization https://bounded-regret.ghost.io/emergent-deception-optimizat...
- burnished 4y ago
  Much obliged! That article is off with a strong start
dweinberger 3y ago
Yes. Lying requires being able to discern truth from falsity. But LLMs don't know what's true even when they say true things. That's why "hallucinate" is a better word ... better but not perfect because for an LLM it's all hallucinations all the time. Some of those hallucinations (most of them) happen to turn out to be true.

norvdamage 4y ago

[dead]

hammyhavoc 4y ago

They neither lie nor hallucinate. They get it wrong. It's a fail. Is the guy who adds an edit whilst citing a seemingly credible source also a liar? No, they're wrong. Intent is everything. AI does not have intent. Don't feed the animals, don't personify the LLM.

memen 4y ago
Well, in some cases, negligence is sufficient for a lie or deception. E.g. forgetting to properly cite results is plagiarism, whether you intended to lie or not. Maybe it makes sense to hold the owner of the hardware that produces these <hallucinations/lies/failures/deceptions/confabulations> accountable for it. In many places with freedom of speech, this will be difficult probably.

petemir 4y ago

Words can take new meanings when they are used in other, sometimes technical, contexts.

jl2718 4y ago

I think this is more accurate for non-emotional reasons. The GPT models have actually been trained to lie in the sense that they will tell you something convincing not just about the topic, but also how they are getting the information, e.g. “I checked my sources and found…”. You can also see jailbreaks where it has been trained to reverse belief on sensitive topics. This is consistently deceptive bias rather than random variance. Imagine if Apple had a generative image model that produces fake metadata and watermarks showing the data cam from a real iPhone, and then signs it with their key. That would show obvious intent to deceive rather than attempt to accurately represent reality.

chpatrick 4y ago

I think "they bullshit" is a better way to put it: https://en.wikipedia.org/wiki/On_Bullshit https://en.wikipedia.org/wiki/On_Bullshit