4 ms·

It really unnerves me, that we call this AI errors "hallucinations". Hallucination is an error in perception (false positive stimuli). LLM cannot hallucinate b

It really unnerves me, that we call this AI errors "hallucinations". Hallucination is an error in perception (false positive stimuli). LLM cannot hallucinate because they have no sensory organs (other than training data). I think the closest psychological conditions are confabulation (memory error) and delusion (false beliefs).

SubiculumCode 2y ago
As a memory researcher, I agree.
jakubmazanec 2y ago
I agree with you, but I'm afraid the ship has sailed... People think "hallucinations are things that don't actually exist" and "this stuff LLM says doesn't exist, therefore it hallucinates".
HarHarVeryFunny 2y ago
There really is no good psychological equivalent. About the best analogy I can come with is freestyle rapping where the genre requires that you just keep talking with minimal time to think. An LLM isn't an intelligent being deciding to talk when it has something to say - it's an LLM - a text continuation machine - that will generate a word every time the generation scaffolding pushes the button. Just like the freestyle rapper, it doesn't have the option of not saying something just because it has nothing to say. It's "concerned" about flow (just keeping the sentence going), not content.
- crabmusket 2y ago
  I love the idea of saying "the computer was just freestyling that bit".
- pixl97 2y ago
  If you seperate your brain into different components and functions, which portion of your brains work like your description? If we take something like an embodied AI that has locomotion, audio/visual IO, and the ability to perform actions and part of that action loop is triggering an LLM when does it start looking more like us?
  - HarHarVeryFunny 2y ago
    I really doubt that an LLM (a transformer) can be extended into a brain just by adding all the missing pieces. A pre-trained transformer is just fundamentally different - it's really just a rules-based expert system (cf. Cyc) not the beginnings of any cognitive architecture. That said, if you wanted to make the least-worst analogy between a pre-trained transformer and part of the brain, then I suppose it'd be like a single pass though the cortex (although the same could be said of a CNN, which tells you how poor the analogy is).
ulbu 2y ago
EXACTLY. There's an assumption that there's a system that by default produces correct representations, but has malfunctioned and produced a representation of an absent object. But isn't the whole process of ML making what would be a malfunction into a procedure? Generating humongous numbers of completely faulty representations until some emerge that look like they're mirroring reality? Is hallucination in ML not the default state? Its correctness being a judgment of an observing subject, and in all cases, a happy accident that has struck the observer as being real by only looking real. Aren't language models just playing on how impressionable we are? How easily we're tricked into believing that something is a real thing based only on superficial attributes?

SubiculumCode 2y ago

As a memory researcher, I agree.

jakubmazanec 2y ago

I agree with you, but I'm afraid the ship has sailed... People think "hallucinations are things that don't actually exist" and "this stuff LLM says doesn't exist, therefore it hallucinates".

HarHarVeryFunny 2y ago

There really is no good psychological equivalent. About the best analogy I can come with is freestyle rapping where the genre requires that you just keep talking with minimal time to think. An LLM isn't an intelligent being deciding to talk when it has something to say - it's an LLM - a text continuation machine - that will generate a word every time the generation scaffolding pushes the button. Just like the freestyle rapper, it doesn't have the option of not saying something just because it has nothing to say. It's "concerned" about flow (just keeping the sentence going), not content.

crabmusket 2y ago
I love the idea of saying "the computer was just freestyling that bit".
pixl97 2y ago
If you seperate your brain into different components and functions, which portion of your brains work like your description? If we take something like an embodied AI that has locomotion, audio/visual IO, and the ability to perform actions and part of that action loop is triggering an LLM when does it start looking more like us?
- HarHarVeryFunny 2y ago
  I really doubt that an LLM (a transformer) can be extended into a brain just by adding all the missing pieces. A pre-trained transformer is just fundamentally different - it's really just a rules-based expert system (cf. Cyc) not the beginnings of any cognitive architecture. That said, if you wanted to make the least-worst analogy between a pre-trained transformer and part of the brain, then I suppose it'd be like a single pass though the cortex (although the same could be said of a CNN, which tells you how poor the analogy is).

ulbu 2y ago

EXACTLY. There's an assumption that there's a system that by default produces correct representations, but has malfunctioned and produced a representation of an absent object. But isn't the whole process of ML making what would be a malfunction into a procedure? Generating humongous numbers of completely faulty representations until some emerge that look like they're mirroring reality? Is hallucination in ML not the default state? Its correctness being a judgment of an observing subject, and in all cases, a happy accident that has struck the observer as being real by only looking real. Aren't language models just playing on how impressionable we are? How easily we're tricked into believing that something is a real thing based only on superficial attributes?