5 ms·

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens

LLMs don't "hallucinate" or "lie." They have no intent. They're Weighted Random Word Generator Machines. They're train mathematically to create series of tokens. Whenever they get something "right," it's literally by accident. If you get that rate of accidental rightness up to 80%, and people suddenly thing the random word generator is some kind of oracle. It's not. It's a large model with an embedded space, tokens and a whole series of computationally expensive perceptron and attention blocks that generate output. The title/introduction is very baited, because it implies some "physical" connection to hallucinations in biological organism, but it's focused on trying to single out certain parts of the model. LLMs are absolutely nothing at all like a biological system, of which our brains are orders of magnitudes more complex than the machines we've built that we no longer fully understand. Believing in these LLMs as being some next stage in understanding intelligence is hubris.

pektezol 10mo ago
It’s been more than 3 years and people still can’t understand this.
- bigfishrunning 10mo ago
  People don't want to understand this.
  - ohyes 10mo ago
    People have a strong financial incentive to not understand this. It’s subprime mortgages.
- user34283 10mo ago
  Three top level comments so far, and as far as I can tell each is entirely pointless yapping about semantics around 'hallucination'. Who cares? I wonder if any of the commenters is qualified enough to understand the research at all. I am not.
- deleted 10mo ago
  [deleted]
- Miraltar 10mo ago
  Yup, three years of fighting on "correct" use of words. Maybe someday people will accept that LLMs do "hallucinate" even tho it's not the same "hallucinate" as for humans.
amelius 10mo ago
You're reasoning at the wrong abstraction level. This will not get you very far.
crazygringo 10mo ago
> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged, and it is now by overwhelming consensus the correct word. > Whenever they get something "right," it's literally by accident. This is obviously false. A great deal of training goes into making sure they usually get things right. If an infinite number of monkeys on typewriters get something right, that's by accident. Not LLM's.
- tsumnia 10mo ago
  > You're just arguing about semantics. It doesn't matter in any substantial way. While I agree for many general aspects of LLMs, I do disagree in terms of some of the meta-terms used when describing LLM behavior. For example, the idea that AI has "bias" is problematic because neural networks literally have a variable called "bias", thus of course AI will always have "bias". Plus, a biases AI is literally the purpose behind classification algorithms. But these terms, "bias" and "hallucinations", are co-opted to spin a narrative of no longer trusting AI. How in the world did creating an overly confident chatbot completely 180 years of AI progress and sentiment?
  - spwa4 10mo ago
    Terminology sucks. There is an ML technique called "hallucinating", that can really improve results. It works, for example, on Alphafold, and allows you to reverse the function of Alphafold (instead of finding the fold that matches a given protein or protein complex, find a protein complex that has a specific shape, or fits on a specific shape). It's called hallucination because it works by imagining you have the solution and then learning what the input needs to be to get that solution. Treat the input or the output as weights and learn an input that fits an output or vice-versa instead of the network. Fix what the network sees as the "real world" to match what "what you already knew", just like a hallucinating human does. You can imagine how hard it is to find papers on this technique nowadays.
markus_zhang 10mo ago
Maybe humans are just random word generators too? Just more sophisticated ones. But I think I read some pieces that we are not random word generators but something better. Forgot the word.
gchamonlive 10mo ago
I think you are reading too much into the title. "Neuron" is a totally valid way of refering to the perception unit in such models. It's got nothing to do with biology apart from the name.
lambdaone 10mo ago
"Hallucinate" is a term of art, and does not imply a philosophical commitment to whether LLMs have minds. "Confabulation" might be a more appropriate term. What is indisputable is that LLMs, even though they are 'just' word generators, are remarkably good at generating factual statements and accurate answers to problems, yet also regrettably inclined to generating apparenly equally confident counterfactual statements and bogus answers. That's all that 'hallucination' means in this context. If this work can be replicated, it may offer a way to greatly improve the signal-to-bullshit ratio of LLMs, and that will be both impressive and very useful if true.
2026iknewit 10mo ago
This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robust than 'random'. Thinking or reasoning models are also finetuning the response by walking the hyperspace and basically collecting and strengthening data. We as humans do very similiar things and no one is calling us just random word predictors... And because of this, "hallucinations -- plausible but factually incorrect outputs" is an absolut accurate description of what an LLM does when it response with a low probability output. Humans also do this often enough btw. Please stop saying an LLM is just a random word predictor.
- jaredcwhite 10mo ago
  OK, it's a semi-random word predictor.
  - 2026iknewit 10mo ago
    I'm lost on your comment. Its like you have an agenda against LLMs by now marking them as 'semi-random' and undermining the complexity and the results we get from current LLMs.
- emp17344 10mo ago
  LLMs work by “walking” “hyperspace” ? Don’t try to correct people if you don’t know what you’re talking about. LLMs are quite literally non-deterministic language models, and that’s all they are.
  - 2026iknewit 10mo ago
    Yes hyperspace: "Hyperspace is a concept from physics and science fiction referring to a higher-dimensional space" I prefer to write hyperspace instead of n-dimensional. Feel free to explain to me why you think my description is wrong. LLMs are not non-deterministic in their nature. We add noise/randomess into specific layers to make them more 'creative'/'engaging' instead of always getting the exact same response we get variations. Your whole sentence doesn't even contain a real description of what a LLM is. You say 'LLM are random LLMs'. I explained that in a different comment already, feel free to check them out.
dpweb 10mo ago
We don't understand the brain. We fully understand what LLM are doing, humans built them. The idea we don't understand what LLMs are doing is magical. Magical is good for clicks and fundraising.
- allears 10mo ago
  We know how we built the machines, but their complexity produces emergent behavior that we don't completely understand.
  - emp17344 10mo ago
    This isn’t settled science, by the way. There’s evidence that the “emergent” behaviors are just a mirage.
cyost 10mo ago
Somewhat on the subject: here's a neuroscientist reflecting on our failure to model a worm's brain, a "mere" 302 neurons (3 parts, this one is the first). https://ccli.substack.com/p/the-biggest-mystery-in-neuroscience https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie... Biological systems are hard.
dudu24 10mo ago
I'm tired of this pseudointellectual reductionist response. It's not "literally by accident" when they're trained to do something, as if we are not also machines that generate next actions based on learned neural weights and abstract (embedded) representations. Your issue is with semantics rather than content. Obviously "hallucinate" and "lie" are metaphors. Get over it. These are still emergent structures that we have a lot to learn from by studying. But I suppose any attempt by researchers to do so should be disregarded because Person On The Internet has watched the 3blue1brown series on Neural Nets and knows better. We know the basic laws of physics, but spend lifetimes studying their emergent behaviors. This is really no different.
- tracker1 10mo ago
  I just kind of wish the behavior for "hallucinations" just didn't have such confident language in the context... actual people will generally be relatively forthcoming at the edge of their knowledge or at least not show as much confidence. I know LLMs are a bit different, but that's about the best comparison I can come up with.
- directorscut82 10mo ago
  Of course they hallucinate because we are training on random mode. +Since you mentioned 3blue1brown there is an excellent video on ANN interpretation based on the works of famous researchers who attempt to provide plausible explanations about how these (transformers based) archs store and retrieve information. Randomness and stochasticity is literally the most basic components which allow all these billions of parameters to represent better embedding spaces almost hilbertian in nature and barely orthogonal as training progresses. The "emergent structures" you are mentioning are just the outcome of randomness guided by "gradiently" descending to data landscapes. There is nothing to learn by studying these frankemonsters. All these experiments have been conducted in the past (decades past) multiple times but not at this scale. We are still missing basic theorems, not stupid papers about which tech bro payed the highest electricity bill to "train" on extremely inefficient gaming hardware.

pektezol 10mo ago

It’s been more than 3 years and people still can’t understand this.

bigfishrunning 10mo ago
People don't want to understand this.
- ohyes 10mo ago
  People have a strong financial incentive to not understand this. It’s subprime mortgages.
user34283 10mo ago
Three top level comments so far, and as far as I can tell each is entirely pointless yapping about semantics around 'hallucination'. Who cares? I wonder if any of the commenters is qualified enough to understand the research at all. I am not.
deleted 10mo ago
[deleted]
Miraltar 10mo ago
Yup, three years of fighting on "correct" use of words. Maybe someday people will accept that LLMs do "hallucinate" even tho it's not the same "hallucinate" as for humans.

amelius 10mo ago

You're reasoning at the wrong abstraction level. This will not get you very far.

crazygringo 10mo ago

> LLMs don't "hallucinate" or "lie." They have no intent. You're just arguing about semantics. It doesn't matter in any substantial way. Ultimately, we need a word to distinguish factual output from confidently asserted erroneous output. We use the word "hallucinate". If we used a different word, it wouldn't make any difference -- the observable difference remains the same. "Hallucinate" is the word that has emerged, and it is now by overwhelming consensus the correct word. > Whenever they get something "right," it's literally by accident. This is obviously false. A great deal of training goes into making sure they usually get things right. If an infinite number of monkeys on typewriters get something right, that's by accident. Not LLM's.

tsumnia 10mo ago
> You're just arguing about semantics. It doesn't matter in any substantial way. While I agree for many general aspects of LLMs, I do disagree in terms of some of the meta-terms used when describing LLM behavior. For example, the idea that AI has "bias" is problematic because neural networks literally have a variable called "bias", thus of course AI will always have "bias". Plus, a biases AI is literally the purpose behind classification algorithms. But these terms, "bias" and "hallucinations", are co-opted to spin a narrative of no longer trusting AI. How in the world did creating an overly confident chatbot completely 180 years of AI progress and sentiment?
- spwa4 10mo ago
  Terminology sucks. There is an ML technique called "hallucinating", that can really improve results. It works, for example, on Alphafold, and allows you to reverse the function of Alphafold (instead of finding the fold that matches a given protein or protein complex, find a protein complex that has a specific shape, or fits on a specific shape). It's called hallucination because it works by imagining you have the solution and then learning what the input needs to be to get that solution. Treat the input or the output as weights and learn an input that fits an output or vice-versa instead of the network. Fix what the network sees as the "real world" to match what "what you already knew", just like a hallucinating human does. You can imagine how hard it is to find papers on this technique nowadays.

markus_zhang 10mo ago

Maybe humans are just random word generators too? Just more sophisticated ones. But I think I read some pieces that we are not random word generators but something better. Forgot the word.

gchamonlive 10mo ago

I think you are reading too much into the title. "Neuron" is a totally valid way of refering to the perception unit in such models. It's got nothing to do with biology apart from the name.

lambdaone 10mo ago

"Hallucinate" is a term of art, and does not imply a philosophical commitment to whether LLMs have minds. "Confabulation" might be a more appropriate term. What is indisputable is that LLMs, even though they are 'just' word generators, are remarkably good at generating factual statements and accurate answers to problems, yet also regrettably inclined to generating apparenly equally confident counterfactual statements and bogus answers. That's all that 'hallucination' means in this context. If this work can be replicated, it may offer a way to greatly improve the signal-to-bullshit ratio of LLMs, and that will be both impressive and very useful if true.

2026iknewit 10mo ago

This is still not true. "Whenever they get something "right," it's literally by accident." "the random word generator" First of, the input is not random at all which allows the question how random the output is. Second, it compresses data which has an impact on that data. Probably cleaning or adjustment which should reduce 'random' even more. It compresses data from us into concepts. A high level concept is more robust than 'random'. Thinking or reasoning models are also finetuning the response by walking the hyperspace and basically collecting and strengthening data. We as humans do very similiar things and no one is calling us just random word predictors... And because of this, "hallucinations -- plausible but factually incorrect outputs" is an absolut accurate description of what an LLM does when it response with a low probability output. Humans also do this often enough btw. Please stop saying an LLM is just a random word predictor.

jaredcwhite 10mo ago
OK, it's a semi-random word predictor.
- 2026iknewit 10mo ago
  I'm lost on your comment. Its like you have an agenda against LLMs by now marking them as 'semi-random' and undermining the complexity and the results we get from current LLMs.
emp17344 10mo ago
LLMs work by “walking” “hyperspace” ? Don’t try to correct people if you don’t know what you’re talking about. LLMs are quite literally non-deterministic language models, and that’s all they are.
- 2026iknewit 10mo ago
  Yes hyperspace: "Hyperspace is a concept from physics and science fiction referring to a higher-dimensional space" I prefer to write hyperspace instead of n-dimensional. Feel free to explain to me why you think my description is wrong. LLMs are not non-deterministic in their nature. We add noise/randomess into specific layers to make them more 'creative'/'engaging' instead of always getting the exact same response we get variations. Your whole sentence doesn't even contain a real description of what a LLM is. You say 'LLM are random LLMs'. I explained that in a different comment already, feel free to check them out.

dpweb 10mo ago

We don't understand the brain. We fully understand what LLM are doing, humans built them. The idea we don't understand what LLMs are doing is magical. Magical is good for clicks and fundraising.

allears 10mo ago
We know how we built the machines, but their complexity produces emergent behavior that we don't completely understand.
- emp17344 10mo ago
  This isn’t settled science, by the way. There’s evidence that the “emergent” behaviors are just a mirage.

cyost 10mo ago

Somewhat on the subject: here's a neuroscientist reflecting on our failure to model a worm's brain, a "mere" 302 neurons (3 parts, this one is the first). https://ccli.substack.com/p/the-biggest-mystery-in-neuroscience https://ccli.substack.com/p/the-biggest-mystery-in-neuroscie... Biological systems are hard.

dudu24 10mo ago

I'm tired of this pseudointellectual reductionist response. It's not "literally by accident" when they're trained to do something, as if we are not also machines that generate next actions based on learned neural weights and abstract (embedded) representations. Your issue is with semantics rather than content. Obviously "hallucinate" and "lie" are metaphors. Get over it. These are still emergent structures that we have a lot to learn from by studying. But I suppose any attempt by researchers to do so should be disregarded because Person On The Internet has watched the 3blue1brown series on Neural Nets and knows better. We know the basic laws of physics, but spend lifetimes studying their emergent behaviors. This is really no different.

tracker1 10mo ago
I just kind of wish the behavior for "hallucinations" just didn't have such confident language in the context... actual people will generally be relatively forthcoming at the edge of their knowledge or at least not show as much confidence. I know LLMs are a bit different, but that's about the best comparison I can come up with.
directorscut82 10mo ago
Of course they hallucinate because we are training on random mode. +Since you mentioned 3blue1brown there is an excellent video on ANN interpretation based on the works of famous researchers who attempt to provide plausible explanations about how these (transformers based) archs store and retrieve information. Randomness and stochasticity is literally the most basic components which allow all these billions of parameters to represent better embedding spaces almost hilbertian in nature and barely orthogonal as training progresses. The "emergent structures" you are mentioning are just the outcome of randomness guided by "gradiently" descending to data landscapes. There is nothing to learn by studying these frankemonsters. All these experiments have been conducted in the past (decades past) multiple times but not at this scale. We are still missing basic theorems, not stupid papers about which tech bro payed the highest electricity bill to "train" on extremely inefficient gaming hardware.