4 ms·

They don’t just seem it. They are by design. We talk about models “hallucinating” but that’s us bringing an external value judgement after the fact. The actua

by lukev 2y ago

They don’t just seem it. They are by design. We talk about models “hallucinating” but that’s us bringing an external value judgement after the fact. The actual process of token generation works precisely the same. It’d be more accurate to say that models always hallucinate.

madeofpalk 2y ago
Yes - this is what i've been saying all the time. The term 'hallucinations' is misleading because the whole point of LLMs is that they recombine all their inputs into something 'new'. They only ever hallucinate outputs - that's their whole point!
- wizzwizz4 2y ago
  Into something probable. The models that underlie these chatbots are usually overfitted, so while they usually don't repeat their training data verbatim, they can.
DougBTX 2y ago
> The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.
- lukev 2y ago
  You misunderstand. I mean that the model itself is doing exactly the same thing whether the output is a “hallucination “ or happens to be fact. There isn’t even a theoretical way to distinguish between the two cases based only on the information encoded in the model.
- skydhash 2y ago
  > it is like saying that all programs run on the same set of CPU instructions Turing machine is the embodiment of all computer programs. And then you come across the halting problem. LLMs can probably generate all books in existence, but it can't apply judgement to it. Just like you need programmers to actually write the program and verify that it correctly solves the problem. Natural languages are more flexible. There are no functions libraries, or paradigms to ease writing. And the problem space can't be specified and usually relies on shared context. Even if we could have snippets of prompts to guide text generations, the result is not that valuable.
intended 2y ago
YES. Humans can hallucinate, its a deviation from what is observable reality. All the stress people are feeling with GenAI comes from the over anthropomorphisation of ... stats. Impressive syntatic ability is not equivalent to semantic capability.
- tremon 2y ago
  The human definition of hallucination has to do with sensory experience, i.e. inputs. Saying that LLMs hallucinate means that we're ascribing them control over their inputs that they simply do not have -- by design. Or, in other words, if a chatbot really were hallucinating, it would probably start giving unprompted responses. > Humans can hallucinate, its a deviation from what is observable reality. What is "observable reality" then, for an LLM? Its training set?

by lukev 2y ago

madeofpalk 2y ago

Yes - this is what i've been saying all the time. The term 'hallucinations' is misleading because the whole point of LLMs is that they recombine all their inputs into something 'new'. They only ever hallucinate outputs - that's their whole point!

wizzwizz4 2y ago
Into something probable. The models that underlie these chatbots are usually overfitted, so while they usually don't repeat their training data verbatim, they can.

DougBTX 2y ago

> The actual process of token generation works precisely the same I’d be wary of generalising it like that, it is like saying that all programs run on the same set of CPU instructions. NNs are function approximators, where the code is expressed in model weights rather than text, but that doesn’t make all functions the same.

lukev 2y ago
You misunderstand. I mean that the model itself is doing exactly the same thing whether the output is a “hallucination “ or happens to be fact. There isn’t even a theoretical way to distinguish between the two cases based only on the information encoded in the model.
skydhash 2y ago
> it is like saying that all programs run on the same set of CPU instructions Turing machine is the embodiment of all computer programs. And then you come across the halting problem. LLMs can probably generate all books in existence, but it can't apply judgement to it. Just like you need programmers to actually write the program and verify that it correctly solves the problem. Natural languages are more flexible. There are no functions libraries, or paradigms to ease writing. And the problem space can't be specified and usually relies on shared context. Even if we could have snippets of prompts to guide text generations, the result is not that valuable.

intended 2y ago

YES. Humans can hallucinate, its a deviation from what is observable reality. All the stress people are feeling with GenAI comes from the over anthropomorphisation of ... stats. Impressive syntatic ability is not equivalent to semantic capability.

tremon 2y ago
The human definition of hallucination has to do with sensory experience, i.e. inputs. Saying that LLMs hallucinate means that we're ascribing them control over their inputs that they simply do not have -- by design. Or, in other words, if a chatbot really were hallucinating, it would probably start giving unprompted responses. > Humans can hallucinate, its a deviation from what is observable reality. What is "observable reality" then, for an LLM? Its training set?