4 ms·
> This "hallucination" come along a lot recently. Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago. > Is it
by masklinn 3y ago
> This "hallucination" come along a lot recently.
Couldn’t exactly be otherwise given how young GPT is. ChatGPT was released a bit under 7 months ago.
> Is it a legit concept or just "the dog ate my homework" type of excuse for anything?
It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens.
So essentially it always hallucinates (makes shit up as it goes along, if you prefer).
Thanks to the model it’s generally quite credible, and often even lines up with actual reality, but it should not be confused for knowledge.
That’s why it will confidently give you citations it just made up, to papers or decisions it’ll happily make up as well (though less and less credibly as things get closer to hard facts).
- Sharlin 3y agoWhich is why "hallucination" is really the wrong word to use, "confabulation" would be more proper. But "hallucination" has stuck because it's the word used back when people first figured out the trick of running image classifiers "in reverse" to generate images from noise.
- masklinn 3y agoSure but nobody knows the word “confabulation”, and lying / making things up implies intent. So “hallucination” hews close enough to have good explanatory powers.
- Sharlin 3y agoConfabulation is unintentional, FWIW: > In psychology, confabulation is a memory error defined as the production of fabricated, distorted, or misinterpreted memories about oneself or the world. […] Confabulation occurs when individuals mistakenly recall false information, without intending to deceive.
- gpderetta 3y ago> It’s an analogy for how LLMs work. An LLM does not know anything, it just adds tokens probabilistically based on the previous tokens This seem a deep statement that keeps getting repeated, but it doesn't mean anything. The probabilistic model that is used to decide the next token could be arbitrarily complex, including encoding knowledge (or just asking a panel of experts). It seems pretty self evident that the model in fact encodes knowledge, just in a very lossy way and recall is also flawed.
- seba_dos1 3y agoIt sure does encode some knowledge, because it's a language model and languages already do so on their own. It's far from what you'd usually call a "knowledge model" though.
- est 3y ago> it just adds tokens probabilistically based on the previous tokens I mean, isn't this what humans do all the time? Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff.
- masklinn 3y ago> I mean, isn't this what humans do all the time? No? Most humans don’t randomly vomit text based on what sounds good. > Bullshitting random topics on the Internet, except humans tend to add disclaimers like "I am not a lawyer but" and stuff. Which shows a much higher level of understanding, both of the field (which may be flawed), and of their own understanding of the field (which they point out). An LLM does not to that, it doesn’t just repeat potentially wrong hearsay or incorrect memories (let alone having actual understanding and knowledge of the field), it confidently writes out delusions.
- est 3y ago> Most humans don’t randomly vomit text based on what sounds good. Unless humans were given a task? e.g. taking exams while un-prepared. My kid usually gives me a long description of imaginary stuff based on the name only or brief intro. It's very fun when finally the real deal was revealed.
- seba_dos1 3y agoThat's absolutely right. That said, people don't usually take exam output of unprepared students and expect it to be useful :)
- number6 3y ago"As a Language Model" is the new "I am not a lawyer"
- deleted 3y ago[deleted]