3 ms·
> reality, every LLM response is a form of hallucination, with some aligning with factual reality and others not. I like to point this out with the analogy of
by Terr_ 2y ago
> reality, every LLM response is a form of hallucination, with some aligning with factual reality and others not.
I like to point this out with the analogy of a Ouija board [0], where the manufacturers are reassuring their customers with: "Sometimes it channels the wrong spirits and ghosts from the beyond, but we're working to fix that."
Much like LLM "hallucinations", the wording implies something about "normal" operation which isn't quite true.
[0] https://en.wikipedia.org/wiki/Ouija https://en.wikipedia.org/wiki/Ouija
- dinfinity 2y agoIsn't it? The key metric is what fraction of things do actually align with factual reality. The Ouija board is a terrible analogy, because it _never_ aligns with reality. In my experience of using LLMs for various tasks, in _a lot_ of use, the output is very much in line with reality. Coding is not a domain where it shines in this. Natural language based domains, though: they're really good at saying true things and do so _way_ more often than not, to the point one could easily call the former "normal" operation.