4 ms·
I think this is a misuse of the term hallucination. When most people talk about AI hallucinating, they're referring to output which violates some desired const
by zdragnar 2y ago
I think this is a misuse of the term hallucination.
When most people talk about AI hallucinating, they're referring to output which violates some desired constraints.
In the context of chess, this would be making an invalid move, or upgrading a knight to a queen.
In other contexts, some real examples are fabricating court cases and legal precedent (several lawyers have gotten in trouble here), or a grocery store recipe generator recommending mixing bleach and ammonia for a delightful cocktail.
None of these hallucinations are an attempt to reason about anything. This is why some people oppose using the term hallucination- it is an anthropomorphizing term that gives too much credit to the AI.
We can tighten the band of errors with more data or compute efficiency or power, but in the search for generic AI, this is a dead end.
- wat10000 2y agoIt’s weird because there’s no real difference between “hallucinations” and other output. LLMs are prediction engines. Given the text so far, what’s most likely to come next? In that context, there’s very little difference between citing a real court case and citing something that sounds like a real court case. The weird thing is that they’re capable of producing any useful output at all.