4 ms·

The explanation of "hallucination" is quite simplified, I am sure there is more there. If there is one problem I have to pick to to trace in LLMs, I would pick

by zerop 2y ago

The explanation of "hallucination" is quite simplified, I am sure there is more there. If there is one problem I have to pick to to trace in LLMs, I would pick hallucination. More tracing of "how much" or "why" model hallucinated can lead to correct this problem. Given the explanation in this post about hallucination, I think degree of hallucination can be given as part of response to the user? I am facing this in RAG use case quite - How do I know model is giving right answer or Hallucinating from my RAG sources?

kittikitti 2y ago
I incredibly regret the term "hallucination" when the confusion matrix exists. There's much more nuance when discussing false positives or false negatives. It also opens discussions on how neural networks are trained, with this concept being crucial in loss functions like categorical cross entropy. In addition, the confusion matrix is how professionals like doctors assess their own performance which "hallucination" would be silly to use. I would go as far to say that it's misleading, or a false positive, to call them hallucinations. If your AI recalls the RAG incorrectly, it's a false positives. If your AI doesn't find the data from the RAG or believes it doesn't exist it's a false negative. Using a term like "hallucination" has no scientific merit.
- esafak 2y ago
  So you never report or pay heed to the overall accuracy?
pcrh 2y ago
The use of the term "hallucination" for LLMs is very deceptive, as it implies that there is a "mind". In ordinary terms, "hallucinations" by a machine would simply be described as the machine being useless, or not fit for purpose. For example, if a simple calculator (or even a person) returned the value "5" for 2+2= , you wouldn't describe it as "hallucinating" the answer....
- astrange 2y ago
  "Hallucination" happened because we got AI images before AI text, but "confabulation" is a better term.
miraculixx 2y ago
"Hallucination" is just to term we use to say "this result is not what it should be". The model always uses the very same process, it does not do one thing for "hallucinations" and something else for "correct" results. In a nutshell it is always predicting the next token from a joint probability distribution. That's it. All other interpretations are speculative.

by zerop 2y ago

kittikitti 2y ago

I incredibly regret the term "hallucination" when the confusion matrix exists. There's much more nuance when discussing false positives or false negatives. It also opens discussions on how neural networks are trained, with this concept being crucial in loss functions like categorical cross entropy. In addition, the confusion matrix is how professionals like doctors assess their own performance which "hallucination" would be silly to use. I would go as far to say that it's misleading, or a false positive, to call them hallucinations. If your AI recalls the RAG incorrectly, it's a false positives. If your AI doesn't find the data from the RAG or believes it doesn't exist it's a false negative. Using a term like "hallucination" has no scientific merit.

esafak 2y ago
So you never report or pay heed to the overall accuracy?

pcrh 2y ago

The use of the term "hallucination" for LLMs is very deceptive, as it implies that there is a "mind". In ordinary terms, "hallucinations" by a machine would simply be described as the machine being useless, or not fit for purpose. For example, if a simple calculator (or even a person) returned the value "5" for 2+2= , you wouldn't describe it as "hallucinating" the answer....

astrange 2y ago
"Hallucination" happened because we got AI images before AI text, but "confabulation" is a better term.

miraculixx 2y ago

"Hallucination" is just to term we use to say "this result is not what it should be". The model always uses the very same process, it does not do one thing for "hallucinations" and something else for "correct" results. In a nutshell it is always predicting the next token from a joint probability distribution. That's it. All other interpretations are speculative.