3 ms·

Well it's interesting - many LLM researchers reject "hallucination" in favour of the term "confabulation" but I'll double down and say that I still don't like e

Well it's interesting - many LLM researchers reject "hallucination" in favour of the term "confabulation" but I'll double down and say that I still don't like either as they are both terms from clinical psych and I'll include a quote from an article[1] interviewing Usama Fayyad (executive director of Northeastern’s Institute for Experiential AI) ''' “I could say — these models hallucinated; or, to be more precise, I could say, well, the model made an error, and we understand that these models make errors,” Fayyad says. ''' For years we called the behaviour that researchers now call "hallucination" test error or misprediction or a failure to forecast correct outputs based on "out-of-sample" inputs and those terms were accurate and would still be accurate to use on today's LLM models. Metaphors are great but can be misleading, people don't often make an argument that a computer mouse can feel pain. [1] https://news.northeastern.edu/2023/11/10/ai-chatbot-hallucinations/ https://news.northeastern.edu/2023/11/10/ai-chatbot-hallucin...

sterlind 2y ago
"test error" or "misprediction" aren't very applicable terms for RL, though. is it a "test error" if the LLM suggests a bad chess move? is it a "misprediction" if the LLM can't find a mating sequence? are those "failures to forecast out-of-sample inputs," or only if the LLM suggests an illegal chess move or call an API that doesn't exist? I say that a model "hallucinates" only if it specifically presents false information as fact - e.g. that Bolognium is element 115, or that chocolate milk comes from brown cows. giving irrelevant or undesired answers isn't hallucination or confabulation. but I don't think your terms are precise enough for that nuance.
- acc_297 2y ago
  I grant that the term has become commonplace and very catchy and no one has been able to supplant it with a better descriptive I still hold that there is good reason to be concerned with researchers claiming to objectively evaluate a silicon based sentience and jumping the gun a bit (imo) by writing articles about LLMs which "dream" and "hallucinate" when the behaviour is a consequence of essentially a non-linear regression model that does not yet show clear signs of the high order emotion/reasoning/awareness/intention Many who research LLMs avoid these terms in their publications for good reason (again imo)

sterlind 2y ago

"test error" or "misprediction" aren't very applicable terms for RL, though. is it a "test error" if the LLM suggests a bad chess move? is it a "misprediction" if the LLM can't find a mating sequence? are those "failures to forecast out-of-sample inputs," or only if the LLM suggests an illegal chess move or call an API that doesn't exist? I say that a model "hallucinates" only if it specifically presents false information as fact - e.g. that Bolognium is element 115, or that chocolate milk comes from brown cows. giving irrelevant or undesired answers isn't hallucination or confabulation. but I don't think your terms are precise enough for that nuance.

acc_297 2y ago
I grant that the term has become commonplace and very catchy and no one has been able to supplant it with a better descriptive I still hold that there is good reason to be concerned with researchers claiming to objectively evaluate a silicon based sentience and jumping the gun a bit (imo) by writing articles about LLMs which "dream" and "hallucinate" when the behaviour is a consequence of essentially a non-linear regression model that does not yet show clear signs of the high order emotion/reasoning/awareness/intention Many who research LLMs avoid these terms in their publications for good reason (again imo)