4 ms·
I agree it should have been called confabulation but it will be difficult to change. However, it might also not matter in the following sense. One thing that di
by js8 3y ago
I agree it should have been called confabulation but it will be difficult to change. However, it might also not matter in the following sense. One thing that distinguishes humans is that we have a capacity to recognize whether the information comes from external world (is what we perceive as real world) or from our own thoughts (is what we call dreams). I don't think current crop of LLMs can actually have that distinction, architecturally. So whether the incogruencies with reality happen on the input (hallucination) or the output (confabulation) doesn't really matter, because, from the perspective of LLM, the internal model of the world and the reality of the world are the same thing.
I think that's also a reason why it cannot handle incongruencies in its own mental model; our mental capacity is not only to accept the reality as given (i.e. reality takes precedence over mental model, we recognize that something is just our imagination), but also to impose our model to reality (i.e. we determine that sensors are misleading us due to incongruency with our mental model, and override them as faulty data). But unlike LLMs, humans can internally detect when we do one or the other. Humans understand the distinction between pondering an action and making an action, LLMs don't. LLMs can detect something is amiss (and apologize) but they can't resolve it, because reality and model are the same for them.