Hallucinations are caused by humans anthropomorphizing LLMs and imagining that they possess properties that don't exist. For example, LLMs cannot test their th

Hallucinations are caused by humans anthropomorphizing LLMs and imagining that they possess properties that don't exist. For example, LLMs cannot test their thoughts against external evidence or other knowledge they may have (such as logic) to think before they output something. That's because they are a frozen computation graph with some random noise on top. Even chain of thought prompting or RL-based "reasoning" are just a pale imitation of the behavior we actually wish we could get. It is just a method of using the same model to generate some context that improves the odds of a good final result. But the model itself does not actually consider the thoughts it is <thinking>. These "thoughts" (and the response that follows them) can and do exhibit the same defects as hallucinations, because they are just more of the same. Of course, the field has made some strides in reducing hallucinations. And it's not a total mystery why some outputs make sense and others don't. For example, just like with any other statistical model, the likelihood of error increases as the input becomes more dissimilar to the training data. But also, similarity to specific training data can be a problem because of overfitting. In those cases, the model is likely to output the common pattern rather than the pattern that would make sense for the given input.