Hallucinations are a feature, not a bug. GPT pre-training teaches the model to always produce an answer, even when it has little or not relevant training experi

Hallucinations are a feature, not a bug. GPT pre-training teaches the model to always produce an answer, even when it has little or not relevant training experience, and on average it does well at that. Part of the point of RLHF in ChatGPT was to teach the model not to hallucinate when it doesn't have good supporting experience encoded in it's weights. This helps but is not perfect. However, it seems like there might be a path to much less hallucinations with more RL training data. As others pointed out, humans hallucinate all the time we just have better training for what level of hallucination is appropriate given supporting evidence and context.