While the training data may be wrong, LLMs still go off the rails that outputs data that doesn't seem like it would be part of the training material in the firs

While the training data may be wrong, LLMs still go off the rails that outputs data that doesn't seem like it would be part of the training material in the first place. I think if the training data has errors or oudated information, then an LLM reproducing that information is not called hallucinations, and that's a different problem. It seems just that it's very difficult for an LLM to know if it's stating something that's part of or based on its training data or not, regardless whether it's the truth or not. For example, I think the goal is to refuse this prompt: https://chatgpt.com/share/943944d2-9b25-47a5-a4eb-23f040273b73 https://chatgpt.com/share/943944d2-9b25-47a5-a4eb-23f040273b...