I dont know why so many people keep repeating the trivial fact that we cant "eliminate" hallucinations. We cant eliminite misinformation from google, social med

I dont know why so many people keep repeating the trivial fact that we cant "eliminate" hallucinations. We cant eliminite misinformation from google, social media or people we know either. Best we can try: 1) better filter the training data 2) design better retrieval and reranking algorithms 3) when context information is provided, make it use the sources and cite the sources (use extractive QA to highlight which part of the source is relevant. This is the type of hallucinations that we should focus on as we can compare the generated result and the context to detect the hallucinations) 4) make the llm break down its reasoning into small steps that can be validated inidividually (COT, PAL) There are some research on how to manipulate the logits during decoding to make the generated text satisfy certain contraints. I suspect that we can use these techniques to make the LLM stick to the provided context. - Controllable Text Generation with Language Constraints - Classifiers are Better Experts for Controllable Text Generation - Stay on topic with Classifier-Free Guidance