3 ms·
"it is uncontroversial to say that given grounding like this LLMs won't hallucinate" I disagree. LLMs that have been "grounded" in text that has been injected
by simonw 3y ago
"it is uncontroversial to say that given grounding like this LLMs won't hallucinate"
I disagree. LLMs that have been "grounded" in text that has been injected into their context (RAG style) can absolutely stolñ hallucinate.
They are less likely to, but that's not the same as saying they "won't hallucinate" at all.
I've spotted this myself. It's not uncommon for example for Google Gemini to include a citation link which, when followed, doesn't support the "facts" it reported.
Furthermore, if you think about how most RAG implementations work you'll spot plenty of potential for hallucination. What if the text that was pulled into the context was part of a longer paragraph that started "The following is a common misconception:" - but that prefix was omitted from the extract?