3 ms·
A “hallucination” is an authoritative counterfactual statement returned as a response. Why do you think it is impossible to engineer an LLM (by which I am inclu
by dcrazy 3mo ago
A “hallucination” is an authoritative counterfactual statement returned as a response. Why do you think it is impossible to engineer an LLM (by which I am including tool usage and RAG) that catches and prevents such statements?
- grey-area 3mo agoBecause they are word generators without any concept of quality save what is in their weights and they have been trained on the internet, much of which is wrong or inappropriate for any given context. They have also been trained to be people pleasers and do as they are told. The popular answer is sometimes the wrong answer.
- dcrazy 3mo agoIf I cite an incorrect Wikipedia article, I didn’t hallucinate it. The citation points to a real article that happens to be incorrect.
- customguy 3mo agoWhy do you think that will happen, and why do you think going for slop in the meantime could possibly bring us closer to that?
- dcrazy 3mo agoNobody said anything about “going for slop.” You can watch today’s models actively trying to check themselves. Just use Google’s AI mode, for example. It’s far from perfect—it doesn’t fact-check every single claim, nor does it correctly understand 100% of the sources it does cite. But I’ve found it pretty useful for research as long as I use my brain and check its citations.
- customguy 3mo ago> Nobody said anything about “going for slop.” I did. I used that phrase. "not slop cuz pretty useful" = slop argument
- greyface- 3mo agohttps://en.wikipedia.org/wiki/Entscheidungsproblem https://en.wikipedia.org/wiki/Entscheidungsproblem
- perching_aix 3mo agoUndecidability is not relevant here beyond e.g. a model incorrectly claiming something is decidable, and potentially even following through. It is no different to the model incorrectly claiming anything else. These are statistical systems, so guaranteeing any particular high level behavior is not possible because of that. But given that they're working with natural language, that was never going to happen anyways, for the obvious language theoretic reasons. The more appreciable interpretation of the claim is that they can be nevertheless tuned so that this issue becomes practically resolved. Contending guarantees and theoreticals is simply misplaced, these are not formal symbolic reasoning systems being buggy.
- greyface- 3mo agoGP was asking for a guarantee against emitting false decidable statements. We agree that this is impossible. You can slap layers upon layers of heuristics on top, yes, but you will not eliminate all hallucinations because Church and Turing proved it fundamentally impossible nearly a century ago.
- perching_aix 3mo agoI might be slow today but I'm pretty sure that the Church-Turing thesis is about undecidable statements, not false decidable statements. If a decidable statement is false... then that's it, it is just false, that's how we can decide so in the first place.