6 ms·
By design, even the same LLM, when asked the same question multiple times, will almost never hallucinate in the same way. By that (flawed) logic, you could have
by kmoser 12d ago
By design, even the same LLM, when asked the same question multiple times, will almost never hallucinate in the same way. By that (flawed) logic, you could have the same LLM check itself.
- qarl 11d agoYes - exactly. Because they don't hallucinate in the same way - they are perfect for detecting each other's hallucinations. But when you ask the SAME LLM (with the same context) they remember the hallucination so it doesn't work. You have to use a fresh one without the same context
- wakawaka28 11d agoYou can have the same LLM check itself, especially if you start over with slightly different prompts or context. They sometimes get things right and sometimes don't. Although, if you genuinely have no way to know whether the answer is right or wrong, or the errors are severe, then these checks will not accomplish much. This checking can be better than nothing.
- qarl 10d agoI disagree. When I started having my agents do an independent code review before allowing a commit - the code quality improved dramatically. I don't have numbers... but by and large the errors mostly went away. I went from struggling to get good code to just... getting good code. Is it perfect? No. But it does what I asked it to do.
- wakawaka28 9d agoI'm confused. How does this disagree with me? You can do the same process with the same model, and that is what I was talking about doing.