3 ms·
if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the r
by 8note 10d ago
if it puts a high confidence value on a wrong answer, thats still hallucinating, no?
llm hallucinations are high probability tokens that are incorrect vs the real world
- jubilanti 10d agoCorrect, they have not made a universal all-knowing omniscient oracle, which is what would be required for "can't hallucinate".
- spencerflem 10d agoNot to be tooo pedantic, but a bot that assigned 0 confidence to everything wouldn’t hallucinate. A calculator either gets the right answer or doesn’t answer. It wouldn’t have to be all knowing as long as it knew perfectly what it doesn’t know
- baq 10d agoA quantum calculator answers in distributions.
- stpedgwdgfhgdd 10d agoIn one universe that is true, in another one not.
- baq 10d agoCopenhagen is a beautiful city.
- eru 10d agoThat seems like a weird standard. I would be happy enough with: only produces what it can verify with sources. If you eg try to remember a court case (ie produce the reference via LLM token generation only), it's easy enough to check with your data whether it really exists. Similar for following links and other references. If your data or sources are wrong, obviously your report about them will be wrong. But I wouldn't call that a hallucination.
- baq 10d agoThere isn’t a single human in this world and hasn’t ever been that meets your happy-enough standard. Make of it what you will.
- eru 10d agoIt's not a binary thing. You can get closer or further away from that standard. And humans also behave differently in different contexts. A conversation at the pub has more such hallucinations than a formal deposit in court. For the latter, a good lawyer will look at her shoes, when you ask him what colour her laces are.
- hdjrudni 10d agoWhy is that at all relevant? Humans are known to hallucinate a lot. Ask 10 different witnesses at a crime scene what they saw and they'll all report different things. A good, non-hallucinating LLM would only report things for which it has evidence. It would consult the facts every single time. It's a pain in the butt for humans to fact-check everything but LLMs can quickly look up all kinds of stuff. That's what makes them useful.
- eru 10d agoYes, and for the LLM you can do it in multiple passes. So you can bolt the fact-check / source-check pass onto whatever other system you have, without having to redesign the underlying system.
- tedbradley 7d agoThat's not true that incorrect sources means incorrect report. Often, LLMs have some sense of what is true, and due to that, they hallucinate plausible sources that appear to back that knowledge up.
- dozerly 10d agoYes, there is no magic sauce here that makes stochastic output binary if that’s what people are looking for.
- adastra22 10d agoNo, I don't believe so. Hallucinations are not "high probability" in a real sense. They are an artifact of the random walk the inference algorithm takes, which causes it to latch on to and chase attractors in the noise. This random walk behavior is necessary for chat interfaces to be useful, but are less critical to typed output predictors. I'm guessing they found some optimization that is possible if you give up caring about chat.
- elil17 10d agoWhat we would want to see if a confidence value that is in line with the actual correctness. If the value is 0.9 for 1000 different answers, then approximately 900 of those answers should be correct.