I think in some circumstances you can detect hallucinations by examining the logits. Consider an LLM generating a phone number (perhaps associated with a partic

I think in some circumstances you can detect hallucinations by examining the logits. Consider an LLM generating a phone number (perhaps associated with a particular service). If the LLM knows the phone number than the logits for each token should be peaked around the actual next token. If it is hallucinating then I would guess that in some situations the logits would be more evenly distributed over the tokens representing the numbers because in the absence of any powerful conditioning on the probabilities, any number will do.