Entropy scores would be, essentially, the probability the output string accrued through the inference process. It’s a poor score for how confident the inference

Entropy scores would be, essentially, the probability the output string accrued through the inference process. It’s a poor score for how confident the inference was by the LLM. It doesn’t mean correctness by any means, but a worse entropy score means, in a way, more guessing on each token.