I think it might mean the opposite of what one would expect. Afaict, calibration error means something along the lines of "how often was the model wrong but con

I think it might mean the opposite of what one would expect. Afaict, calibration error means something along the lines of "how often was the model wrong but confident that the answer was correct". That means a low calibration error would be a good thing, ie the model correctly recognizes when it is unsure about answers instead of confidently stating the wrong answer.