They kind of do, since the predictions are well calibrated before they go through RLHF, so inside the model activations there is some notion of confidence. Eve

They kind of do, since the predictions are well calibrated before they go through RLHF, so inside the model activations there is some notion of confidence. Even with a RLHF model, you can say "is that correct?" and after an incorrect statement it is far more likely to correct itself than after a correct statement.