4 ms·

I believe the GPT4 "paper" mentions that the RHLF part destroyed the models ability to gauge it's confidence.