3 ms·
Another trick I have found interesting and somewhat useful: asking it to give you his confidence score for it's statements, and a lot of the time his confidence
by spyder 4y ago
Another trick I have found interesting and somewhat useful: asking it to give you his confidence score for it's statements, and a lot of the time his confidence are lower for his wrong statements but also lot of the times has high confidence in some wrong answers too (for example gets simple math wrong with 100% confidence), so not sure it's really that useful just an interesting thing to try.
- gbasin 4y agoThe RLHF process performed on that chat models actively confuses their ability to estimate confidence. The raw models are quite well calibrated though