4 ms·
> Isn't your input your confidence that GPT-4 gives the correct answer You may be right that that’s the intent, however what’s the point (other than collecting
by layer8 3y ago
> Isn't your input your confidence that GPT-4 gives the correct answer
You may be right that that’s the intent, however what’s the point (other than collecting data about user confidences)? If I enter 0.3 and GPT provides a correct answer, then that doesn’t mean that the 0.3 was somehow wrong.
- jefftk 3y agoIn that case 0.3 would be more wrong than 0.4 and less wrong than 0.2. The closer your predictions are to reality over a bunch of questions, the better you understand reality.
- layer8 3y agoYou can’t really say that for a single data point. The 0.3 may be completely correct. Now, if you try ten times, things might be different.
- croes 3y agoIsn't the whole point to show you how right your confidence is about GPT's capabilities? At least the results are about the quiz taker and his confidence.
- whimsicalism 3y agoIt’s a noisy measurement of how right you were.
- kqr 3y agoYou're right. But if you enter 0.3 on average over 28 questions and the actual number of true answers differ by a lot from 8 then you have learned your general sense of GPT-4s abilities is uncalibrated.