4 ms·
Your point on scoring is correct, if you're 100% confident and right on everything you would score a perfect 0. The calibration insight is in how you handle the
by convexly 6mo ago
Your point on scoring is correct, if you're 100% confident and right on everything you would score a perfect 0. The calibration insight is in how you handle the questions where you don't know the answer. Say you're highly knowledgeable and 95% confident on everything, but get 2 wrong scores compared to someone that says they are 70% confident on those same two questions. That would indicate that you are overconfident compared to the other person!
- gcanyon 6mo agoI think the 100% certain and always right scenario invalidates the calculation. In that outcome you know nothing about my (over) confidence level when I am wrong. You should either return NA in that circumstance, or keep asking questions until you have actual data to work with.