2 ms·
If someone just predicted 30% every day, you'd notice the lack of samples for any other predicted probability, and no one would judge such a discontinuous calib
by ivank 12y ago
If someone just predicted 30% every day, you'd notice the lack of samples for any other predicted probability, and no one would judge such a discontinuous calibration graph as well-calibrated.
Calibration graphs can still be gamed - you can predict dishonestly to "dent" the graph. For example, if your 70% is happening 60% of the time, predict 70% even if you're 100% sure an event will happen.