2 ms·
Some students will want to maximize their score. Some will want to maximize their probability of getting an A. Some will want to maximize their probability of n
by gjm11 10y ago
Some students will want to maximize their score. Some will want to maximize their probability of getting an A. Some will want to maximize their probability of not failing. There's obviously no scoring rule you can use that will make all of these be optimized by giving accurate probabilities.
The logarithmic scoring rule (or any other proper scoring rule) means that a student wanting to maximize their expected score is incentivized to give accurate probabilities. I don't think it's reasonable to ask for more.
- dllthomas 10y agoI agree that it's probably (let's say 60% confidence) impossible to satisfy all of these at once. That said, if we have to pick just one, I think it's worth asking what information is conveyed by the test. For instance, if all that is reported is "pass/fail", we should pick a scoring rule that maximizes a test taker's chance of passing when they are well calibrated. Most interesting to me is how different these scoring rules wind up being...