4 ms·
You can't have a probability lower than zero, but you're still able to obtain negative infinity points. Why is there an upper bound but not a lower bound?
by mikeash 10y ago
You can't have a probability lower than zero, but you're still able to obtain negative infinity points. Why is there an upper bound but not a lower bound?
- Jabbles 10y agoBecause it takes a lot of skill to be "always wrong" (you would have to know the right answer in order to answer incorrectly). No skill is represented by getting 50% right/wrong, and indeed it scores you 0 points.
- scott_s 10y agoI don't think that answers mikeash's question. He is not talking about always being wrong. He is pointing out that the incentives are not symmetric. If you say that you are 100% certain, and you are wrong, then you fail the class. If the incentives were symmetric, if you said you were 100% certain and you were correct, then you would automatically pass the class. A potential answer is: this asymmetry is similar to incentives outside the classroom. Claiming 100% certainty and being wrong can be disastrous to your reputation, and likely much more negative than the positive benefits of claiming 100% certainty and being correct.
- mikeash 10y agoGetting 50% right/wrong while expressing 100% confidence in your answers scores you negative infinity points. Indeed, getting 99.9% right/wrong while expressing 100% confidence scores you negative infinity points. But getting 100% right with 100% confidence only scores you a finite positive number of points.
- Houshalter 10y agoThat's just the way log probabilities work. Log(0) is negative infinity, but log(1) is just 0. You can have infinitely high scores if you use log odds though: http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ http://lesswrong.com/lw/mp/0_and_1_are_not_probabilities/ But that can only happen if you get every question 100% right. Whereas getting a single question 100% wrong is easy to do.
- mikeash 10y agoWhy is log probability the correct way to evaluate this?
- Houshalter 10y agoIt's convenient, because multiplying probabilities together quickly gets very very small. Typically smaller than can be represented in floating point even. Whereas logarithms make multiplication just addition, and they get small very very slowly. Personally I prefer geometric mean. Which is equivalent to the average log likelihood. It has an upper bound at 1 and lower bound at 0, and represents the "average" probability you assigned the correct answer.
- ryanmonroe 10y agoI think I read a blog post about that somewehre.