3 ms·
Smarter Ranking with Bayesian Average Ratings
- contravariant 9y agoThe proof in the mathematical appendix can be simplified and generalized a bit. To avoid confusion I'll let 'S' be the score and 'R' the random variable representing the actual rating. In that case the 'loss' (denoted by L) as defined in the article is (R - S) if R > S and k(S - R) if R < S. Now the goal is to minimize the expected loss E[L]. Assuming everything is well behaved, the optimal value S should satisfy: dE[L] / dS = 0. and therefore 0 = E[ dL / dS ] = E[ dL/dS | R>S ] P[R>S] + E[ dL/dS | R<S ] P[R<S] = (-1) P[R<S] + (k) P[R<S] = (P[R<S] - 1) + k P[R<S]. Solving this last line results in the formula: P[R<S] = 1 / (1+k). Now note that P[R<S] is the cumulative distribution of R, which in the case that R has a beta distribution is the incomplete beta function. In fact, all the formula tells you is that if being too high is k times worse than being too low, you should try to place your estimate such that it is k times more likely that you're too low than that you're too high. Which seems kind of obvious really, but I guess it's nice to have a proof.