9 ms·
>What human can understand it? Lets start with Wilson's midpoint, since that's just high school math. def mid(upvotes:Int, downvotes:Int) = { val t
by dxbydt 15y ago
>What human can understand it?
Lets start with Wilson's midpoint, since that's just high school math.
def mid(upvotes:Int, downvotes:Int) = {
val total = upvotes+downvotes+0.0
val up = upvotes/total
val half = 0.5
val a = total/(4+total)
val b = 4/(4+total)
a * up + b * half
}
So there are two weights a and b. Using these weights, the midpoint is a weighted average of half and the proportion of upvotes.
It should be very clear that if the total becomes large, a goes to 1 and b to zero. At that point you end up using the proportion of upvotes, just like Amazon.
Now, lets bring in the entire confidence interval.
def wilson(upvotes:Int, downvotes:Int) = {
val z = 1.96
val n = upvotes+downvotes+0.0d
val phat = upvotes/n
val lower = (phat + z*z/(2*n) - z * sqrt((phat*(1-phat)+z*z/(4*n))/n))/(1+z*z/n)
val upper = (phat + z*z/(2*n) + z * sqrt((phat*(1-phat)+z*z/(4*n))/n))/(1+z*z/n)
(lower,upper)
}
Using the author's data points, the results look like this:
val itemVotes = List((600,400),(5500,4500),(2,0), (100,1))
itemVotes.foreach( x => {
val s = mid(x._1,x._2)
val w = wilson(x._1,x._2)
printf("Up:%5d\tDown:%5d\tMid:%.3f\tW:[%.3f,%.3f]\n",
x._1, x._2,s,w._1,w._2)
})
scala> Up: 600 Down: 400 Mid:0.600 W:[0.569,0.630]
Up: 5500 Down: 4500 Mid:0.550 W:[0.540,0.560]
Up: 2 Down: 0 Mid:0.667 W:[0.342,1.000]
Up: 100 Down: 1 Mid:0.971 W:[0.946,0.998]
Basically, we prefer the lower bound of the confidence interval instead of the midpoint of that same interval.
- DanI-S 15y agoI think the original post still stands - try explaining the above to an irate customer.
- 27182818284 15y agoI've dealt with people who have wanted a retraction of negative comments to their business on the web, I explained it didn't work that way as best I could, but there isn't much you can do. Same with this, if they are irate about a low rating, you play nice, but you don't expect them to learn the ins and outs.
- robryan 15y agoYeah, most merchants only really want to participate in rating schemes when it puts them above their competitors.
- biot 15y agoThe problem isn't that it's impossible for humans to understand, nor that one can't hire the best in the field to implement this solution. The problem is that the people who need to understand the outcome won't be able to understand it. Case in point, the article mentions Amazon. Amazon could easily hire the best and brightest mathematicians and developers to make the implementation completely rigorous and provably correct. However, they're still going to field questions from merchants saying "My product has a 5-star average, but this competitor's product only has a 4.5-star average. Why the hell is my product shown further down in the list?" Similarly, you may work in the acquisitions department of BigCorp and you need to purchase some equipment. When you prepare your report for Big Boss of your recommendations, how do you explain that you are recommending something which has a 4.5-star average rating instead of the 5-star average rated one? The ratings may be 100% mathematically sound, but the users of the rating system are not the same as the producers of the rating system. So it's largely irrelevant if the producers made it perfect if the users don't understand what "perfect" means.
- gcb 15y agothe article also misses the human factor. And intent. for amazon, is it better to show the best product first or to help sell a few of the lacking-rating ones to generate more ratings and work out the uncertainty with real world data instead of crazy math?
- vog 15y agoI don't think it is a good idea to misuse your customers for that kind of testing.
- gliese1337 15y agoThey'll only have to field those questions if they display the plain averages despite not sorting on them. If we've established that average rating is the wrong metric, why would you even show it to the user/merchant/whoever? Display the same metric that you sort by, and then the merchants will be perfectly happy with higher-rated items showing up higher on the list, and Big Boss will be happy that you've chosen the #1-recommended option.