3 ms·

If a comment has one upvote and zero downvotes, it has a 100% upvote rate, but since there's not very much data, the system will keep it near the bottom. But if

If a comment has one upvote and zero downvotes, it has a 100% upvote rate, but since there's not very much data, the system will keep it near the bottom. But if it has 10 upvotes and only 1 downvote, the system might have enough confidence to place it above something with 40 upvotes and 20 downvotes -- figuring that by the time it's also gotten 40 upvotes, it's almost certain it will have fewer than 20 downvotes. And the best part is that if it's wrong (which it is 5% of the time), it will quickly get more data, since the comment with less data is near the top -- and when it gets that data, it will quickly correct the comment's position. The bottom line is that this system means good comments will jump quickly to the top and stay there, and bad comments will hover near the bottom. [1] http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s...

davidw 15y ago
That much was fairly clear on the site. I'm talking about the math itself.
- dxbydt 15y ago
  With zero knowledge of statistics, here's how to think about it - Wilson wants to construct some score. That score can be as low as 0 and as high as 1. So he wants some interval [x,y]. The center of that interval would obviously be c = (x+y)/2. Wilson first decides what c to pick. So Wilson says, the center c must be decided by the proportion of upvotes. But then he thinks, c must also be close to a half. So he says, okay, lets figure out c using some sort of an average. So he chooses two weights a and b. The weighted average would then be a times half plus b times the upvote proportion. He chooses those weights in such a fashion that if you have lots of data, one of the weights vanish & the other becomes unity. So the weights matter only if you have too few upvotes & downvotes. Having figured out the midpoint c, Wilson has to actually figure out the lower bound x and upper bound y. Now he draws a distribution centered at c....ok so at this point you would need to know what is a distribution, and why you would need one, whether that distribution has a skew & whether its homoskedastic & so on...which is stats 101, so I won't go there. But if you've gotten this far, you should be able to atleast see the intuition behind Wilson's procedure.
- dpritchett 15y ago
  Wilson's 1927 paper is freely available [1]. I can't say I have brushed up on my statistics enough this decade to verify the math but the basics as I read them are as follows: - You have a normal distribution (bell curve) of data points, in this case quality scores. - You wish to sort these points based on their respective vote totals. Any given data point has pos positive votes out of n total votes for that item. - You have a confidence interval, e.g. 95%. This confidence is expressed in terms of the bell curve, so a 95% confidence is within 1.96 standard deviations of the mean [2]. - You have a Ruby function accepting the aforementioned n, pos, and confidence variables and returning a decimal value representing the normalized confidence_interval_lower_bound, that is the quality score that our input data point has a 95% chance of meeting or exceeding. - Given a set of data points, evaluate the ci_lower_bound for each, and then sort them accordingly. The results will give you a best-guess sorting that accounts for the fact that some data points will have more votes cast for/against them than others. [1] http://www.med.mcgill.ca/epidemiology/hanley/tmp/Proportion/wilson_jasa_1927.pdf http://www.med.mcgill.ca/epidemiology/hanley/tmp/Proportion/... [2] http://en.wikipedia.org/wiki/1.96 http://en.wikipedia.org/wiki/1.96

davidw 15y ago

That much was fairly clear on the site. I'm talking about the math itself.

dxbydt 15y ago
With zero knowledge of statistics, here's how to think about it - Wilson wants to construct some score. That score can be as low as 0 and as high as 1. So he wants some interval [x,y]. The center of that interval would obviously be c = (x+y)/2. Wilson first decides what c to pick. So Wilson says, the center c must be decided by the proportion of upvotes. But then he thinks, c must also be close to a half. So he says, okay, lets figure out c using some sort of an average. So he chooses two weights a and b. The weighted average would then be a times half plus b times the upvote proportion. He chooses those weights in such a fashion that if you have lots of data, one of the weights vanish & the other becomes unity. So the weights matter only if you have too few upvotes & downvotes. Having figured out the midpoint c, Wilson has to actually figure out the lower bound x and upper bound y. Now he draws a distribution centered at c....ok so at this point you would need to know what is a distribution, and why you would need one, whether that distribution has a skew & whether its homoskedastic & so on...which is stats 101, so I won't go there. But if you've gotten this far, you should be able to atleast see the intuition behind Wilson's procedure.
dpritchett 15y ago
Wilson's 1927 paper is freely available [1]. I can't say I have brushed up on my statistics enough this decade to verify the math but the basics as I read them are as follows: - You have a normal distribution (bell curve) of data points, in this case quality scores. - You wish to sort these points based on their respective vote totals. Any given data point has pos positive votes out of n total votes for that item. - You have a confidence interval, e.g. 95%. This confidence is expressed in terms of the bell curve, so a 95% confidence is within 1.96 standard deviations of the mean [2]. - You have a Ruby function accepting the aforementioned n, pos, and confidence variables and returning a decimal value representing the normalized confidence_interval_lower_bound, that is the quality score that our input data point has a 95% chance of meeting or exceeding. - Given a set of data points, evaluate the ci_lower_bound for each, and then sort them accordingly. The results will give you a best-guess sorting that accounts for the fact that some data points will have more votes cast for/against them than others. [1] http://www.med.mcgill.ca/epidemiology/hanley/tmp/Proportion/wilson_jasa_1927.pdf http://www.med.mcgill.ca/epidemiology/hanley/tmp/Proportion/... [2] http://en.wikipedia.org/wiki/1.96 http://en.wikipedia.org/wiki/1.96