4 ms·

I have actually done quite some research into HN's and Reddit's algorithms (i.e. http://amix.dk/blog/post/19574 http://amix.dk/blog/post/19574 and http://amix.d

by amix 14y ago

I have actually done quite some research into HN's and Reddit's algorithms (i.e. http://amix.dk/blog/post/19574 http://amix.dk/blog/post/19574 and http://amix.dk/blog/post/19588 http://amix.dk/blog/post/19588 ) and I would say that using The Wilson score confidence interval would improve HN's comment sections a lot. The real meat is here: http://www.evanmiller.org/how-not-to-sort-by-average-rating.html http://www.evanmiller.org/how-not-to-sort-by-average-rating.... Even if it looks hairy it's fairly simple to implement in e.g. Python (from Reddit's code, rewritten from PyRex): https://gist.github.com/amix/5230165 https://gist.github.com/amix/5230165 Maybe pg could try this out for a few days and we could see what the results are!

deleted 14y ago
[deleted]
- ketralnis 14y ago
  For instance?
  - DannyBee 14y ago
    So, there are some that require offline calculations (I can't really get into more than this for various reasons).
    - victorh 14y ago
      Not even a Wikipedia link? This was so anti-climactic.
      - DannyBee 14y ago
        Wikipedia does not always have the answers :) Sorry, I fully intended to publish a description, but it turns out at least two of them are the subject of pending patent apps that haven't been published yet, so I can't. In any case, do note there are a wide variety of wilson variants or alternatives that may be applicable to various situations (agresti-coull, etc), as well as jeffrey's prior bayesian intervals. http://stat.wharton.upenn.edu/~tcai/paper/Binomial-Annals.pdf http://stat.wharton.upenn.edu/~tcai/paper/Binomial-Annals.pd... is a reasonable comparison of the current common methods. Anyway, I deleted the original comment since it doesn't add anything now.
    - knite 14y ago
      Perhaps I'm a bit dense today, but why are you unable to link to descriptions of one or more of these better approaches? Many people would find the information useful and interesting.
KajMagnus 14y ago
I'm actually using the algorithm that Evan Miller writes about (i.e. the lower bound of a binomial proportion confidence interval), in the discussion system that powers the demo in the linked article. I'm using the Agresti-Coull interval, http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Agresti-Coull_Interval http://en.wikipedia.org/wiki/Binomial_proportion_confidence_..., however, because it seemed easier to implement. (I didn't know about Evan Miller's article (or any other).)
zeteo 14y ago
While this is more sophisticated, I doubt it's the "correct solution". The underlying assumption of independent draws from a Bernoulli distribution is vitiated, as the order in which the siblings are read usually affects their proportion of upvotes to downvotes. It still significantly penalizes late comments, because the 95% confidence interval is much wider for them (and its inferior limit consequently lower). Last but not least, it brings down comments that are at all controversial; whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval. In short, this sorting algorithm seems ideal for quickly placing on top bland comments, that nobody really disagrees with but will upvote anyway because they haven't yet seen the much more incisive discussion below.
- KajMagnus 14y ago
  > the 95% confidence interval is much wider for them If you change this to an 80% confidence interval, it'd become narrower and might actually somewhat favor new comments, with only a few votes (upvotes)? So this might be configurable? > whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval Isn't it more likely that the 1000 upvotes and 500 downvotes is a cute kitten photo? Or something similar (a very short but strong and popular comment)? :-) And that the one that gathered 50 upvotes and no downvotes is the truly interesting read? However! If you're able to estimate how many people actually read the comment that got 50 upvotes — then you'll know if it's truly interesting, but too long to read — only 50 people have read it. Or if it's boring — 1500 people have read it. Perhaps a good sorting algorithm would be: Lower bound of confidence interval for: (upvotes - downvotes) / num-people-who-read-the-comment
  - robryan 14y ago
    Letting different comments reach the top wouldn't be such a bad thing most of the time, as long as there is more than 1 solid comment. Only time this would be bad is if there is something important, such as a correction or a key reply by the author or the person they were calling out etc where it makes sense that everyone should read first.
deleted 14y ago
[deleted]

by amix 14y ago

deleted 14y ago

[deleted]

ketralnis 14y ago
For instance?
- DannyBee 14y ago
  So, there are some that require offline calculations (I can't really get into more than this for various reasons).
  - victorh 14y ago
    Not even a Wikipedia link? This was so anti-climactic.
    - DannyBee 14y ago
      Wikipedia does not always have the answers :) Sorry, I fully intended to publish a description, but it turns out at least two of them are the subject of pending patent apps that haven't been published yet, so I can't. In any case, do note there are a wide variety of wilson variants or alternatives that may be applicable to various situations (agresti-coull, etc), as well as jeffrey's prior bayesian intervals. http://stat.wharton.upenn.edu/~tcai/paper/Binomial-Annals.pdf http://stat.wharton.upenn.edu/~tcai/paper/Binomial-Annals.pd... is a reasonable comparison of the current common methods. Anyway, I deleted the original comment since it doesn't add anything now.
  - knite 14y ago
    Perhaps I'm a bit dense today, but why are you unable to link to descriptions of one or more of these better approaches? Many people would find the information useful and interesting.

KajMagnus 14y ago

I'm actually using the algorithm that Evan Miller writes about (i.e. the lower bound of a binomial proportion confidence interval), in the discussion system that powers the demo in the linked article. I'm using the Agresti-Coull interval, http://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Agresti-Coull_Interval http://en.wikipedia.org/wiki/Binomial_proportion_confidence_..., however, because it seemed easier to implement. (I didn't know about Evan Miller's article (or any other).)

zeteo 14y ago

While this is more sophisticated, I doubt it's the "correct solution". The underlying assumption of independent draws from a Bernoulli distribution is vitiated, as the order in which the siblings are read usually affects their proportion of upvotes to downvotes. It still significantly penalizes late comments, because the 95% confidence interval is much wider for them (and its inferior limit consequently lower). Last but not least, it brings down comments that are at all controversial; whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval. In short, this sorting algorithm seems ideal for quickly placing on top bland comments, that nobody really disagrees with but will upvote anyway because they haven't yet seen the much more incisive discussion below.

KajMagnus 14y ago
> the 95% confidence interval is much wider for them If you change this to an 80% confidence interval, it'd become narrower and might actually somewhat favor new comments, with only a few votes (upvotes)? So this might be configurable? > whereas an opinion with 1000 upvotes and 500 downvotes is probably more interesting to read than one that only gathered 50 upvotes within the same interval Isn't it more likely that the 1000 upvotes and 500 downvotes is a cute kitten photo? Or something similar (a very short but strong and popular comment)? :-) And that the one that gathered 50 upvotes and no downvotes is the truly interesting read? However! If you're able to estimate how many people actually read the comment that got 50 upvotes — then you'll know if it's truly interesting, but too long to read — only 50 people have read it. Or if it's boring — 1500 people have read it. Perhaps a good sorting algorithm would be: Lower bound of confidence interval for: (upvotes - downvotes) / num-people-who-read-the-comment
- robryan 14y ago
  Letting different comments reach the top wouldn't be such a bad thing most of the time, as long as there is more than 1 solid comment. Only time this would be bad is if there is something important, such as a correction or a key reply by the author or the person they were calling out etc where it makes sense that everyone should read first.

deleted 14y ago

[deleted]