12 ms·

I don't care about total points per se, I care about the probability that I'll appreciate having read something. What'd be a good proxy for this is the upvote r

by abeppu 16y ago

I don't care about total points per se, I care about the probability that I'll appreciate having read something. What'd be a good proxy for this is the upvote rate, roughly upvotes divided by views (or rather, a MAP estimate of the upvote rate as the parameter of a binomial distribution, drawn from a beta prior...). The upvote count system, and the logged upvote count system both have the problem that if comment A has been seen 500 times and has 20 upvotes, and comment B has been seen 1000 times and has 30 upvotes, A is probably actually a bit better than B. An upvote rate fixes that problem, and still gives you a useful indicator about which comments are worth your attention. The issue with upvote rates is that they're likely to be relatively low, and arithmetic differences between low probabilities aren't useful, so we should really use the log upvote rate -- but people aren't good at dealing with negative numbers, so it should be mapped to color or size or opacity of some indicator symbol or something. Suppose you had just a little dot next to each comment, the opacity of which is 100 + k * <log upvote rate>. Suppose k=8; exp(-12) is like 6 upvotes per million views, so almost no comment's dot would totally vanish, but if you had a 5% upvote rate your dot would be at ~75% opacity.

6ren 16y ago
The ordering of comments approximates this. It would be nice if there was some way to re-nest comments, based on scores, while retaining their logical relationship (as replies). e.g. It could be done with lines or labels, but it sounds too messy.
- pygy_ 16y ago
  The ordering is influenced by other factors, I guess either the post length or the average karma of the poster... I've had several comments both older and with less points than others at the same level, but that were still above the rest.
  - Mz 16y ago
    The ordering is influenced by other factors, I guess either the post length or the average karma of the poster... My understanding is that average karma of the poster influences it. I used to be more impressed but my average karma score has been as high as about 6 but most of the time sits somewhere around 2, give or take. One good debate where multiple comments of mine got substantially upvoted raised it that high and then it steadily slid (ie regularly changed if you checked it) until it was back down to around 2. Maybe other people have high averages that are more stable than mine but this experience has changed my view of average karma and made me somewhat less impressed. From what I gather, it's a moving, weighted average. It isn't the average across all comments for all of your history. I can understand the logic of that -- if average for all comments were used and you have been here forever and your comment quality has gone up, a million old comments that were never voted up could prevent you from ever seeing any substantial improvements in that area. But, like all such things, it's a bit of a game in some sense.
    - 6ren 16y ago
      Yes, it's the average of recent comments, with the top and bottom discarded (pg said this somewhere or other). The averages of the top 100 (by abs karma) is interesting: http://news.ycombinator.com/leaders http://news.ycombinator.com/leaders (accessible via "lists", frontpage bottom). BTW: edw regularly has amazing averages - all the avgs are much higher than usual now, I suspect because of karma inflation due to not seeing scores (this happened last time pg experimented with it). And yes, it influences the ordering of ones comments. When I said "approximates", I really meant it. ;-)
- 6ren 16y ago
  FOLLOWUP thinking more on re-nesting, slashdot detaches high-scoring comments from their logical nesting (and you have to follow parent links to find the post to which they were replying). I think slashdot knows much of this - I wonder if it would be helpful for cmdtaco and pg to chat (even, a voting-site conference)... or is it still too proprietary/competitive? Another way is too hide a comment if it has lower karma than a reply (using reddit-style [+] so it can be unhidden - or perhaps just omit it, and just use "parent" to see it). The key benefit is that comments that only make sense as a reaction (e.g. criticism) to another comment won't be as interesting, and (hopefully) will get downvoted. This might lead people to change their voting behaviour, to also vote the parent up if they think it does contribute after all (it's a problem that interesting questions don't get voted up, only the interesting answers, because then it's lost down the page... and interesting questions should be encouraged/rewarded, IMHO). A downside might be that people will try to "win" the thread. I really like this idea. tl;dr: don't display parent if a child scores higher (still accessible by "parent" link), so they rank higher. Like slashdot.
pygy_ 16y ago
I suggested a similar solution yesterday, but not by color coding the log(vote/views), but the log(votes/max votes in the thread). I posted it as the discussion was about to fall off the first page, so here is it again for your reading pleasure :-) --------------------- I have some suggestions that could help regulate the current issues. I'd change the way the karma is calculated in order to reward people who are consistently good rather than heavy posters. I suggest using the geometric mean of the post scores. As such, you wouldn't have to give a karma penalty to comment. A weak comment would reduce the karma automatically. I recommend the geometric mean since the scores are most likely Zipf-distributed. It could be calculated in the background as a running arithmetic mean of the logarithm of the scores, which shouldn't be too expensive. I'd display the karma not as an absolute number, but as color-coded rectangle. The color would be based on percentile of the score described above, or perhaps as a function of this percentile. The post count (or better, the post frequency) of the user could be represented as the length of the rectangle. You could then display this "badge" next to the user name in threads. Since it would be a continuous thing, it would not divide the comunity in have and have nots as the badge system introduced and retracted earlier. In order not to discourage people from posting, users below a certain threshold (say the 50th or 60th percentile) would all have the same value displayed. I'd also grant the various rights based on the basis of the percentile rather than based on hard numbers. Every time a new right is granted, users should get guidelines on how to use it, i.e. don't upvote based on agreement, don't downvote unless the post is offensive/shallow. Commenting in topics with little points could perhaps weight more than commenting in threads with a big score. This would encourage posting in less popular threads. Downvoting should cost karma. This could be implemented as a comment with a low score. You'd keep two counts: a post count and a downvote count. The karma would be calculated based on the sum of these numbers, but the post count would still be displayed. The value of a downvote would be calculated according to the number of post, otherwise it would cost less to people with lots of comments/votes. The karma of the voter may also be taken into account. The up and down votes should be weighted as a function of the karma of the voter. They could be implemented as a floating point number. Removing the number of votes per post prevents to detect a highly upvoted answer to a little upvoted comment. If the votes are weighted as described above, the vote count has little meaning anyway, but the post score could still be displayed in a meaningful way by using a color-coded system similar to the one described for the karma. I'd suggest two distinct scores log( comment score ) / log( highest score in the thread ) log( comment score ) / log( highest score ever at the time of thread submission ) Both of these could be calculated in javascript on the client. The real score could still be extracted with a browser extension, but I don't think it matters much. Making the first value relative to the max score at the time of voting would allow that value remains relevant as time passes, even if the community grows or shrinks. It would require to add a "max score" field to submissions, though.
snprbob86 16y ago
I've made a similar suggestion a few times: Leave the point system intact behind the scenes, as is. Collect the set of comments with a positive score. For threads with < N positive comments comments, don't show anything When there are >= N positive comments, show a percentile rank within that set. In the event of a tie, assign both comments the higher percentile (ie. "92% of the positive comments in this thread have equal or fewer points than this comment) My initial guess at N would be 10. Anything less and the percentiles are somewhat meaningless and not really needed, as it's easy to skim. The benefit of this approach: It's now a relative scale. Down voting no longer means "I don't like this" it means "the majority of other comments are better than this one". The net result will be less absolute value points for the best comments, as they will stall once they reach 90+% and they will receive far more down votes. However, there is now a community incentive to collectively sort the comment set, rather than award points.
- intended 16y ago
  Well some ask HN threads could easily have one or two good answers which shuts down the thread. This would break the <n positive comments rule. You could have <(n positive comments/total comments) as a work around perhaps. ---------------------------------- The thing is that all mathematical rules end up being terrible at specific edge cases which are better served by another set of rules. In my opinion, the best possible use of mathematical rules are ones which are simplistic and easy to follow. Leave the rest up to the quality of the community. (There is a different set of changes to cues and layout, which control social/behavioral action - such as removing the vote count, which I am not targeting.)
rubergly 16y ago
The problem with this that I foresee is that it seems rather impossible to predict actual views of a comment. I heavily doubt that most viewers of the comment page will read most of the comments. So this will create a positive feedback loop that just creates a large divide between comments that (somehow, probably mostly arbitrarily) start off more popular than others; comments with high "rates" will be pushed to the top, and will be seen most of the time someone views the page, but comments with low "rates" will be pushed to the bottom, meaning they won't be seen most time a page is viewed, and so the estimate of their rate will be biased downwards.
comex 16y ago
For what it's worth, a year or so ago Reddit tried a statistical solution to the same problem: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s... Not sure how effective it was.

by abeppu 16y ago

6ren 16y ago

The ordering of comments approximates this. It would be nice if there was some way to re-nest comments, based on scores, while retaining their logical relationship (as replies). e.g. It could be done with lines or labels, but it sounds too messy.

pygy_ 16y ago
The ordering is influenced by other factors, I guess either the post length or the average karma of the poster... I've had several comments both older and with less points than others at the same level, but that were still above the rest.
- Mz 16y ago
  The ordering is influenced by other factors, I guess either the post length or the average karma of the poster... My understanding is that average karma of the poster influences it. I used to be more impressed but my average karma score has been as high as about 6 but most of the time sits somewhere around 2, give or take. One good debate where multiple comments of mine got substantially upvoted raised it that high and then it steadily slid (ie regularly changed if you checked it) until it was back down to around 2. Maybe other people have high averages that are more stable than mine but this experience has changed my view of average karma and made me somewhat less impressed. From what I gather, it's a moving, weighted average. It isn't the average across all comments for all of your history. I can understand the logic of that -- if average for all comments were used and you have been here forever and your comment quality has gone up, a million old comments that were never voted up could prevent you from ever seeing any substantial improvements in that area. But, like all such things, it's a bit of a game in some sense.
  - 6ren 16y ago
    Yes, it's the average of recent comments, with the top and bottom discarded (pg said this somewhere or other). The averages of the top 100 (by abs karma) is interesting: http://news.ycombinator.com/leaders http://news.ycombinator.com/leaders (accessible via "lists", frontpage bottom). BTW: edw regularly has amazing averages - all the avgs are much higher than usual now, I suspect because of karma inflation due to not seeing scores (this happened last time pg experimented with it). And yes, it influences the ordering of ones comments. When I said "approximates", I really meant it. ;-)
6ren 16y ago
FOLLOWUP thinking more on re-nesting, slashdot detaches high-scoring comments from their logical nesting (and you have to follow parent links to find the post to which they were replying). I think slashdot knows much of this - I wonder if it would be helpful for cmdtaco and pg to chat (even, a voting-site conference)... or is it still too proprietary/competitive? Another way is too hide a comment if it has lower karma than a reply (using reddit-style [+] so it can be unhidden - or perhaps just omit it, and just use "parent" to see it). The key benefit is that comments that only make sense as a reaction (e.g. criticism) to another comment won't be as interesting, and (hopefully) will get downvoted. This might lead people to change their voting behaviour, to also vote the parent up if they think it does contribute after all (it's a problem that interesting questions don't get voted up, only the interesting answers, because then it's lost down the page... and interesting questions should be encouraged/rewarded, IMHO). A downside might be that people will try to "win" the thread. I really like this idea. tl;dr: don't display parent if a child scores higher (still accessible by "parent" link), so they rank higher. Like slashdot.

pygy_ 16y ago

I suggested a similar solution yesterday, but not by color coding the log(vote/views), but the log(votes/max votes in the thread). I posted it as the discussion was about to fall off the first page, so here is it again for your reading pleasure :-) --------------------- I have some suggestions that could help regulate the current issues. I'd change the way the karma is calculated in order to reward people who are consistently good rather than heavy posters. I suggest using the geometric mean of the post scores. As such, you wouldn't have to give a karma penalty to comment. A weak comment would reduce the karma automatically. I recommend the geometric mean since the scores are most likely Zipf-distributed. It could be calculated in the background as a running arithmetic mean of the logarithm of the scores, which shouldn't be too expensive. I'd display the karma not as an absolute number, but as color-coded rectangle. The color would be based on percentile of the score described above, or perhaps as a function of this percentile. The post count (or better, the post frequency) of the user could be represented as the length of the rectangle. You could then display this "badge" next to the user name in threads. Since it would be a continuous thing, it would not divide the comunity in have and have nots as the badge system introduced and retracted earlier. In order not to discourage people from posting, users below a certain threshold (say the 50th or 60th percentile) would all have the same value displayed. I'd also grant the various rights based on the basis of the percentile rather than based on hard numbers. Every time a new right is granted, users should get guidelines on how to use it, i.e. don't upvote based on agreement, don't downvote unless the post is offensive/shallow. Commenting in topics with little points could perhaps weight more than commenting in threads with a big score. This would encourage posting in less popular threads. Downvoting should cost karma. This could be implemented as a comment with a low score. You'd keep two counts: a post count and a downvote count. The karma would be calculated based on the sum of these numbers, but the post count would still be displayed. The value of a downvote would be calculated according to the number of post, otherwise it would cost less to people with lots of comments/votes. The karma of the voter may also be taken into account. The up and down votes should be weighted as a function of the karma of the voter. They could be implemented as a floating point number. Removing the number of votes per post prevents to detect a highly upvoted answer to a little upvoted comment. If the votes are weighted as described above, the vote count has little meaning anyway, but the post score could still be displayed in a meaningful way by using a color-coded system similar to the one described for the karma. I'd suggest two distinct scores log( comment score ) / log( highest score in the thread ) log( comment score ) / log( highest score ever at the time of thread submission ) Both of these could be calculated in javascript on the client. The real score could still be extracted with a browser extension, but I don't think it matters much. Making the first value relative to the max score at the time of voting would allow that value remains relevant as time passes, even if the community grows or shrinks. It would require to add a "max score" field to submissions, though.

snprbob86 16y ago

I've made a similar suggestion a few times: Leave the point system intact behind the scenes, as is. Collect the set of comments with a positive score. For threads with < N positive comments comments, don't show anything When there are >= N positive comments, show a percentile rank within that set. In the event of a tie, assign both comments the higher percentile (ie. "92% of the positive comments in this thread have equal or fewer points than this comment) My initial guess at N would be 10. Anything less and the percentiles are somewhat meaningless and not really needed, as it's easy to skim. The benefit of this approach: It's now a relative scale. Down voting no longer means "I don't like this" it means "the majority of other comments are better than this one". The net result will be less absolute value points for the best comments, as they will stall once they reach 90+% and they will receive far more down votes. However, there is now a community incentive to collectively sort the comment set, rather than award points.

intended 16y ago
Well some ask HN threads could easily have one or two good answers which shuts down the thread. This would break the <n positive comments rule. You could have <(n positive comments/total comments) as a work around perhaps. ---------------------------------- The thing is that all mathematical rules end up being terrible at specific edge cases which are better served by another set of rules. In my opinion, the best possible use of mathematical rules are ones which are simplistic and easy to follow. Leave the rest up to the quality of the community. (There is a different set of changes to cues and layout, which control social/behavioral action - such as removing the vote count, which I am not targeting.)

rubergly 16y ago

The problem with this that I foresee is that it seems rather impossible to predict actual views of a comment. I heavily doubt that most viewers of the comment page will read most of the comments. So this will create a positive feedback loop that just creates a large divide between comments that (somehow, probably mostly arbitrarily) start off more popular than others; comments with high "rates" will be pushed to the top, and will be seen most of the time someone views the page, but comments with low "rates" will be pushed to the bottom, meaning they won't be seen most time a page is viewed, and so the estimate of their rate will be biased downwards.

comex 16y ago

For what it's worth, a year or so ago Reddit tried a statistical solution to the same problem: http://blog.reddit.com/2009/10/reddits-new-comment-sorting-system.html http://blog.reddit.com/2009/10/reddits-new-comment-sorting-s... Not sure how effective it was.