6 ms·

Makes sense, due to a self-reinforcing dynamic: early comments that are good enough to gather upvotes will stay near the top of the page, gathering more upvotes

by cs702 10y ago

Makes sense, due to a self-reinforcing dynamic: early comments that are good enough to gather upvotes will stay near the top of the page, gathering more upvotes. I suspect HN is subject to the same "first comment effect," often to the detriment of later comments that might be more deserving of the top spot. Does anyone here have ideas for fixing this?

no_protocol 10y ago
I am guessing that HN will have a lower percentage than reddit. New comments get a bit of time at the top and have a chance to gain votes if they are worthy.
- ethanbond 10y ago
  That's how Reddit comments work too...
  - no_protocol 10y ago
    Sorry, I am not familiar with Reddit, I just took the author's word for it: Anecdotally, that doesn’t be the case with Reddit’s modern algorithm; comments made late in the thread appear at the bottom, where they likely will not receive any upvotes In my experience with Hacker News, late comments are given some space right at the very top, even if they are a day late. I thought this was a distinction worth commenting on, but maybe I misunderstood the quoted passage.
    - ethanbond 10y ago
      So this is kind of bizarre wording on the author's part. He describes that in fact Reddit does use time decay for exactly the reason you mentioned literally two sentences before what you quoted. It seems weird to state something as a fact then dispute it based on an anecdote without addressing the obvious tension between those things... > Reddit’s ranking algorithm attempts to rectify this by determining comment ranking using both time and community voting; comments in a thread, by default, are ordered based on the points score (upvotes - downvotes) the comment receives, subject to a rank decay based on the age of the comment. > In theory, this system should allow comments that posted later in the thread’s lifetime to rank much higher temporarily, then Redditors can vote on the new comment; if the new comment is good, it can now rise to the top and therefore the content which would otherwise be buried is now surfaced. And then it continues on to the section you quoted. I'm not sure what to believe!
    - minimaxir 10y ago
      I commented about the HN behavior in another thread. (I did not comment about the HN behavior in the article because I have no statistical evidence of HN behavior.)
malnourish 10y ago
Some weighted random sorting of top level comments (at each stage in a tree) or sorting by "hot/rising" would go a long way, I think.
david-given 10y ago
Well, the trend is so smooth that it would be very simple to apply a modifier to comments to counter it. Effectively, votes would slowly expire. In order to remain relevant, a comment would have to be constantly upvoted. I actually think this would be an interesting experiment to try. Who's hacker friendly and has a well-used comment system that we know?
- singham 10y ago
  > votes would expire Basically, ant-algorithm. To stay relevant, you have constantly gather new votes. But it has one problem. Comments with higher votes already have momentum, hence they will be shown to new users most of the time, which gives it more chance to gather new votes, thus continuing the momentum.
- mrec 10y ago
  Which is roughly the algo already used for Reddit's default "Hot" view of submissions, isn't it?
  - david-given 10y ago
    Beats me, I don't use Reddit. Does it work? Does it seem to do a decent job at selecting submissions?
    - mrec 10y ago
      Yeah, seems good to me, in that it strikes a reasonable balance between keeping good stuff around long enough for you to see it, and keeping it around so long that you've read everything and get bored. Thinking about this some more, though, I don't think I'd want to see the exact same algo on comments, though; comments don't have the same infinite-firehose problem, and you don't want to encourage spammy necrocommenting on old-but-still-popular threads just to get "new comment" karma. Maybe something more like making vote fuzzing (which Reddit already does to some extent) inversely proportional to age, and actually sorting on the fuzzed value rather than the real one.
- 10y ago
mason240 10y ago
Randomizing the top three comments. Have three categories: New, Best (up:down ratio is high), Controversial (up:down ratio of 1:1). Pick one from each category and then randomly make them the top three. For larger threads you can salt them into the top 10 comments. New comments that are truly good have a chance of being picked up to Best or Hot, and Controversial comments spur discussion and moderate the self-reinforcing circle-jerk that threads descend into.
- jessriedel 10y ago
  Yep, and you can apply this recursively for nested comments, do that the first reply doesn't dominate either.
sotojuan 10y ago
This is just a "con" of nested threads. A flat, traditional forum/message board would fix this but introduces its own set of negatives.
joezydeco 10y ago
I've always wondered why the karma of the upvoter isn't weighed into the equation. If someone like tptacek (with 259K karma) votes my comment up, I think that's a lot more relevant than someone with 300 karma.
- vit05 10y ago
  I remember something similar to this on digg. The results wasn't great. "Power users"
- pbhjpbhj 10y ago
  I'd like to have the comments delivered with metadata and have a small JavaScript to sort them. A few defaults (hot, new, etc.) and chance to try popular sort algos. Then chance to weight by popularity of the user, upvotes/downvotes, or location, or whatever is possible with the metadata.
- DanBC 10y ago
  I have 38,000 karma. I think it's a terrible idea for my karma value to make my upvotes weigh more.
  - CPLX 10y ago
    I have 6,000 karma. I agree with one sixth of your comment.
- sparky_z 10y ago
  In the aggregate, karma is a measure of volume, not of quality. Members with lots of karma are often the ones who spend all of their free time writing comments and submitting links, focusing on quick, easy wins. There's no reason to think they're the ones with the most discerning taste. Using average karma per comment would be an improvement, but it still favors people who are on the site constantly (so they can be the first to make the obvious comment that everybody upvotes) and frequent the most popular (and lowest effort) subreddits. Another problem: the last thing reddit needs is more incentive for users to improve their personal karma score. Karma is useful for readers, but that benefit is balanced against the incentives it provides for writers to optimize their karma score above all other considerations. I think HN achieves this balance better than reddit does by deemphasizing personal karma scores in various ways, and this is reflected in the user experience. It's bad enough when karma's just a driver of personal gamification. If it actually provided users with social clout and ability to frame the conversation, well, then it would be something with real world value. You wouldn't need click farms to manufacture viral content, just a few well-placed accounts with high scores. High-karma upvotes as a service. As a side-gig for a careful power user, it would be nigh-undetectable. Think of the difference between HN and reddit, then multiply it 10-fold. I think that's what you would end up with.
greggman 10y ago
I don't know if this would change the ranking but I've always wondered if it would be good if threads were ranked by the waited sum of all replies to it. In otherwords the original comment might not have a high vote but if a reply did, even a deep one, the thread might appear at the top. Or does it already work that way?
pjc50 10y ago
The effect is very strong on HN. If you want to be read it's much better to piggyback on it than start a new one.
- rm_-rf_slash 10y ago
  Even more so when it's a major topic with hundreds of upvotes and comments. Often the top comment of big big big stories on HN tends to be responsible for around 25% of total thread comments, if not more. Niche stories with less attention tend to have a more rounded discussion with a larger number of parent comments relative to replies.
  - greenmountin 10y ago
    How do you beat the "tyranny of the reply"? Automatic karma deductions for 3rd tier replies? Detachment + quoting after the 2nd tier? Positioning based on the cumulative karma of the whole tree? Automatic collapse of higher tiers (a la reddit...) Super-powered collapse buttons, for when people get tired mid-tree I wish something were done, but I bet it will be an addon, not HN, first.
    - stonith 10y ago
      Maybe some per-user jitter in the ordering so that different people are seeing very slightly different things at the upper end, giving those posts the chance to be seen?
    - rm_-rf_slash 10y ago
      I like the comment collapse idea but i think it would be worth having a sorter so you can see the top comment thread but also newest first and most up-and-coming threads.
    - pjc50 10y ago
      Take the natural logarithm of the comment karma, and then use as the weight in a weighted shuffle algorithm? Higher-voted comments will then be more likely to move up, but not guaranteed. (You might want to consider putting a thumb on the scale to distinguish between comments which are low voted because they're uninteresting, and comments which contain flame-bait words etc.)
deleted 10y ago
[deleted]

by cs702 10y ago

no_protocol 10y ago

I am guessing that HN will have a lower percentage than reddit. New comments get a bit of time at the top and have a chance to gain votes if they are worthy.

ethanbond 10y ago
That's how Reddit comments work too...
- no_protocol 10y ago
  Sorry, I am not familiar with Reddit, I just took the author's word for it: Anecdotally, that doesn’t be the case with Reddit’s modern algorithm; comments made late in the thread appear at the bottom, where they likely will not receive any upvotes In my experience with Hacker News, late comments are given some space right at the very top, even if they are a day late. I thought this was a distinction worth commenting on, but maybe I misunderstood the quoted passage.
  - ethanbond 10y ago
    So this is kind of bizarre wording on the author's part. He describes that in fact Reddit does use time decay for exactly the reason you mentioned literally two sentences before what you quoted. It seems weird to state something as a fact then dispute it based on an anecdote without addressing the obvious tension between those things... > Reddit’s ranking algorithm attempts to rectify this by determining comment ranking using both time and community voting; comments in a thread, by default, are ordered based on the points score (upvotes - downvotes) the comment receives, subject to a rank decay based on the age of the comment. > In theory, this system should allow comments that posted later in the thread’s lifetime to rank much higher temporarily, then Redditors can vote on the new comment; if the new comment is good, it can now rise to the top and therefore the content which would otherwise be buried is now surfaced. And then it continues on to the section you quoted. I'm not sure what to believe!
  - minimaxir 10y ago
    I commented about the HN behavior in another thread. (I did not comment about the HN behavior in the article because I have no statistical evidence of HN behavior.)

malnourish 10y ago

Some weighted random sorting of top level comments (at each stage in a tree) or sorting by "hot/rising" would go a long way, I think.

david-given 10y ago

Well, the trend is so smooth that it would be very simple to apply a modifier to comments to counter it. Effectively, votes would slowly expire. In order to remain relevant, a comment would have to be constantly upvoted. I actually think this would be an interesting experiment to try. Who's hacker friendly and has a well-used comment system that we know?

singham 10y ago
> votes would expire Basically, ant-algorithm. To stay relevant, you have constantly gather new votes. But it has one problem. Comments with higher votes already have momentum, hence they will be shown to new users most of the time, which gives it more chance to gather new votes, thus continuing the momentum.
mrec 10y ago
Which is roughly the algo already used for Reddit's default "Hot" view of submissions, isn't it?
- david-given 10y ago
  Beats me, I don't use Reddit. Does it work? Does it seem to do a decent job at selecting submissions?
  - mrec 10y ago
    Yeah, seems good to me, in that it strikes a reasonable balance between keeping good stuff around long enough for you to see it, and keeping it around so long that you've read everything and get bored. Thinking about this some more, though, I don't think I'd want to see the exact same algo on comments, though; comments don't have the same infinite-firehose problem, and you don't want to encourage spammy necrocommenting on old-but-still-popular threads just to get "new comment" karma. Maybe something more like making vote fuzzing (which Reddit already does to some extent) inversely proportional to age, and actually sorting on the fuzzed value rather than the real one.
10y ago

mason240 10y ago

Randomizing the top three comments. Have three categories: New, Best (up:down ratio is high), Controversial (up:down ratio of 1:1). Pick one from each category and then randomly make them the top three. For larger threads you can salt them into the top 10 comments. New comments that are truly good have a chance of being picked up to Best or Hot, and Controversial comments spur discussion and moderate the self-reinforcing circle-jerk that threads descend into.

jessriedel 10y ago
Yep, and you can apply this recursively for nested comments, do that the first reply doesn't dominate either.

sotojuan 10y ago

This is just a "con" of nested threads. A flat, traditional forum/message board would fix this but introduces its own set of negatives.

joezydeco 10y ago

I've always wondered why the karma of the upvoter isn't weighed into the equation. If someone like tptacek (with 259K karma) votes my comment up, I think that's a lot more relevant than someone with 300 karma.

vit05 10y ago
I remember something similar to this on digg. The results wasn't great. "Power users"
pbhjpbhj 10y ago
I'd like to have the comments delivered with metadata and have a small JavaScript to sort them. A few defaults (hot, new, etc.) and chance to try popular sort algos. Then chance to weight by popularity of the user, upvotes/downvotes, or location, or whatever is possible with the metadata.
DanBC 10y ago
I have 38,000 karma. I think it's a terrible idea for my karma value to make my upvotes weigh more.
- CPLX 10y ago
  I have 6,000 karma. I agree with one sixth of your comment.
sparky_z 10y ago
In the aggregate, karma is a measure of volume, not of quality. Members with lots of karma are often the ones who spend all of their free time writing comments and submitting links, focusing on quick, easy wins. There's no reason to think they're the ones with the most discerning taste. Using average karma per comment would be an improvement, but it still favors people who are on the site constantly (so they can be the first to make the obvious comment that everybody upvotes) and frequent the most popular (and lowest effort) subreddits. Another problem: the last thing reddit needs is more incentive for users to improve their personal karma score. Karma is useful for readers, but that benefit is balanced against the incentives it provides for writers to optimize their karma score above all other considerations. I think HN achieves this balance better than reddit does by deemphasizing personal karma scores in various ways, and this is reflected in the user experience. It's bad enough when karma's just a driver of personal gamification. If it actually provided users with social clout and ability to frame the conversation, well, then it would be something with real world value. You wouldn't need click farms to manufacture viral content, just a few well-placed accounts with high scores. High-karma upvotes as a service. As a side-gig for a careful power user, it would be nigh-undetectable. Think of the difference between HN and reddit, then multiply it 10-fold. I think that's what you would end up with.

greggman 10y ago

I don't know if this would change the ranking but I've always wondered if it would be good if threads were ranked by the waited sum of all replies to it. In otherwords the original comment might not have a high vote but if a reply did, even a deep one, the thread might appear at the top. Or does it already work that way?

pjc50 10y ago

The effect is very strong on HN. If you want to be read it's much better to piggyback on it than start a new one.

rm_-rf_slash 10y ago
Even more so when it's a major topic with hundreds of upvotes and comments. Often the top comment of big big big stories on HN tends to be responsible for around 25% of total thread comments, if not more. Niche stories with less attention tend to have a more rounded discussion with a larger number of parent comments relative to replies.
- greenmountin 10y ago
  How do you beat the "tyranny of the reply"? Automatic karma deductions for 3rd tier replies? Detachment + quoting after the 2nd tier? Positioning based on the cumulative karma of the whole tree? Automatic collapse of higher tiers (a la reddit...) Super-powered collapse buttons, for when people get tired mid-tree I wish something were done, but I bet it will be an addon, not HN, first.
  - stonith 10y ago
    Maybe some per-user jitter in the ordering so that different people are seeing very slightly different things at the upper end, giving those posts the chance to be seen?
  - rm_-rf_slash 10y ago
    I like the comment collapse idea but i think it would be worth having a sorter so you can see the top comment thread but also newest first and most up-and-coming threads.
  - pjc50 10y ago
    Take the natural logarithm of the comment karma, and then use as the weight in a weighted shuffle algorithm? Higher-voted comments will then be more likely to move up, but not guaranteed. (You might want to consider putting a thumb on the scale to distinguish between comments which are low voted because they're uninteresting, and comments which contain flame-bait words etc.)

deleted 10y ago

[deleted]