6 ms·

The Hacker News New Page Scroll of Death and Some Possible Solutions

58 points by curtis 11y ago

minimaxir 11y ago
It may be worthwhile to reference my "If your HN submission doesn't get any upvotes, don't worry about it" note (https://news.ycombinator.com/item?id=9864254 https://news.ycombinator.com/item?id=9864254) The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png http://i.imgur.com/MdUvMB9.png) A separate /new list won't make a difference because it won't improve the average content quality. Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest. Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though.
- mkagenius 11y ago
  > The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png http://i.imgur.com/MdUvMB9.png) Not entirely convinced that you can draw that conclusion from that plot. 10+ votes articles percentage might increase with the number of readers online, but so will the percentage of votes on other articles, hence 10+ voted articles != popular front page article. I think that 10+ votes may be a wrong way to look at it.
- realusername 11y ago
  > Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though. It's especially true on 'Show HN' and 'Ask HN' posts, since they appear for a longer period of time in the other sections.
- dang 11y ago
  > Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though. It might be because of the experimental story-rescuing program I've written about at https://news.ycombinator.com/item?id=9866140 https://news.ycombinator.com/item?id=9866140 and https://news.ycombinator.com/item?id=8790134 https://news.ycombinator.com/item?id=8790134.
- curtis 11y ago
  > Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest. This is certainly true. It is also the case that some things submitted to HN deserve upvotes and don't get them. A case in point, this very article: Try #1: https://news.ycombinator.com/item?id=10007658 ==> 4 points by curtis 17 hours ago | discuss Try #2: https://news.ycombinator.com/item?id=10011219 (this submission) ==> 42 points by curtis 2 hours ago | 18 comments The obvious conclusion is that the first submission didn't get upvotes just because it wasn't very good -- it simply didn't deserve them. And the second submission got upvotes because it was better. It was quality content. But of course in this example, they're the same article! Now let me be clear. I understand that it is the nature of this particular beast that there is simply going to be a lot of randomness. However, I believe we can make it noticeably less random with some fairly modest effort. And if we can make it less random, then I think that would be a good thing. Would the proposed "preview" page actually do that? I don't know. But I certainly think it's worth talking about.
lwhalen 11y ago
c'mon now, they aren't even about to fix the simple 'comment layout on mobile' problem. You really think they're going to go through the trouble of adding a whole new ranking system? :-)
- dang 11y ago
  > they aren't even about to fix the simple 'comment layout on mobile' problem Why do you say that? We've put a ton of work into that problem, which is not quite as simple as you'd think.
  - zajd 11y ago
    How about say... collapsable threads? It is the year 2015 after all.
  - stephengillie 11y ago
    Why do comments change font size when I tap their Reply link?
  - hobs 11y ago
    Hey dang! Just wanted to first say that I absolutely have enjoyed your work and time on HN, but from my experience as a user (important distinction here), the mobile interface hasn't changed much in the last few years. The only thing I see is a slightly bigger triangle button and a whole lot of tiny tiny text. I completely understand that my phone isn't the target audience (S3), and so I use an Android app to make up the difference, but when I saw your comment I reopened HN on my phone to see if had improved since I had last checked a year ago. Here's what I saw: http://i.imgur.com/CRonP5c.png http://i.imgur.com/CRonP5c.png I am NOT a UX expert (or even novitiate) but if you would, compare the size of the refresh button to the size of the upvote or comments buttons and notice the vast difference in area. I took the liberty of coloring in the comment link in red, and comparing it to the refresh button: http://i.imgur.com/GZEs7cN.png http://i.imgur.com/GZEs7cN.png and that it is less than 1/10th of the same clickable area. Now, I am 100% not saying you should be targeting this button size or that specifically, but that would be a reason I would make this type of comment. Hopefully that isnt too frustrating of a response, I assume there are a lot of factors going on that I am not aware of. Thanks.
jimminy 11y ago
Let's be realistic, even when the number of articles coming in is really high, you're talking about 100-150 posts for an hour. The HN algorithm has temporal-gravity to help pull down old content that wasn't hugely popular. Due to that algorithm if the content doesn't take off in the first hour, it likely won't take off. Those 2 points point to a far simpler solution. Just show most of the items from the last hour on the new page. So instead of 30 items, you show 100 items on the new page, which should accommodate most of the recent posts without requiring any clicks. (minimaxir's point is also very valid.)
- dang 11y ago
  > Just show most of the items from the last hour on the new page. Someone else suggested this recently, and we've heard it before, too. We're game to try it as an experiment. What's the best way to measure whether it works?
  - minimaxir 11y ago
    Determine if a nontrivial percentage of upvotes for a story are by users who are reading /new and the ranking of the story on /new at the time of the upvote is 30 or greater.
  - deleted 11y ago
    [deleted]
  - tikhonj 11y ago
    Well, the first step is identifying what the problem is. To me it's not obvious and a concrete statement of what we want to solve would help. That is, we know it's that "articles that should be on the front page get overlooked"—but votes are the main metric we have for what should be on the front page. How can we tell an article that fell off /new because it was bad from one that fell off because nobody saw it? One way of evaluating this would be to exercise editorial discretion, pick out articles you (or some set of people) believe are good and compare how they progress with different versions of the page. However, this is rather labor intensive and risks biasing the experiment based on who picks the articles out. I can't think of other approaches off-hand. Coming up with a concrete statement of the problem would probably help here.
x0054 11y ago
I feel that a lot of HN visitors do not at all check out the "new" page. How about another solution, where the front page shows a box on top with last 3 submissions from the new page, so that every new post get's at least some exposure, and get's a chance to get up voted once or twice. Each new post would be shown 2-5 times in the new box on the front page (depending on the volume of traffic to the front page vs. volume of new submissions). So, for instance, let's say there are 9 new posts in the "queue" and there are 3 concurrent visitors to the front page. Visitor A will see links 1-3 from the new queue, visitor B will see links 4-6, etc. After being shown at least 3 times to front page visitors the link get's dropped from the new queue if it did not gather a single up vote. However, if it does get an up vote, that also resets it's show counter, and it has 3 more chances to gather another up vote. The number of showings (lives) can be adjusted based on the ratio of new submissions to front page visitors. But I suspect that that ratio is more than high enough to accommodate 3-5 showings. I feel like this would be a relatively easy system to implement, and could result in better overall content. Thoughts?
- ectoplasm 11y ago
  I just use the comments page to find interesting discussions.
- jimminy 11y ago
  I personally think this is probably a good solution for visibility. If the issue is visibility of new submissions, random promotion of them to the front page seems like a valid solution. Though it might also mean that scores will be more variable for new submissions, and that may have an impact on front-page/2nd-page variability and bounciness, which doesn't seem like it would be friendly to casual users.
  - x0054 11y ago
    Hmm, well the way I was imagining it, every new post will get the same amount of promotion, but it's true that it is random in that you never know who you are going to show it to. Some people might pay attention to the New Posts box, some may not.
- 11y ago
curtis 11y ago
One of the possible advantages of the "preview" page from the proposal is that it might draw more eyeballs to new submissions than what they're receiving now. This is because the design interleaves new submissions with submissions that have received some degree of filtering. I really spend very little time checking the new page, and I suspect I'm not alone in that. I don't know if HN would be better if regular users spent more time looking at the new page, but I suspect that it would. It's also worth noting that HN could interleave new submissions into the front page to get them more visibility, forgoing a preview type page altogether.
kryptiskt 11y ago
Put in a random recently posted story on the front page, different for each visitor.
nsns 11y ago
I think many important entries get to the first page not because people upvote them on the "new" page, but because people keep on re-submitting them, where resubmition actually functions as an (unintended) upvote. That said, it seems that HN has grown so big, many "top stories" don't get enough time for discussion before completely disappearing from the front page. It might be time (shudder the thought) to introduce channels...
- curtis 11y ago
  I have long suspected that this is true for certain types of articles. If a new article appears on some really popular website like CNN or Slate (for example) and it is about some HN-worthy topic, there's a good chance that multiple HN users will encounter the article browsing the original source site and submit it from there. Now you've got a submission with 3 or 4 upvotes, none of them from the /new page.
  - jacquesm 11y ago
    Especially 'Ask HN' and 'Show HN' posts.
jedberg 11y ago
This is a problem we had to solve at reddit a long time ago. It's called the "rising" sort: https://github.com/reddit/reddit/blob/c6f959504466333c0d7d51c131240473aaf78b04/r2/r2/lib/rising.py https://github.com/reddit/reddit/blob/c6f959504466333c0d7d51...
joeyspn 11y ago
Allowing submissions only from users with a minimum karma (50, 100, 200, x) would be a better solution for preventing spam and flooding. Something similar to down-voting permissions that currently requires 500 points. It would be also much easier to implement.
- dang 11y ago
  A few problems with that: missing a good story costs HN far more than tolerating a bad one (kind of like startup investing that way); users with higher karma by no means reliably submit high-quality stories; by closing newcomers out, karma thresholds heighten the risk of the community becoming insular, something we definitely don't want.
ZenoArrow 11y ago
Perhaps the New feed could be tweaked, so that if a New submission gets an upvote it goes back to the top of the New feed. If the submission reaches the front page then its chronological position on the New feed could be restored.

58 points by curtis 11y ago

It may be worthwhile to reference my "If your HN submission doesn't get any upvotes, don't worry about it" note (https://news.ycombinator.com/item?id=9864254 https://news.ycombinator.com/item?id=9864254) The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png http://i.imgur.com/MdUvMB9.png) A separate /new list won't make a difference because it won't improve the average content quality. Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest. Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though.

mkagenius 11y ago
> The probability of hitting the front page is low at all times of day and all days of week. (see: http://i.imgur.com/MdUvMB9.png http://i.imgur.com/MdUvMB9.png) Not entirely convinced that you can draw that conclusion from that plot. 10+ votes articles percentage might increase with the number of readers online, but so will the percentage of votes on other articles, hence 10+ voted articles != popular front page article. I think that 10+ votes may be a wrong way to look at it.
realusername 11y ago
> Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though. It's especially true on 'Show HN' and 'Ask HN' posts, since they appear for a longer period of time in the other sections.
dang 11y ago
> Personally, I've had a number of submissions get upvotes after it's fallen off the first page of /new. I admit it's counterintuitive, though. It might be because of the experimental story-rescuing program I've written about at https://news.ycombinator.com/item?id=9866140 https://news.ycombinator.com/item?id=9866140 and https://news.ycombinator.com/item?id=8790134 https://news.ycombinator.com/item?id=8790134.
curtis 11y ago
> Not everything submitted to HN "deserves" to receive upvotes, to be perfectly honest. This is certainly true. It is also the case that some things submitted to HN deserve upvotes and don't get them. A case in point, this very article: Try #1: https://news.ycombinator.com/item?id=10007658 ==> 4 points by curtis 17 hours ago | discuss Try #2: https://news.ycombinator.com/item?id=10011219 (this submission) ==> 42 points by curtis 2 hours ago | 18 comments The obvious conclusion is that the first submission didn't get upvotes just because it wasn't very good -- it simply didn't deserve them. And the second submission got upvotes because it was better. It was quality content. But of course in this example, they're the same article! Now let me be clear. I understand that it is the nature of this particular beast that there is simply going to be a lot of randomness. However, I believe we can make it noticeably less random with some fairly modest effort. And if we can make it less random, then I think that would be a good thing. Would the proposed "preview" page actually do that? I don't know. But I certainly think it's worth talking about.

lwhalen 11y ago

c'mon now, they aren't even about to fix the simple 'comment layout on mobile' problem. You really think they're going to go through the trouble of adding a whole new ranking system? :-)

dang 11y ago
> they aren't even about to fix the simple 'comment layout on mobile' problem Why do you say that? We've put a ton of work into that problem, which is not quite as simple as you'd think.
- zajd 11y ago
  How about say... collapsable threads? It is the year 2015 after all.
- stephengillie 11y ago
  Why do comments change font size when I tap their Reply link?
- hobs 11y ago
  Hey dang! Just wanted to first say that I absolutely have enjoyed your work and time on HN, but from my experience as a user (important distinction here), the mobile interface hasn't changed much in the last few years. The only thing I see is a slightly bigger triangle button and a whole lot of tiny tiny text. I completely understand that my phone isn't the target audience (S3), and so I use an Android app to make up the difference, but when I saw your comment I reopened HN on my phone to see if had improved since I had last checked a year ago. Here's what I saw: http://i.imgur.com/CRonP5c.png http://i.imgur.com/CRonP5c.png I am NOT a UX expert (or even novitiate) but if you would, compare the size of the refresh button to the size of the upvote or comments buttons and notice the vast difference in area. I took the liberty of coloring in the comment link in red, and comparing it to the refresh button: http://i.imgur.com/GZEs7cN.png http://i.imgur.com/GZEs7cN.png and that it is less than 1/10th of the same clickable area. Now, I am 100% not saying you should be targeting this button size or that specifically, but that would be a reason I would make this type of comment. Hopefully that isnt too frustrating of a response, I assume there are a lot of factors going on that I am not aware of. Thanks.

jimminy 11y ago

Let's be realistic, even when the number of articles coming in is really high, you're talking about 100-150 posts for an hour. The HN algorithm has temporal-gravity to help pull down old content that wasn't hugely popular. Due to that algorithm if the content doesn't take off in the first hour, it likely won't take off. Those 2 points point to a far simpler solution. Just show most of the items from the last hour on the new page. So instead of 30 items, you show 100 items on the new page, which should accommodate most of the recent posts without requiring any clicks. (minimaxir's point is also very valid.)

dang 11y ago
> Just show most of the items from the last hour on the new page. Someone else suggested this recently, and we've heard it before, too. We're game to try it as an experiment. What's the best way to measure whether it works?
- minimaxir 11y ago
  Determine if a nontrivial percentage of upvotes for a story are by users who are reading /new and the ranking of the story on /new at the time of the upvote is 30 or greater.
- deleted 11y ago
  [deleted]
- tikhonj 11y ago
  Well, the first step is identifying what the problem is. To me it's not obvious and a concrete statement of what we want to solve would help. That is, we know it's that "articles that should be on the front page get overlooked"—but votes are the main metric we have for what should be on the front page. How can we tell an article that fell off /new because it was bad from one that fell off because nobody saw it? One way of evaluating this would be to exercise editorial discretion, pick out articles you (or some set of people) believe are good and compare how they progress with different versions of the page. However, this is rather labor intensive and risks biasing the experiment based on who picks the articles out. I can't think of other approaches off-hand. Coming up with a concrete statement of the problem would probably help here.

x0054 11y ago

I feel that a lot of HN visitors do not at all check out the "new" page. How about another solution, where the front page shows a box on top with last 3 submissions from the new page, so that every new post get's at least some exposure, and get's a chance to get up voted once or twice. Each new post would be shown 2-5 times in the new box on the front page (depending on the volume of traffic to the front page vs. volume of new submissions). So, for instance, let's say there are 9 new posts in the "queue" and there are 3 concurrent visitors to the front page. Visitor A will see links 1-3 from the new queue, visitor B will see links 4-6, etc. After being shown at least 3 times to front page visitors the link get's dropped from the new queue if it did not gather a single up vote. However, if it does get an up vote, that also resets it's show counter, and it has 3 more chances to gather another up vote. The number of showings (lives) can be adjusted based on the ratio of new submissions to front page visitors. But I suspect that that ratio is more than high enough to accommodate 3-5 showings. I feel like this would be a relatively easy system to implement, and could result in better overall content. Thoughts?

ectoplasm 11y ago
I just use the comments page to find interesting discussions.
jimminy 11y ago
I personally think this is probably a good solution for visibility. If the issue is visibility of new submissions, random promotion of them to the front page seems like a valid solution. Though it might also mean that scores will be more variable for new submissions, and that may have an impact on front-page/2nd-page variability and bounciness, which doesn't seem like it would be friendly to casual users.
- x0054 11y ago
  Hmm, well the way I was imagining it, every new post will get the same amount of promotion, but it's true that it is random in that you never know who you are going to show it to. Some people might pay attention to the New Posts box, some may not.
11y ago

curtis 11y ago

One of the possible advantages of the "preview" page from the proposal is that it might draw more eyeballs to new submissions than what they're receiving now. This is because the design interleaves new submissions with submissions that have received some degree of filtering. I really spend very little time checking the new page, and I suspect I'm not alone in that. I don't know if HN would be better if regular users spent more time looking at the new page, but I suspect that it would. It's also worth noting that HN could interleave new submissions into the front page to get them more visibility, forgoing a preview type page altogether.

kryptiskt 11y ago

Put in a random recently posted story on the front page, different for each visitor.

nsns 11y ago

I think many important entries get to the first page not because people upvote them on the "new" page, but because people keep on re-submitting them, where resubmition actually functions as an (unintended) upvote. That said, it seems that HN has grown so big, many "top stories" don't get enough time for discussion before completely disappearing from the front page. It might be time (shudder the thought) to introduce channels...

curtis 11y ago
I have long suspected that this is true for certain types of articles. If a new article appears on some really popular website like CNN or Slate (for example) and it is about some HN-worthy topic, there's a good chance that multiple HN users will encounter the article browsing the original source site and submit it from there. Now you've got a submission with 3 or 4 upvotes, none of them from the /new page.
- jacquesm 11y ago
  Especially 'Ask HN' and 'Show HN' posts.

jedberg 11y ago

This is a problem we had to solve at reddit a long time ago. It's called the "rising" sort: https://github.com/reddit/reddit/blob/c6f959504466333c0d7d51c131240473aaf78b04/r2/r2/lib/rising.py https://github.com/reddit/reddit/blob/c6f959504466333c0d7d51...

joeyspn 11y ago

Allowing submissions only from users with a minimum karma (50, 100, 200, x) would be a better solution for preventing spam and flooding. Something similar to down-voting permissions that currently requires 500 points. It would be also much easier to implement.

dang 11y ago
A few problems with that: missing a good story costs HN far more than tolerating a bad one (kind of like startup investing that way); users with higher karma by no means reliably submit high-quality stories; by closing newcomers out, karma thresholds heighten the risk of the community becoming insular, something we definitely don't want.

ZenoArrow 11y ago

Perhaps the New feed could be tweaked, so that if a New submission gets an upvote it goes back to the top of the New feed. If the submission reaches the front page then its chronological position on the New feed could be restored.