4 ms·

We explained some of the problems of the ranking algorithm in an article and backed it up with data: https://felx.me/2021/08/29/improving-the-hacker-news-rankin

by manx 4y ago

We explained some of the problems of the ranking algorithm in an article and backed it up with data: https://felx.me/2021/08/29/improving-the-hacker-news-ranking-algorithm.html https://felx.me/2021/08/29/improving-the-hacker-news-ranking... From that point on, we continued to research this problem and are working on an improved HN frontpage website right now, which takes live data from HN and factors out the feedback loops. It should be ready to use soon. If you want to get early access or get notified, please write me an email.

elevaet 4y ago
Great work, and well-presented findings. I'm really excited to see this in the wild.
- onos 4y ago
  Agreed. One aspect I’d like to understand better is how the dynamics will change. Eg today a highly popular post can stay up for a whole day. I get the sense that the new algo may flush things through more rapidly: Consider two posts, with upvote rates p1 and p2 with p1 =2p2. That is the first is twice as good as the second. Now if the second post is half as old as the first it would actually have about 2x the score of the first, despite being half as good. This suggests more quality posts will get a chance to sit on the front page but each may stay less time — if so this would exacerbate the OP’s concern.
dang 4y ago
It would certainly be interesting to see an improved HN front page (assuming people could agree on what counted as improved). But how do you measure views? Your article talks about that and it seems to be part of your algorithm, but we don't publish that data, and in the case of article clicks, we don't even collect it.
- pvg 4y ago
  I asked pretty much the same question at the time because it seemed like 'backed up with data' is a bit of a stretch. The answer then was 'it's actually not backed up by data'. https://news.ycombinator.com/item?id=28405819 https://news.ycombinator.com/item?id=28405819
  - manx 4y ago
    Yes, you are right. Only the existence of the problem is "backed up with data". The proposed solution is just an idea so far. I hope our implementation brings new insights.
- manx 4y ago
  Yes, the main feedback from HN back then was that we don't have any click-through or impression data. That is true, and we don't want to introduce any user tracking to make our idea possible. That was a very hard nut to crack, but after a year of research we finally think we have a practical solution: Instead of views, we estimate the attention a story has received. The main idea is to track how much time a story spent on different ranks. We created a causal statistical model to approximate the amount of attention a story gets on specific ranks. These are used to calculate a total accumulative attention number for every story. This attention value is then used instead of the views mentioned in the article. That's the main idea. A more detailed writeup will be published, together with the implementation of that frontpage.

by manx 4y ago

elevaet 4y ago

Great work, and well-presented findings. I'm really excited to see this in the wild.

onos 4y ago
Agreed. One aspect I’d like to understand better is how the dynamics will change. Eg today a highly popular post can stay up for a whole day. I get the sense that the new algo may flush things through more rapidly: Consider two posts, with upvote rates p1 and p2 with p1 =2p2. That is the first is twice as good as the second. Now if the second post is half as old as the first it would actually have about 2x the score of the first, despite being half as good. This suggests more quality posts will get a chance to sit on the front page but each may stay less time — if so this would exacerbate the OP’s concern.

dang 4y ago

It would certainly be interesting to see an improved HN front page (assuming people could agree on what counted as improved). But how do you measure views? Your article talks about that and it seems to be part of your algorithm, but we don't publish that data, and in the case of article clicks, we don't even collect it.

pvg 4y ago
I asked pretty much the same question at the time because it seemed like 'backed up with data' is a bit of a stretch. The answer then was 'it's actually not backed up by data'. https://news.ycombinator.com/item?id=28405819 https://news.ycombinator.com/item?id=28405819
- manx 4y ago
  Yes, you are right. Only the existence of the problem is "backed up with data". The proposed solution is just an idea so far. I hope our implementation brings new insights.
manx 4y ago
Yes, the main feedback from HN back then was that we don't have any click-through or impression data. That is true, and we don't want to introduce any user tracking to make our idea possible. That was a very hard nut to crack, but after a year of research we finally think we have a practical solution: Instead of views, we estimate the attention a story has received. The main idea is to track how much time a story spent on different ranks. We created a causal statistical model to approximate the amount of attention a story gets on specific ranks. These are used to calculate a total accumulative attention number for every story. This attention value is then used instead of the views mentioned in the article. That's the main idea. A more detailed writeup will be published, together with the implementation of that frontpage.