9 ms·

Hacker News Ranking Algorithm

181 points by vignesh_warar 4y ago

cinntaile 4y ago
The actual algorithm seems to take into account post rate in some way. I think the assumption is that high early post rate implies controversial topic. If the ratio is too high it seems to get penalized pretty hard. Maybe downvotes and flaggings of posts is a factor too?
- jpalomaki 4y ago
  You are probably right. Without any analytics, based on gut feeling a good indicator for controversial topic is having more comments than upvotes.
- gerikson 4y ago
  It should be possible to derive the algorithm using an LLM given that timestamps, number of comments etc are available using the API.
toastal 4y ago
I'd like to see a negative multiplier for Substack and Medium posts for the pop-up spam.
- nigamanth 4y ago
  I'm pretty sure that already exists. Any post that doesn't link or links to sites like GitHub, Substack and Medium are downranked.
deleted 4y ago
[deleted]
tbonza 4y ago
Using page rank as suggested by OP, would provide weird incentives like frequent comments, or higher volume, could rank a person higher than if they had occasional comments further down the thread. If OP is interested in the most influential commenters, people who write frequently and have a posse, then PR would be a good way to do it. What would happen to a helpful comment from a throwaway account? If there's more of a ranking algorithm, identifying who is most likely to have actually read the article could be neat.
- cornholio 4y ago
  What would happen is that the old accounts would accumulate "CommentRank" and the new users would always be sorted to the bottom. This is good for search, since we are looking for "the" answer, the site, a limited number of best pages for a topic etc., but it's quite bad for discussion; a bad take from Animats would forever be above a good comment from a greenhorn.
  - vignesh_warar 4y ago
    I thought about it while writing this post. It would be useful to have a decay factor for PageRank. In simple words, if someone wants to maintain their PR score, they need to actively engage in the HN community.
    - dsr_ 4y ago
      You can put epicycle around epicycle, but there's a lot of computational efficiency in a score which can be calculated from an integer and a timestamp.
    - buran77 4y ago
      > if someone wants to maintain their PR score, they need to actively engage Wouldn't that encourage people to "actively engage" even when they have nothing to say just to keep their score? Goodhart's law comes into mind here. People who want to stay relevant on this site will take your measure as their target. There's no system that can't be gamed if you know the rules. But the more "rock solid" you try to make those rules, the more you embed intrinsic discrimination between participants. You put "Proof of Work" in the rules and you alienate newcomers while all the rest do "busywork". Your own comments and submissions would sink like a stone because you didn't engage enough, regardless of how interesting your points are.
      - vignesh_warar 4y ago
        I think this part of the post answers your question. > Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No, because Pagerank can be manipulated by people despite its reputation. In fact, Pagerank is being exploited for years. Moreover, there are financial incentives for companies to get on the first page in HN.
AraceliHarker 4y ago
I always read Hacker News through Feedly because the web page has too small letters and it’s hard to follow them, so I hardly had a chance to see the ranking.
jonathanstrange 4y ago
I don't understand this blog post. The authors says that HN is great for discussions. Then they go on to suggest a different ranking algorithm? Why? To answer the authors question: I would change absolutely nothing about HN. It's great for discussions as is. Edit: I suppose there is some insinuation that HN doesn't really use the original algorithm, and this is the author's attempt to reconstruct what's going on. Is that what I was missing?
- jasfi 4y ago
  If it works, don't fix it. You'll most likely break it.
  - jitix 4y ago
    I think product management across the industry needs to take this more seriously. In my personal experience, YouTube and Windows 11 are some recent examples of usability degradation due to unnecessary changes.
    - Dalewyn 4y ago
      I would argue they are taking it seriously, just not in the way most of us would like.
      - shjake 4y ago
        Well they need ways to justify their jobs and advance their careers. Which is perfectly understandable, UX is almost always is secondary to that (as it should be from the perspective of individuals).
  - jorvi 4y ago
    Is it working? Post ranking works pretty well, but as far as discussions go I often see insightful responses sent into low-contrast oblivion whilst a glib comment or quip gets accolades.
iamsanteri 4y ago
Is HN also built with RoR or is it something custom?
- ldjb 4y ago
  HN is built with Arc. http://arclanguage.org/ http://arclanguage.org/
nebulous1 4y ago
I don't like this idea. If hacker news comments were so numerous that the best comments by the "best" commenters always got lost in the maelstrom then there could be some sense to it. However, this has not been my experience of hacker news, and this shift towards a reputation based system seems unwarranted.
- krapp 4y ago
  Karma is already intended to be a reputation based system - it just doesn't work as intended, since karma doesn't really measure content quality over time.
pluralistic 4y ago
last month, "update to kagi search pricing" ranked 275 with metadata 116 points by exist 8 hours ago with 132 comments its cohorts ranked 270 to 282 (by my screenshoot) are of age 1day+ and with ten or more comments it's weird that post which is new, popular and engaging then sunk to the bottom 275 very fast within hours
- deleted 4y ago
  [deleted]
- pluralistic 4y ago
  it seems that someone replied with a link and it's gone here's the link: https://hnrankings.info/35078392/ https://hnrankings.info/35078392/ it shows "update to kagi search pricing" started and stayed in the front of hn for 1.5 hours it then left the front page and raced to rank 300 within the next 4 hours the disturbing thing is that this hn post with id 35078392 generates 116 points with 132 comments poof, just like that
slushh 4y ago
HN is not the place for new ranking algorithms since it is more than good enough. However, the other social networks could benefit from better sorting. Would it be possible to create a Youtube client that only shows the interesting comments?
- b112 4y ago
  Would it be possible to create a Youtube client that only shows the interesting comments? But what is interesting? To you and I, perhaps, it is an intellectually well thought out comment. To YouTube, perhaps, it is what makes people engage emotion.
prepend 4y ago
> I can't find the official page on why Paul Graham created HN, but to me, HN is the place to find the most interesting links and have a healthy discussion about various topics. In fact, discussions are my favorite part, except for the useless comments. Is there an official reason? I’ve wondered this as well and sort of assumed it was because hackers like having some community aspect and are novelty-seeking so YC created this due to the largesse of PG for the YC group and someone said “why not just make it public so applicants and future YCers can participate.” And job postings were added on to generate some revenue so it wasn’t a pure cost center. This headcanon reenforces my belief that the best community sites are accessories to something else and that you can’t make money from a community without diminishing its value (since the incentives for monetizing conversation are not the same and I’d guess 90% at odds with good conversation- mainly because bad things draw attention).
- krapp 4y ago
  AFAIK the "official" reason for creating HN was as a proof of concept for a viable web forum written in Arc, the lisp dialect PG was writing at the time. And bear in mind it was called Startup News before it became Hacker News.
- paulcole 4y ago
  Remember that there’s often the story and then the ret-conned “story.” From Wikipedia: > Initially called Startup News or occasionally News.YC. HN exists to serve the interests of YC. It’s a marketing/advertising tool. It’s great to believe that it’s a community, a place where interesting conversations happen, etc. (and those things may also be true) but remember what it is at its core.
walterbell 4y ago
https://hnrankings.info https://hnrankings.info graphs the history of every HN story, e.g. https://hnrankings.info/35510413/ https://hnrankings.info/35510413/
deleted 4y ago
[deleted]
manx 4y ago
Great to see more people thinking about this stuff! Our group, "Social Protocols" recently published a concrete idea on how to improve the ranking algorithm by introducing a new metric: https://github.com/social-protocols/news/blob/master/README.md https://github.com/social-protocols/news/blob/master/README.... The metric in action: https://news.social-protocols.org/ https://news.social-protocols.org/ HN Discussion: https://news.ycombinator.com/item?id=35183317 https://news.ycombinator.com/item?id=35183317
Scubabear68 4y ago
I think the article ranking is probably good enough. Where I have more trouble with is with comment-ranking within an article. The interface makes it hard to understand conversational flow, and of course that flow changes over time according to whatever the algorithm does. This makes it very hard to catch up later on an article’s comments and get a view of what has changed since my last view. Maybe there is a way to do this via the interface, but if so it’s not obvious.
- djbusby 4y ago
  Comments I haven't seen are a different color. But I read HN from a "custom" UI.
  - 8organicbits 4y ago
    Which?
    - djbusby 4y ago
      Some junk I put together.
- sillysaurusx 4y ago
  At one point you could email hn@ycombinator.com and ask to participate in the new beta, which among other things highlights new comments since you’ve last visited the page. I don’t know the status of it, but it seemed like an interesting idea.
  - 8organicbits 4y ago
    That should be pretty easy to do in a browser extension. Keep track of the last time the comment section was opened, highlight comments newer than that.
- thunky 4y ago
  > This makes it very hard to catch up later on an article’s comments My biggest complaint related to this is that all threads are dead within 24 hours, which kills a lot of interesting discussion. And commenters have no incentive to reply to dead threads. This discourages in depth back and forth conversation which I usually find is the most interesting.
didgetmaster 4y ago
I am often faced with a dilemma when trying to decide whether to vote on a particular post or comment. There are times when it is very informative, well written, and you can tell a lot of thought went into it; but I happen to disagree with the author's point of view. The same occurs when I agree with the author's sentiment, but the comment itself is poorly written and not well argued. There is only a single up/down vote. You can't upvote a thoughtful comment for its quality, while also downvoting its conclusion. Sometimes I have strong opinions about something; but I also try to educate myself with the other side of the issue by reading thoughtful articles and/or comments from people with different points of view. It would be nice if you could choose between two different 'modes' in the interface. One where the highest quality comments are near the top. The other where the most popular viewpoints are prioritized.
- andromaton 4y ago
  I upvoted your comment on both criteria. It would be very useful to have 2 dimensions. It's amazing nobody does afaik! @dang, as a Arc expert, let me assure it's only 4 lines of code, 1 field on db, 52 bytes of js, css, and html, and it won't cause performance, security, accessibility, privacy, or usability issues /s
- karaterobot 4y ago
  I think it's probably better to just upvote well-written or well-argued comments whether you agree with the author's point of view or not. Makes for a more interesting place where people occasionally change their minds. I was going to say that the HN guidelines advise doing this, but I cannot actually find anything about what an upvote 'means' on HN. Plenty about what makes a good submission, or a good comment, but nothing on voting. Maybe they just gave up on trying to do that! The above notwithstanding, I would like to see something like your proposal implemented, just to see how it works out empirically.
- lonk11 4y ago
  The dilemma you face stems from the ambiguous nature of the upvote button: 1. As an "agree" button - you upvote to get the perspective you share to be displayed more prominently to other people. It is a way for you to influence other readers. 2. As "it was useful to me and a good use of my time" button. You use it because you think other users will find it informative as well. In this case you are doing purely for the benefit of others. 3. As "it made me laugh or evoked some other emotions" button. All three could be viewed as different dimensions of content and one item/comment can have different combinations along these dimensions. It is complicated by the fact that content creators are actively optimizing their content along these dimensions to gain more visibility. When I come to HN, as a reader, I want to see mostly informative content - which is only dimension 2. I don't want to be influenced or persuaded. And while I enjoy a good laugh - that's not my goal coming to HN. This makes the other two content dimensions contribute to noise. Your suggestion to split out the upvote into separate labels like "agree", "informative" and "funny" [1] and to let users sort by the dimension of their choice may not solve the problem of separating "agree" from "informative" because there is no incentive for the voters to vote truthfully. People who upvote content they agree with want other people to see that content. If most users are sorting by informative then those voters will upvote content they agree with using the "informative" button. I think you need to change the system of incentives in order for voters to upvote only informative content and not 1 and 3. One way to do this is to make what you upvote have more consequences for the type of content you see in the future and less directly affect the ranking of that content for all users [2]. With a feedback loop like this the upvote button is no longer "what content do I want other people to see", but "what type of content do I want my future self to see". It makes you think before voting - "Was it worth my time?", "Is this kind of content aligned with my long term goals?" [1] - Slashdot has something like this: "funny", "interesting", "insigthful" https://en.wikipedia.org/wiki/Slashdot#Peer_moderation https://en.wikipedia.org/wiki/Slashdot#Peer_moderation but I don't think you can sort by it. Also, only a subset of users are given these moderator points. [2] - I'm building https://linklonk.com https://linklonk.com that works this way.
bitshiftfaced 4y ago
We may be getting to the point where we can directly predict the quality, interestingness, and originality of a comment before it has a chance to receive viewer feedback. This would be similar to how we can now rank historical chess players according to how chess bots evaluate their moves. A forum could then start comments off where it approximates their rank will end up, and then potentially let viewer feedback take over from there. What's more: if you can't stop bots from dominating your signal, then directly predicting things like quality may be one of the only good options left.
bluelightning2k 4y ago
Two obvious points: why not machine learning, and a more explicit mix of exploration and exploitation Meaning give more posts a chance, a certain number of views to determine the upvote rate. Conventional wisdom would apply spam filter, boost new users (e.g. first post), detect blatant manipulation (or more sophisticated manipulation using clustering).
- muyuu 4y ago
  With machine learning you have the same kind of problems with an added factor of unpredictability. ML to make the recommendations more dynamic per user means people are not seeing the same things and engagement tanks. ML to make the recommendations more global means a lot of people are going to see stuff they don't care about and they're going to have no idea why. It's usually this family of problems that ML adds an extra layer of opacity to the most. Then you add extra tabs to give the user more control, and now you've added a fresh complexity trade off. The sanest aggregators have simplistic recommendation algorithms and a bunch of janitorial work on top in the form of moderation/censorship. There must be something to this formula.
karmakaze 4y ago
Downvotes on posts is another way. It can require a minimum karma level and the value of downvotes can reduce with use to avoid abuse by individuals.
phailhaus 4y ago
> why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post Absolutely not, think about the incentives this creates! Imagine yourself to be an adversary, how would you game this system? Easy: spam low-effort highly agreeable comments across the site. The more the better, they're basically guaranteed to get some upvotes and boost your karma. Now, with your newfound power, you can shape the HN front page to your will. Instead of the front page being a signal of quality, it would be correlated to the posters' karma. And because the front page is a limited resource, you basically get a crappier version of Twitter where relatively low-karma accounts have practically no chance to get on the front page even with high-quality posts, and high-karma accounts will be able to roll their front-page posts into even more karma. This is pretty much one of the biggest flaws of major social media websites today: they concentrate power in a few users with massive influence.
lonk11 4y ago
1. The article suggests running PageRank on a graph of users as nodes and edges as - "has user A upvoted a comment of user B" as an edge: """ Since it's likely that one user may upvote multiple comments from the same user, we check whether a user has already upvoted a comment from that specific user before considering their upvote. In other words, we treat user profiles as nodes and upvotes for comments as edges. """ This is a very lossy conversion of the actual data: a. It does not distinguish if user C and D upvoted the same comment of user B or not. Maybe one comment was good and the other was bad. But when you convert it to the above graph you only get that C and D upvoted some comment of user B. b. It does not account for the number of comments user B left - 1000 or 5? This incentivizes spam because there is no upside to not posting. c. It ignores users that do not comment but upvote valuable comments themselves. Then the PageRank is used to weigh upvotes of users. This is backwards. The PageRank values should capture the value of each user's past upvotes in order to use it as a prediction of how valuable their future upvotes will be. But the suggested algorithm uses the value of the user's past comments as a weight of their future upvotes. To fix this I think the graph needs to be changed to a bipartite graph of users and comments as nodes and upvotes and flags as directed edges (when the author posts a comment - this should be represented as an implicit upvote of the comment). Then you can calculate how valuable each user's upvotes (and flags) are. 2. The "gameability" of PageRank stems from the fact that the random walk algorithm treats each users equally as a starting point. It means that you can create a ton of fake users and upvote the comments of a target user you want to artificially boost in upvote-power. Each time the random walk starts at one of those fake users the walk will end up in the target user - increasing their PageRank score. My proposal to solve the "gameability" problem is to start each walk from you - the user that views HackerNews. It means that your past upvotes become the starting step of the random walk and so the resulting PageRank will be personalized for you. Instead of a single PageRank reputation score (which captures how user A's contributions to HN have been to all users), there is a set of personalized scores that capture how useful other users have been to you. I'm building https://linklonk.com https://linklonk.com which uses this kind of algorithm to rank both links and comments. The details of the ranking algorithm are here: https://linklonk.com/item/3292763817660940288 https://linklonk.com/item/3292763817660940288
kens 4y ago
The article mentions a couple of formulas for the ranking algorithm. I'd like to point out my data-driven reverse-engineering of the HN ranking algorithm from 2013. The basic ranking formula uses votes and age with various exponents. But in my analysis, I found that "penalties" had a large effect on the ranking. Some penalties are applied automatically based on title words or the domain. Other penalties would be applied later. Too many comments would trigger the "controversy" penalty which would cause a sudden and drastic ranking drop. Link: https://www.righto.com/2013/11/how-hacker-news-ranking-really-works.html https://www.righto.com/2013/11/how-hacker-news-ranking-reall...
sideproject 4y ago
I recently launched a tool called HN+ https://hn.plus https://hn.plus It's a tool where you can create your own HackerNews clone. Obviously, while working on this, I had to emulate the HN ranking algorithm. But what I found is that the algorithm actually doesn't seem to work well for a "young" forums. I had to tweak it so that on the frontpage, the posts that are more relatively recent rises to the top over older posts of higher votes. I know these are considered in the original algorithm, but I had to tweak it. Which made me think that the algorithm should be considering the age of the community and also how active it is.
EFruit 4y ago
Perhaps I'm in the minority here, but I would like to see the entirety of HN's codebase posted as-is, no redactions, cleanup, etc. The entire site is officially moderated by ~2 people, and autonomously by the community. Whatever secret sauce they're using, that should be proof positive that it works. Keeping it a secret deprives countless communities of best-in-class tools and knowledge. Would it be detrimental to HN? In the short term, possibly (gaming the voting ring detector comes to mind), but it's hard to say what kind of impact there would be without knowing what goes on under the surface.

181 points by vignesh_warar 4y ago

The actual algorithm seems to take into account post rate in some way. I think the assumption is that high early post rate implies controversial topic. If the ratio is too high it seems to get penalized pretty hard. Maybe downvotes and flaggings of posts is a factor too?

jpalomaki 4y ago
You are probably right. Without any analytics, based on gut feeling a good indicator for controversial topic is having more comments than upvotes.
gerikson 4y ago
It should be possible to derive the algorithm using an LLM given that timestamps, number of comments etc are available using the API.

toastal 4y ago

I'd like to see a negative multiplier for Substack and Medium posts for the pop-up spam.

nigamanth 4y ago
I'm pretty sure that already exists. Any post that doesn't link or links to sites like GitHub, Substack and Medium are downranked.

deleted 4y ago

[deleted]

tbonza 4y ago

Using page rank as suggested by OP, would provide weird incentives like frequent comments, or higher volume, could rank a person higher than if they had occasional comments further down the thread. If OP is interested in the most influential commenters, people who write frequently and have a posse, then PR would be a good way to do it. What would happen to a helpful comment from a throwaway account? If there's more of a ranking algorithm, identifying who is most likely to have actually read the article could be neat.

cornholio 4y ago
What would happen is that the old accounts would accumulate "CommentRank" and the new users would always be sorted to the bottom. This is good for search, since we are looking for "the" answer, the site, a limited number of best pages for a topic etc., but it's quite bad for discussion; a bad take from Animats would forever be above a good comment from a greenhorn.
- vignesh_warar 4y ago
  I thought about it while writing this post. It would be useful to have a decay factor for PageRank. In simple words, if someone wants to maintain their PR score, they need to actively engage in the HN community.
  - dsr_ 4y ago
    You can put epicycle around epicycle, but there's a lot of computational efficiency in a score which can be calculated from an integer and a timestamp.
  - buran77 4y ago
    > if someone wants to maintain their PR score, they need to actively engage Wouldn't that encourage people to "actively engage" even when they have nothing to say just to keep their score? Goodhart's law comes into mind here. People who want to stay relevant on this site will take your measure as their target. There's no system that can't be gamed if you know the rules. But the more "rock solid" you try to make those rules, the more you embed intrinsic discrimination between participants. You put "Proof of Work" in the rules and you alienate newcomers while all the rest do "busywork". Your own comments and submissions would sink like a stone because you didn't engage enough, regardless of how interesting your points are.
    - vignesh_warar 4y ago
      I think this part of the post answers your question. > Will I ever publicly write about how HN ranks posts if I am Dang (HN moderator)? No, because Pagerank can be manipulated by people despite its reputation. In fact, Pagerank is being exploited for years. Moreover, there are financial incentives for companies to get on the first page in HN.

AraceliHarker 4y ago

I always read Hacker News through Feedly because the web page has too small letters and it’s hard to follow them, so I hardly had a chance to see the ranking.

jonathanstrange 4y ago

I don't understand this blog post. The authors says that HN is great for discussions. Then they go on to suggest a different ranking algorithm? Why? To answer the authors question: I would change absolutely nothing about HN. It's great for discussions as is. Edit: I suppose there is some insinuation that HN doesn't really use the original algorithm, and this is the author's attempt to reconstruct what's going on. Is that what I was missing?

jasfi 4y ago
If it works, don't fix it. You'll most likely break it.
- jitix 4y ago
  I think product management across the industry needs to take this more seriously. In my personal experience, YouTube and Windows 11 are some recent examples of usability degradation due to unnecessary changes.
  - Dalewyn 4y ago
    I would argue they are taking it seriously, just not in the way most of us would like.
    - shjake 4y ago
      Well they need ways to justify their jobs and advance their careers. Which is perfectly understandable, UX is almost always is secondary to that (as it should be from the perspective of individuals).
- jorvi 4y ago
  Is it working? Post ranking works pretty well, but as far as discussions go I often see insightful responses sent into low-contrast oblivion whilst a glib comment or quip gets accolades.

iamsanteri 4y ago

Is HN also built with RoR or is it something custom?

ldjb 4y ago
HN is built with Arc. http://arclanguage.org/ http://arclanguage.org/

nebulous1 4y ago

I don't like this idea. If hacker news comments were so numerous that the best comments by the "best" commenters always got lost in the maelstrom then there could be some sense to it. However, this has not been my experience of hacker news, and this shift towards a reputation based system seems unwarranted.

krapp 4y ago
Karma is already intended to be a reputation based system - it just doesn't work as intended, since karma doesn't really measure content quality over time.

pluralistic 4y ago

last month, "update to kagi search pricing" ranked 275 with metadata 116 points by exist 8 hours ago with 132 comments its cohorts ranked 270 to 282 (by my screenshoot) are of age 1day+ and with ten or more comments it's weird that post which is new, popular and engaging then sunk to the bottom 275 very fast within hours

deleted 4y ago
[deleted]
pluralistic 4y ago
it seems that someone replied with a link and it's gone here's the link: https://hnrankings.info/35078392/ https://hnrankings.info/35078392/ it shows "update to kagi search pricing" started and stayed in the front of hn for 1.5 hours it then left the front page and raced to rank 300 within the next 4 hours the disturbing thing is that this hn post with id 35078392 generates 116 points with 132 comments poof, just like that

slushh 4y ago

HN is not the place for new ranking algorithms since it is more than good enough. However, the other social networks could benefit from better sorting. Would it be possible to create a Youtube client that only shows the interesting comments?

b112 4y ago
Would it be possible to create a Youtube client that only shows the interesting comments? But what is interesting? To you and I, perhaps, it is an intellectually well thought out comment. To YouTube, perhaps, it is what makes people engage emotion.

prepend 4y ago

> I can't find the official page on why Paul Graham created HN, but to me, HN is the place to find the most interesting links and have a healthy discussion about various topics. In fact, discussions are my favorite part, except for the useless comments. Is there an official reason? I’ve wondered this as well and sort of assumed it was because hackers like having some community aspect and are novelty-seeking so YC created this due to the largesse of PG for the YC group and someone said “why not just make it public so applicants and future YCers can participate.” And job postings were added on to generate some revenue so it wasn’t a pure cost center. This headcanon reenforces my belief that the best community sites are accessories to something else and that you can’t make money from a community without diminishing its value (since the incentives for monetizing conversation are not the same and I’d guess 90% at odds with good conversation- mainly because bad things draw attention).

krapp 4y ago
AFAIK the "official" reason for creating HN was as a proof of concept for a viable web forum written in Arc, the lisp dialect PG was writing at the time. And bear in mind it was called Startup News before it became Hacker News.
paulcole 4y ago
Remember that there’s often the story and then the ret-conned “story.” From Wikipedia: > Initially called Startup News or occasionally News.YC. HN exists to serve the interests of YC. It’s a marketing/advertising tool. It’s great to believe that it’s a community, a place where interesting conversations happen, etc. (and those things may also be true) but remember what it is at its core.

walterbell 4y ago

https://hnrankings.info https://hnrankings.info graphs the history of every HN story, e.g. https://hnrankings.info/35510413/ https://hnrankings.info/35510413/

deleted 4y ago

[deleted]

manx 4y ago

Great to see more people thinking about this stuff! Our group, "Social Protocols" recently published a concrete idea on how to improve the ranking algorithm by introducing a new metric: https://github.com/social-protocols/news/blob/master/README.md https://github.com/social-protocols/news/blob/master/README.... The metric in action: https://news.social-protocols.org/ https://news.social-protocols.org/ HN Discussion: https://news.ycombinator.com/item?id=35183317 https://news.ycombinator.com/item?id=35183317

Scubabear68 4y ago

I think the article ranking is probably good enough. Where I have more trouble with is with comment-ranking within an article. The interface makes it hard to understand conversational flow, and of course that flow changes over time according to whatever the algorithm does. This makes it very hard to catch up later on an article’s comments and get a view of what has changed since my last view. Maybe there is a way to do this via the interface, but if so it’s not obvious.

djbusby 4y ago
Comments I haven't seen are a different color. But I read HN from a "custom" UI.
- 8organicbits 4y ago
  Which?
  - djbusby 4y ago
    Some junk I put together.
sillysaurusx 4y ago
At one point you could email hn@ycombinator.com and ask to participate in the new beta, which among other things highlights new comments since you’ve last visited the page. I don’t know the status of it, but it seemed like an interesting idea.
- 8organicbits 4y ago
  That should be pretty easy to do in a browser extension. Keep track of the last time the comment section was opened, highlight comments newer than that.
thunky 4y ago
> This makes it very hard to catch up later on an article’s comments My biggest complaint related to this is that all threads are dead within 24 hours, which kills a lot of interesting discussion. And commenters have no incentive to reply to dead threads. This discourages in depth back and forth conversation which I usually find is the most interesting.

didgetmaster 4y ago

I am often faced with a dilemma when trying to decide whether to vote on a particular post or comment. There are times when it is very informative, well written, and you can tell a lot of thought went into it; but I happen to disagree with the author's point of view. The same occurs when I agree with the author's sentiment, but the comment itself is poorly written and not well argued. There is only a single up/down vote. You can't upvote a thoughtful comment for its quality, while also downvoting its conclusion. Sometimes I have strong opinions about something; but I also try to educate myself with the other side of the issue by reading thoughtful articles and/or comments from people with different points of view. It would be nice if you could choose between two different 'modes' in the interface. One where the highest quality comments are near the top. The other where the most popular viewpoints are prioritized.

andromaton 4y ago
I upvoted your comment on both criteria. It would be very useful to have 2 dimensions. It's amazing nobody does afaik! @dang, as a Arc expert, let me assure it's only 4 lines of code, 1 field on db, 52 bytes of js, css, and html, and it won't cause performance, security, accessibility, privacy, or usability issues /s
karaterobot 4y ago
I think it's probably better to just upvote well-written or well-argued comments whether you agree with the author's point of view or not. Makes for a more interesting place where people occasionally change their minds. I was going to say that the HN guidelines advise doing this, but I cannot actually find anything about what an upvote 'means' on HN. Plenty about what makes a good submission, or a good comment, but nothing on voting. Maybe they just gave up on trying to do that! The above notwithstanding, I would like to see something like your proposal implemented, just to see how it works out empirically.
lonk11 4y ago
The dilemma you face stems from the ambiguous nature of the upvote button: 1. As an "agree" button - you upvote to get the perspective you share to be displayed more prominently to other people. It is a way for you to influence other readers. 2. As "it was useful to me and a good use of my time" button. You use it because you think other users will find it informative as well. In this case you are doing purely for the benefit of others. 3. As "it made me laugh or evoked some other emotions" button. All three could be viewed as different dimensions of content and one item/comment can have different combinations along these dimensions. It is complicated by the fact that content creators are actively optimizing their content along these dimensions to gain more visibility. When I come to HN, as a reader, I want to see mostly informative content - which is only dimension 2. I don't want to be influenced or persuaded. And while I enjoy a good laugh - that's not my goal coming to HN. This makes the other two content dimensions contribute to noise. Your suggestion to split out the upvote into separate labels like "agree", "informative" and "funny" [1] and to let users sort by the dimension of their choice may not solve the problem of separating "agree" from "informative" because there is no incentive for the voters to vote truthfully. People who upvote content they agree with want other people to see that content. If most users are sorting by informative then those voters will upvote content they agree with using the "informative" button. I think you need to change the system of incentives in order for voters to upvote only informative content and not 1 and 3. One way to do this is to make what you upvote have more consequences for the type of content you see in the future and less directly affect the ranking of that content for all users [2]. With a feedback loop like this the upvote button is no longer "what content do I want other people to see", but "what type of content do I want my future self to see". It makes you think before voting - "Was it worth my time?", "Is this kind of content aligned with my long term goals?" [1] - Slashdot has something like this: "funny", "interesting", "insigthful" https://en.wikipedia.org/wiki/Slashdot#Peer_moderation https://en.wikipedia.org/wiki/Slashdot#Peer_moderation but I don't think you can sort by it. Also, only a subset of users are given these moderator points. [2] - I'm building https://linklonk.com https://linklonk.com that works this way.

bitshiftfaced 4y ago

We may be getting to the point where we can directly predict the quality, interestingness, and originality of a comment before it has a chance to receive viewer feedback. This would be similar to how we can now rank historical chess players according to how chess bots evaluate their moves. A forum could then start comments off where it approximates their rank will end up, and then potentially let viewer feedback take over from there. What's more: if you can't stop bots from dominating your signal, then directly predicting things like quality may be one of the only good options left.

bluelightning2k 4y ago

Two obvious points: why not machine learning, and a more explicit mix of exploration and exploitation Meaning give more posts a chance, a certain number of views to determine the upvote rate. Conventional wisdom would apply spam filter, boost new users (e.g. first post), detect blatant manipulation (or more sophisticated manipulation using clustering).

muyuu 4y ago
With machine learning you have the same kind of problems with an added factor of unpredictability. ML to make the recommendations more dynamic per user means people are not seeing the same things and engagement tanks. ML to make the recommendations more global means a lot of people are going to see stuff they don't care about and they're going to have no idea why. It's usually this family of problems that ML adds an extra layer of opacity to the most. Then you add extra tabs to give the user more control, and now you've added a fresh complexity trade off. The sanest aggregators have simplistic recommendation algorithms and a bunch of janitorial work on top in the form of moderation/censorship. There must be something to this formula.

karmakaze 4y ago

Downvotes on posts is another way. It can require a minimum karma level and the value of downvotes can reduce with use to avoid abuse by individuals.

phailhaus 4y ago

> why don't we apply PageRank for every user based on the upvotes they receive for the comments they leave on any post Absolutely not, think about the incentives this creates! Imagine yourself to be an adversary, how would you game this system? Easy: spam low-effort highly agreeable comments across the site. The more the better, they're basically guaranteed to get some upvotes and boost your karma. Now, with your newfound power, you can shape the HN front page to your will. Instead of the front page being a signal of quality, it would be correlated to the posters' karma. And because the front page is a limited resource, you basically get a crappier version of Twitter where relatively low-karma accounts have practically no chance to get on the front page even with high-quality posts, and high-karma accounts will be able to roll their front-page posts into even more karma. This is pretty much one of the biggest flaws of major social media websites today: they concentrate power in a few users with massive influence.

lonk11 4y ago

1. The article suggests running PageRank on a graph of users as nodes and edges as - "has user A upvoted a comment of user B" as an edge: """ Since it's likely that one user may upvote multiple comments from the same user, we check whether a user has already upvoted a comment from that specific user before considering their upvote. In other words, we treat user profiles as nodes and upvotes for comments as edges. """ This is a very lossy conversion of the actual data: a. It does not distinguish if user C and D upvoted the same comment of user B or not. Maybe one comment was good and the other was bad. But when you convert it to the above graph you only get that C and D upvoted some comment of user B. b. It does not account for the number of comments user B left - 1000 or 5? This incentivizes spam because there is no upside to not posting. c. It ignores users that do not comment but upvote valuable comments themselves. Then the PageRank is used to weigh upvotes of users. This is backwards. The PageRank values should capture the value of each user's past upvotes in order to use it as a prediction of how valuable their future upvotes will be. But the suggested algorithm uses the value of the user's past comments as a weight of their future upvotes. To fix this I think the graph needs to be changed to a bipartite graph of users and comments as nodes and upvotes and flags as directed edges (when the author posts a comment - this should be represented as an implicit upvote of the comment). Then you can calculate how valuable each user's upvotes (and flags) are. 2. The "gameability" of PageRank stems from the fact that the random walk algorithm treats each users equally as a starting point. It means that you can create a ton of fake users and upvote the comments of a target user you want to artificially boost in upvote-power. Each time the random walk starts at one of those fake users the walk will end up in the target user - increasing their PageRank score. My proposal to solve the "gameability" problem is to start each walk from you - the user that views HackerNews. It means that your past upvotes become the starting step of the random walk and so the resulting PageRank will be personalized for you. Instead of a single PageRank reputation score (which captures how user A's contributions to HN have been to all users), there is a set of personalized scores that capture how useful other users have been to you. I'm building https://linklonk.com https://linklonk.com which uses this kind of algorithm to rank both links and comments. The details of the ranking algorithm are here: https://linklonk.com/item/3292763817660940288 https://linklonk.com/item/3292763817660940288

kens 4y ago

The article mentions a couple of formulas for the ranking algorithm. I'd like to point out my data-driven reverse-engineering of the HN ranking algorithm from 2013. The basic ranking formula uses votes and age with various exponents. But in my analysis, I found that "penalties" had a large effect on the ranking. Some penalties are applied automatically based on title words or the domain. Other penalties would be applied later. Too many comments would trigger the "controversy" penalty which would cause a sudden and drastic ranking drop. Link: https://www.righto.com/2013/11/how-hacker-news-ranking-really-works.html https://www.righto.com/2013/11/how-hacker-news-ranking-reall...

sideproject 4y ago

I recently launched a tool called HN+ https://hn.plus https://hn.plus It's a tool where you can create your own HackerNews clone. Obviously, while working on this, I had to emulate the HN ranking algorithm. But what I found is that the algorithm actually doesn't seem to work well for a "young" forums. I had to tweak it so that on the frontpage, the posts that are more relatively recent rises to the top over older posts of higher votes. I know these are considered in the original algorithm, but I had to tweak it. Which made me think that the algorithm should be considering the age of the community and also how active it is.

EFruit 4y ago

Perhaps I'm in the minority here, but I would like to see the entirety of HN's codebase posted as-is, no redactions, cleanup, etc. The entire site is officially moderated by ~2 people, and autonomously by the community. Whatever secret sauce they're using, that should be proof positive that it works. Keeping it a secret deprives countless communities of best-in-class tools and knowledge. Would it be detrimental to HN? In the short term, possibly (gaming the voting ring detector comes to mind), but it's hard to say what kind of impact there would be without knowing what goes on under the surface.