17 ms·
How Hacker News ranking really works: scoring, controversy, and penalties
- minimaxir 13y agoNot sure why the HN submission didn't link to the original post: http://www.righto.com/2013/11/how-hacker-news-ranking-really-works.html http://www.righto.com/2013/11/how-hacker-news-ranking-really... Discussion here: https://news.ycombinator.com/item?id=6755071 https://news.ycombinator.com/item?id=6755071
- nwh 13y agoIt's mentioned in the post that it was weighed down so much that it dropped off the front page almost instantly.
- micampe 13y agoBecause duplicates are useful. I hadn't seen the original post, I've seen this one. Disallowing reposts is one of the HN policies I don't agree with.
- lnanek2 13y agoAgreed. I'm glad I caught this on the repost.
- alexkus 13y ago> Not sure why the HN submission didn't link to the original post It's not a duplicate (as far as HN is concerned), the URLs are slightly different.
- flexie 13y agoThe avoidance of controversial topics when talking together is one of those things we Europeans are typically not so good at. I know from many Europeans who like me lived in the US for a while that they had to learn the art of talking without touching controversial subjects. At first it seemed superficial but then I realised that it makes discussions that are not controversial but nevertheless important possible and I came to appreciate it every now and then. Anyways, it would be nice if we in the settings could apply our own penalizing to subjects that we don't care about or that we find controversial instead of having others decide for us. But that would mean that submissions ranked differently for different users, of couse...
- alan_cx 13y agoDoesn't that sound like a very unhealthy national self censorship?
- davidw 13y ago> Doesn't that sound like a very unhealthy national self censorship? Fortunately, Hacker News is not a nation, and can censor all it likes to keep out all the crap.
- alan_cx 13y agoThe post I was replying to was about US citizens, not HN. BTW, who are you to say what is crap and what is not? I mean, I may consider your reply to be crap.
- davidw 13y ago> who are you to say what is crap and what is not Someone who has seen lots of good forums wrecked by political bullshit. And to be perfectly clear, I do not mean "politics that I happen to disagree with", but politics, in general.
- sejje 13y agoPolitics seems to elicit a substantial amount of unwanted behavior from people who otherwise would behave normally. I like forums that remove it as much as possible.
- moocowduckquack 13y agoI remember when I first heard of the concept that you should not discuss sex, politics or religion in polite company. My thought then, which I still hold now, is that polite company sounds pretty damn boring.
- ceol 13y ago
- brudgers 13y agoThis article is worse than the original, and the original article was crap because it never addressed flagging. I have on occasion flagged articles on a particular topic...sometimes I just get frustrated with recycled topics. It also doesn't address the possibility that penalties are applied when HN's heuristics suspect vote rings. And given that the hackaday article is blossoms, don't be surprised to see it fall.
- mgunes 13y agoHow HN page rankings really work: you vote stuff up, and then the flag mob and hidden moderators axe right off the front page whatever irritates the pro-capitalist internet-libertarian techno-optimist idelogical sensibilities of the white male Californian HN hive mind even in the subtlest of ways.
- JonnieCache 13y agoIsn't that the point of this site? That's mainly why I come here, to peer into the collective unconcious of the californian ivy-league portfolio-owning hivemind. It's like a zoo but with tesla owners instead of chimps.
- mgunes 13y agoI don't imply an ideal of neutrality; no online community is ideologically neutral, and neutrality isn't a merit to strive for. It's just that most participants seem to have internalized a supposedly meritocratic "whatever is interesting to good hackers floats to the top" mental model regarding how HN works, when that simply isn't the case. Every online community turns into a self-censoring echo chamber with time, and given its origins and initial purpose, the precise kind of echo chamber that HN has been turned into shouldn't be a surprise to anyone.
- phillmv 13y agoThe part that bothers me is that it's not self-censoring. It's being actively censored, and sometimes the motives being the penalization are obnoxious; everything that doesn't directly promote california imperialism gets shunted to the side. I believe it to be uncontroversial that all communities need some moderation, but the lack of transparency or involved around these parts has always made me feel uncomfortable. I distinctly recall some articles on gender inequality in the industry…
- davidgerard 13y agoWell, duh. The actual purpose of HN is as an advertising and recruitment tool for YC. It's certainly not for the benefit of casual commenters like ourselves. I think of it as a more tech-oriented form of television. Possibly one of those high-hundreds cable channels.
- hmsimha 13y agoAccording to this article, the link currently in #3 [Vote Now: Who Should Be TIME’s Person of the Year? Edward Snowden](https://news.ycombinator.com/item?id=6800145 https://news.ycombinator.com/item?id=6800145) must be the victim of a very harsh penalty. I'm guessing 'Snowden' is a heavily penalized term as well, and having less than twice as many upvotes as comments might not be helping either.
- tehwalrus 13y agoInterestingly, the best strategy for keeping an article you like up high is to upvote it and not comment (or, if you must comment, do so only once.) If you do comment, however, you can be as verbose as you like (as long as you are bland enough not to provoke replies.) I wonder if this will change the strategy some post authors have of "hosting comments on HN" (and replying to every comment, even just to say "thanks".) EDIT: and to edit your posts instead of replying. I think this is penalisation of comments is a shame - I certainly come to HN for the comments, not the articles (although they're interesting stimulus for discussion).
- Zaephyr 13y agoAgreed - often I read the comments and never get to the original linked article.
- swombat 13y agoHilarious that the original article was flagged off the front page, but this one isn't... I find it very disheartening that the negative voices are being given so much weight. Everything that's worth doing will have detractors, and when it's something really worth doing it will have vocal detractors. Back when I had comments on my blog, every article I wrote that was any good had at least one person commenting that I was a moron or some equivalent statement. Great things arouse passion - on both sides. Giving 10x the power to the people on the negative side just creates an environment where new ideas are discouraged, where important but difficult discourse is pushed aside, where things of true import are penalised out of the group's attention by a few detractors. There does need to be a system for flagging and removing spam articles, but if this system can (as it plainly regularly is) be co-opted to remove articles from sight just based on not liking them much, then it is broken. The people who have flagging powers are not responsible enough to use them wisely, perhaps. I see at least one simple solution: lift the flagging privileges so it only becomes available to a much smaller segment of the population. Perhaps making the limit 10'000 instead of 500 would do that. That would still include hundreds of people, based on a quick extrapolation from https://news.ycombinator.com/leaders https://news.ycombinator.com/leaders ). An even better model would be to make it dynamic - perhaps the top 200 commenters...
- baking 13y agoWait until 9am or 40 comments, whichever comes first.
- 001sky 13y agoHN commenters have shown enough restraint !
- jamesbritt 13y agoWhich 9am?
- deleted 13y ago[deleted]
- ximeng 13y ago
- david927 13y agoThe post this morning, "Ask HN: What kind of side projects are you working on?"* got a lot of great responses and was killed because of it. There was nothing "controversial" about the post; it was merely popular with comments of what people are working on. This algorithm needs to be tweaked or HN risks losing its base. *https://news.ycombinator.com/item?id=6799694 https://news.ycombinator.com/item?id=6799694
- brudgers 13y agoThe number of links within it probably drags it down. Though I am ambivalent whether it's good or bad, I sort of see the point...there's not much discussion or analysis in the comments.
- david927 13y agoIt's bad because this is exactly the kind of thing that many on HN come here to see. I hate sifting through the "YC startup raises $50" news, but I love to see what people are working on, and I'm not alone.
- brudgers 13y agoI think the issue is that it's mostly just links, and not much meaningful feedback. People are mostly talking past one another and there was [when I looked] very little dialog. It was more like the monthly job thread and HN might have legitimate reasons for discouraging analogous threads for everyone's side projects. It is also possible that the thread's pattern of growth triggered some heuristic.
- kens 13y agoYes, with 82 comments and 52 upvotes (currently), that post will get a fairly substantial penalty. For the most part you don't need to be "afraid" of comments - an article is only penalized if it has more than 40 comments AND more comments than upvotes. Most articles have considerably more upvotes than comments, so commenting won't hurt them at all. But you might want check the upvote and comment numbers before adding a comment just in case...
- 001sky 13y agoOn average, about 20% of the articles on the front page have been penalized, while 38% of the articles on the second page have been penalized. (The front page rate is lower since penalized articles are less likely to be on the front page, kind of by definition.) There is a lot more penalization going on than you might expect. ==Why there will never be a Flat tax...
- muyuu 13y agoI see this is being penalised already >:-D
- alexkus 13y agoIf someone wants a 'weekend project': an interesting browser plugin would be one that undoes the effect of the penalties and reorders the page accordingly, it could even allow the user to choose the level of penalties enforced against stories in their view in case they did want some to disappear off the front page.
- DanielBMarkham 13y agoAt some point over the past 2 years HN has stopped being my friend. The folks here? Great people. Very happy to have gotten to know many of them. But the system itself? Not so much. People expect machines they interact with to behave in some kind of logical manner. After 2 or 3 times of submitting an article that HN has traditionally liked -- and watching it tank -- just not that motivated to submit more. After submitting my own articles, having people stop me in the hall and tell me they liked it and voted up for it on HN, only to see it have no votes? Not so motivated to submit more. After the tenth conversation about how people expect HN to act one way and instead it acts another? Not so crazy about it. I think the problem here is that PG wants folks to participate, but only to a certain extent. People want to interact with the system, but on some kind of mutually-fair terms. I'm not sure PG's goals line up with the average user any more. There are good reasons for this, and I'm not trying to trash the entire effort. It's just that this is a tough problem. I don't think you can code your way out of dealing with messy human issues at scale. If you could, we'd all be managed by computers in 50 years, and that's not a future I would wish for my children.
- larrys 13y ago"I think the problem here is that PG wants folks to participate, but only to a certain extent." Short answer is HN is basically deal flow for YC. To that goal it obviously works very well. PG and YC need "everybody" but yet they need "nobody". As a business HN works to achieve the greater goals of YC. I don't think it is about anything else and I haven't been around long enough to know if things were ever different. They may have been back at the start because I guess if you were to arbitrary you would never build up an audience or fan base. On a personal level I've always thought that people who get up to the karma level you are at actually deserve some special treatment because to me the top commenters are some of the glue that makes a place like HN special. In any business I have ever run you always give special treatment to the better customers.
- marshray 13y agoWatching https://news.ycombinator.com/newest https://news.ycombinator.com/newest for just a little while convinced me that there are far more decent submissions than there are upvotes. Got a better idea? There's never been an easier time for one to start their own link aggregator site.
- jader201 13y agoAssuming this article is correct regarding the penalization of comments, I'm a bit surprised (maybe even disappointed) that it is assumed that discussion is a sign of controversy. And maybe it is, historically? It's a shame for those articles sparking insightful discussion though. It seems like a weighted penalization could be implemented, potentially looking for red-flag words like "pedantic", or "not to be *". Or maybe it already is. Hope I didn't just set it off. :)
- deleted 13y ago[deleted]
- sethbannon 13y agoI find it really disheartening to learn that any article with "NSA" in the title is pretty severely penalized by HN's algos. This seems like one of the seminal issues of the decade, for this community in particular.
- ximeng 13y agoOn the other hand there's an article with "N.S.A." in the title at the top of the front page now.
- BrandonMarc 13y agoThus the back-and-forth continues.
- joering2 13y agoThis remind me initial war with Gmail on trying to change colors of links in your HTML email body. Google was replacing #000000 with blue links to make sure some won't deceive users that underscored black text is or is not a link. So of course people start using #000001 color, then this one got blocked they went into #000002. Gmail quickly realized they wont win this one :) so they revert it back to replacing only 000000.
- gamegoblin 13y agoMaybe they should search for [Nn][^A-z]?[Ss][^A-z]?[Aa] instead ;)
- clone1018 13y agoI think it's more there are SO many NSA articles it tries to make sure only the actually important and good ones get to the front.
- at-fates-hands 13y agoYou can apply this logic to several other topics of the day as well. What about bitcoin posts? What about all the Linux/MogoDB posts? I find it interesting only the NSA articles get penalized. When in reality, you can make the same argument for a host of other topics not getting penalized.
- lnanek2 13y agoInteresting. So if you want to get rid of the stories on the front page, and see more stuff, you should comment a lot. Because on HN a lot of comments is a death sentence for an article. On the other hand, if you are an article writer and add a "discuss this on HN" link in your articles, you should remove the link as soon as you get a good ranking. Or actually don't ask people to discuss at all, because it is harmful, just ask them to vote and have your own comment system for discussion. HN basically reinvented "sage", the concept from 4chan and its Japanese origins where people sometimes comment on a thread just to get it closer to the comment limit before it would no longer be bumped up to the front page when replied to.
- unreal37 13y agoArticles with a lot of comments without a lot of upvotes are probably rightly labelled "controversial". I sometimes comment without upvoting the OP on something I don't like. So this behavior by the algorithm matches my use of the site.
- damon_c 13y agoThe question is: Deep down, whether we realize it or not, are these unspeakable manipulations the reason we come here?
- cloudflare 13y agoIt would be interesting to know if there are 'flagging rings' in the same way there are 'voting rings' and whether HN actively detects the former as it does the latter.
- mcintyre1994 13y agoFor what it's worth, there are a lot of stories around of people losing flag rights after flagging too often.
- mikeevans 13y agoInteresting that github.com links are automatically weighted down, especially since almost every blog post they make hits the front page, and often projects hosted there show up fairly regularly as well.
- skeletonjelly 13y agoHN actually stands for Holman News
- franstereo 13y agoAs other folks have mentioned there is a raw feed if you want to see a non-penalized version. It would be interesting if it somehow incorporated other elements to determine article "value": - Open rate - Ratio of comments to opens - Time spent on article or comments - Depth of comments
- anaphor 13y agoSentiment analysis is becoming pretty accurate so maybe you could take into account the number of negativew/positive comments as well. Maybe that will eventually replace voting systems, who knows?
- sixtypoundhound 13y agoUsing score adjustments as automatic community management makes a lot of sense; there are certain topics which are more likely to upvote/rank than others. Similarly, there's also a good bit of research that controversial articles generally do better on Reddit and other social news sites. Applying an automatic penalty to certain topics / tactics which are likely to gather excessive upvotes, due to the nature of the content vs. it's quality, helps ensure you've got a diverse mix of content occupying the front page. Which is generally good for the overall user experience. Otherwise, the front page will be a massive list of shock jock posts about the NSA.... [since controversial posts about those subjects will get sympathy votes, regardless of their actual contribution to the community...]
- brosco45 13y agoAnd censorship
- pearjuice 13y agoI think what is more interesting, is what drives people to upvote certain things.
- brador 13y agoBlog spam link. Real content from the article at http://www.righto.com/2013/11/how-hacker-news-ranking-really-works.html http://www.righto.com/2013/11/how-hacker-news-ranking-really...
- alok-g 13y agoInstead of using ad hoc scoring rules like that, HN could use a machine-learning based system. This can also help solve another issue -- automatically determining the initial score of the new stories.
- ChuckMcM 13y agoIts a wonderful analysis, kens if you would ever like to come work for me identifying robot search clients just give a shout :-). I chafed a bit though at calling it a 'penalty'. Isn't it really a 'moderation' ? The scoring is adjusted by the moderators to be more the site they want to have and so they moderate articles that they feel aren't appropriately more heavily than those that are appropriate? I understand that for some people the moderation choices offend them, I think that is unavoidable, but the goal is, I believe, to make a 'better' collection not to shoot down particular articles.
- Samuel_Michon 13y agoTL;DR: “If an article has more comments than votes, don’t add your comment to it or you may kill it off entirely!” Rings true to me and, if indeed accurate, it seems like a good practice for HN.
- ternaryoperator 13y agoInterestingly, one of the few ways to vote down an article on HN.
- krapp 13y agoComments should matter more than votes. A thread with all votes but no comments is a meaningless number, but a thread with all comments but no votes is exactly the same as that thread with votes. That particular weighting doesn't make sense to me. Clearly some people vote by commenting. Which is as it should be.
- michaelochurch 13y agoI get a personal penalty, because someone with moderation powers (possibly PG, but I'll give him the benefit of the doubt) is a wolfbagging pissant. Apparently being brutally honest about VC means that everything I say is of low value. For more, go here: http://michaelochurch.wordpress.com/2013/11/03/heres-why-paul-graham-probably-owes-me-an-apology/ http://michaelochurch.wordpress.com/2013/11/03/heres-why-pau...
- grey-area 13y agoThis article was more interesting than I anticipated. While I admire the tinkering which goes on with moderation here in an attempt to keep discussion civil and interesting, sometimes it has counter-productive effects. In particular this rule doesn't seem to work very well: In order to prevent flamewars on Hacker News, articles with too many comments will get heavily penalized as controversial. In the published code, the contro-factor function kicks in for any post with more than 20 comments and more comments than upvotes. Is a vigorous discussion bad? Should everyone commenting also upvote?
- md224 13y agoI find this interesting as well. Flamewars are counterproductive, but if I had to pick between flamewar and no discussion at all, I'd probably pick flamewar. After all, it seems like flamewars sprout when ideologies clash, and shielding ourselves from ideological clashes feels like it encourages insularity of opinion. On the other hand, one could argue that heated arguments increase polarization, so I'm not really sure what should be done. I just think there needs to be more dialogue on topics that people disagree on, not less.
- danielweber 13y agoI've seen communities that banned flame wars. They succeeded, in the same way that ghost towns have no crime.
- jamesbritt 13y agoI used to imagine that scores were positively influenced by commenting. If there are comments then people thought the post worth noting. I typically comment without upvoting. Maybe now that needs to change if I want to see discussion continue on topics I'm involved in.
- mturmon 13y agoMy personal heuristic for reading is "if more comments than upvotes, don't bother". Such posts usually have a lot of repetitive, obvious comments -- typically bike-shedding, people trading anecdotes, etc. Some specific examples might be questions that touch on child-rearing, or college experiences, or city-of-residence questions. Everyone knows next to nothing (sample size 1) but feels compelled to share. It's not information-dense enough to bother with.
- vijayboyapati 13y agoWhat I find quite strange is that I have posted articles which show up in the new section, then later when they make the front page, it shows them posted as someone else. Either dupe detection isn't working (these are very simple URLs typically, like from the New York Times) or HN is rewarding the post to someone else after I've posted it. Weird.
- twotwotwo 13y agoNot everyone has the same criteria for what content they want to read. Nor does it really help the world for all HN visitors to read the same content. Would love to see ideas that broke from the model of a single ranked list: let folks tune their personal penalty amounts and gravity; add random jitter to rankings and throw a couple random new stories onto each list; classify/cluster users by their votes, so people who vote for jokes or NSA articles or their neighbors' articles (automatically) see more of those things. It's maybe a bit much to ask PG and co. to architect radical alternatives to HN, because HN is a handful as it is and, besides, I hear they have day jobs. It could be cool to let a thousand flowers bloom: publish most of the now-hidden ranking data (maybe not all, because it can be useful to obscure how anti-spam algorithms work); let users opt in to publishing anonymous votestreams for clustering, etc.; then let other folks use all of this to make their own homebrew HN frontends within certain limits. I suppose that, too, is kind of a pipe dream, because opening HN up for people to easily build their own frontpages is far-from-trivial for both tech and policy reasons. But it's a nice pipe dream.
- analog31 13y agoBeautiful MatPlotLib work. ;-)
- atmosx 13y agoHm, this post was kinda bad for me to read. I thought this was a completely community driven board. But I'm a fairly new member and some things never change, apparently.
- Eye_of_Mordor 13y agoWhy doesn't everyone vote for this article and leave a comment?