3 ms·

I agree with this post but I'm more optimistic than the author that a solution will be found. Let me use HN for examples, as that's the site I'm most familiar

I agree with this post but I'm more optimistic than the author that a solution will be found. Let me use HN for examples, as that's the site I'm most familiar with. The author correctly identifies that HN does a poor job of selecting new items to display. Technically, HN exploits too much (gives existing high ranking stories too much front page time) and doesn't explore enough (give new stories a chance). I also expect that the population who reads the "new" list behaves differently to that who read exclusively the front page. We can fix both problems! We know of algorithms to balance exploration and exploitation -- they're known as bandit algorithms. Instead of displaying a fixed front page to everyone, choose the stories to show to balance exploration and exploitation. This way new stories get a fairer chance and the population observing both is the same. There are some technical issues but they are solvable. I work on this stuff in my day job (Myna -- mynaweb.com -- go sign up now! ;-) and we have an algorithm for handling preference data. It would need some adjustment but it's doable. Next up is voting rings and so on. Also solvable -- stop using votes as the only input signal. Observe what people actually do -- do they actually click the link? Come back and comment? These are strong indications of interest. How much text do they write? It's just a few words it's more likely to be a spammy "ata boy!" comment. How long do they spend on the link (maybe we can track this by using an iframe?) and do they scroll? The point is there are a whole heap more signals that people give to indicate real interest. Start using these and it becomes much harder to forge interest. This area is known as implicit feedback, and again there is a decent body of work available.

jamesmiller5 14y ago
I have noticed that when the comment count surpasses the up votes the article is usually very confrontational but isn't necessarily authoritative or insightful on the subject matter.
nwzpaperman 14y ago
HN and reddit are really social curating dashboards with a commenting system. both are better than self-described curating sites like storify. a major problem is the overwhelming flow of links from the entire internet because of the third-party content structure. exploration platforms like stumbleupon send garbage traffic, so that's not the answer, either. you don't want traffic for traffic's sake, you want real people that read and contribute. news is a special species of information: 1) all "open" information in the sense that it is intended for the general community. 2) it is time-sensitive 3) it is geographically relevant the current internet is structured for historical or aged-content. that's why you see all of the indexing of third-party content instead of an integrated platform that standardizes the news format and can personalize content for anyone around the world. you have to look at the production side of the equation, not just the product, if you want to solve the news problem. considering that all of the VCs have given up and the legacy publishing institutions are approaching the problem from the perspective of adapting their structure, no one you know of is actually working on the news problem. it's a multi-billion market opportunity with very little competition. the potential financial gains are exciting, but at the end of the day ( and the beginning), everyone wants an information edge. I want the information edge! i've got years invested in the problem and a public beta platform to iterate up from. it will be obvious in hindsight! :)

jamesmiller5 14y ago

I have noticed that when the comment count surpasses the up votes the article is usually very confrontational but isn't necessarily authoritative or insightful on the subject matter.

nwzpaperman 14y ago

HN and reddit are really social curating dashboards with a commenting system. both are better than self-described curating sites like storify. a major problem is the overwhelming flow of links from the entire internet because of the third-party content structure. exploration platforms like stumbleupon send garbage traffic, so that's not the answer, either. you don't want traffic for traffic's sake, you want real people that read and contribute. news is a special species of information: 1) all "open" information in the sense that it is intended for the general community. 2) it is time-sensitive 3) it is geographically relevant the current internet is structured for historical or aged-content. that's why you see all of the indexing of third-party content instead of an integrated platform that standardizes the news format and can personalize content for anyone around the world. you have to look at the production side of the equation, not just the product, if you want to solve the news problem. considering that all of the VCs have given up and the legacy publishing institutions are approaching the problem from the perspective of adapting their structure, no one you know of is actually working on the news problem. it's a multi-billion market opportunity with very little competition. the potential financial gains are exciting, but at the end of the day ( and the beginning), everyone wants an information edge. I want the information edge! i've got years invested in the problem and a public beta platform to iterate up from. it will be obvious in hindsight! :)