4 ms·
Hey guys, Being a long time reader of HN I'm sometimes frustrated that top HN content is often coming from popular websites or talk too much about the same thi
by polote 6y ago
Hey guys,
Being a long time reader of HN I'm sometimes frustrated that top HN content is often coming from popular websites or talk too much about the same thing (Zoom security issues, Facebook leaks, ...) I wanted to understand if this was possible to get only the 'original content' posts of HN (from the not-as-popular blogs and sites). Because to me this is the most interesting part of HN.
So I started analyzing HN new posts, and made a few discoveries :
- Some HN users post a lot of content, and post several links in a row, which will push away your post if they publish just after you
- You get a 30 min time window at busy hours (and 1h time window at non busy hours) between the time a new link is posted, and the link disappear from the 1st page of new links
- There is a second-chance pool for good stuff if a moderator detects it
As a result HN is overflowed with not really useful content and it is not always easy for original content to be noticed (Even if I think HN is doing a very good job compared to any other link aggregator that you can find)
So I tried to built a tool to filter out things like news websites, words that I want to blacklist, and users whose posts haven't been relevant to me.
That way I'm able to remove almost 80% of content and I can go through the list of all the links of the day before going to bed
For those who are interested, this is just a cron job querying the HN API every 3 minutes inserting the new links into a db, and a web server rendering the last 500 links.
You can see more on how the filters works here: http://hn.luap.info/about http://hn.luap.info/about and you can also understand which links have been filtered here: http://hn.luap.info/links_flagged http://hn.luap.info/links_flagged
- tbt43p 6y agothank you, it had to be done before or after... good job!
- optimuspaul 6y agoclearly we have different interests in what we like from HN. many of the sites you filter are sites that I like prefer their content because they do offer commentary that provides context and perspective that I don't always get from what might be considered 'original content' by this. Interesting experiment though, might provide more value to someone like me if rather than a black or white list it was more of a content assessment model using NLP.
- polote 6y agoHello, clearly the goal is not to replace HN, I love also reading the comments of people on global topics, but I also love reading original content
- amelius 6y agoPerhaps what we really need is a personalized feed.
- KhoomeiK 6y agoI'd honestly be surprised if something like this hasn't been built yet by the HN community. A service where you can input your likes and interests (and maybe a blacklist) to get a tailored feed of HN posts including/excluding the right content. Maybe you could even feed in a few links to examples of content you like and it would cater to you based on that.
- JetSpiegel 6y agolobste.rs tags?
- oceanbreeze83 6y agomaagnit.com also similar to lobsters tags
- kube-system 6y agoFor some of us, the whole reason we're here is to get away from personalized filter bubbles.
- kick 6y ago30 minutes during busy hours seems optimistic. I've seen it get as low as ten, but very few people check past the first five links on /new anyway.
- Tempest1981 6y agoI wonder what % of readers move on to Page 2 of HN? Or Page 3?
- blondin 6y agothis offers a fresh perspective. thank you! once in a while there is a blogger that offers a different view or perspective on a topic that has been beaten to death here. them being noticed at all is simply magical since they have to compete with people's friends and colleagues around here... i absolutely love that you have more than 30 articles per page.
- aabbcc1241 6y agoI agree your mindset but links from github and gist may still be original. Maybe we can use the username of the poster and linked resource to district that.
- aabbcc1241 6y agoI've been constantly using your site, thank you