8 ms·
Tagger News takes a subset of HN articles, analyzes using ML, and applies tags
- nickpsecurity 9y agoStrength of Hacker News is the network effect of a diverse, intelligent crowd. It would be hard to replace. A supplemental site tagging it and aiding search has value. Biggest problem I have searching HN, though, is Google mixing up stories and comments. The fix might be as simple as two domains that contain stories and comments cloned from HN, one domain for each, followed by Google Searches within those domains. Not sure if Google would automatically crawl it, though.
- CaliforniaKarl 9y agohttps://hn.algolia.com/ https://hn.algolia.com/ does the job pretty well!
- SparkyMcUnicorn 9y agoThe brand new https://hacker-search.net https://hacker-search.net isn't bad either.
- gnicholas 9y agoAlso, you can easily search HN in Chrome: start typing news.ycombinator.com, it will be ready for autocomplete, and it will say in grey text "Press [tab] to search HN". The results show up from algolia. If only there were a way to make the default search order recency instead of popularity — most of my searching is before posting something, to make sure it hasn't already been posted.
- zulln 9y agoThe search bar at the bottom of news.ycombinator.com also takes you to that site.
- oneeyedpigeon 9y agoWeirdly, the default values for the search url are slightly different from what the Chrome search populates them with. The 'address bar search' defaults 'dateRange' to 'all', whilst the script itself defaults 'dateRange' to 'last24h'. Does anyone know how the Chrome address bar search is implemented?
- DanBC 9y agoYes, I used to be critical of the search but it's actually pretty good. I do wish there was a way I could set the defaults. I almost always want to search comments and sort by date.
- deleted 9y ago[deleted]
- searchhn 9y agohttp://searchhn.com http://searchhn.com is a small demo we have been building at Searchera (https://searchera.io https://searchera.io)
- rileymat2 9y agoPersonally, I find the visual weight that the tags have exceed their value to me.
- Candles123 9y agoSame first impression. No subtlety there
- t3f 9y agoI would agree, for me the bright blue tag with the sharp box really makes it hard to scan the headlines without a ton of concentration ignoring the tags separately.
- qznc 9y agoThat page is to show off their work, which it does. The goal is probably not to replace news.ycombinator.com.
- deleted 9y ago[deleted]
- nixhope 9y agoAgreed. Tag colours draw too much attention, particularly contrasted against visited links. Custom tag groups could be a useful extension.
- retreatguru 9y agoI find the white on light blue very unreadable.
- asymmetric 9y agoCongratulations to the team, although it seems the algorithm isn't very accurate, since this article[0] from 1997 was tagged with "Blockchain". [0]: https://www.gnu.org/philosophy/right-to-read.html?source=techstories.org https://www.gnu.org/philosophy/right-to-read.html?source=tec...
- georgemcbay 9y agoOr maybe the algorithm is scary accurate and has correctly deduced that Richard Stallman is, in fact, Satoshi Nakamoto! Somebody contact Newsweek!
- thedevil 9y agoAs a sincere question, why did people downvote this comment? I thought he was funny and I think lighthearted humor has value to it. It didn't seem snarky to me, did it to someone else? Did it seem off-topic? It's a joke rather than useful information, but I'd argue that it is on-topic per the rules: "Anything that good hackers would find interesting." Am I missing something?
- minimaxir 9y agoIt's a Reddit-esque comment that adds no value. (and at best, it's off topic where the topic is about an improved Hacker News) There are instances where humorous comments can add value (e.g. irony/satire), but it's not common and hard to do properly.
- minimaxir 9y agoDirect URL to project details: https://devpost.com/software/tagger-news https://devpost.com/software/tagger-news A few comments: 1. To other commenters, as with the HN Vue demo a week ago (https://news.ycombinator.com/item?id=14284877 https://news.ycombinator.com/item?id=14284877), the project is a technical proof-of-concept; the aesthetics aren't the primary focus. 2. The Algolia API is better for scraping because it allows for bulk requests, unlike the official API (my old 2014 script still works I think: https://github.com/minimaxir/get-all-hacker-news-submissions-comments https://github.com/minimaxir/get-all-hacker-news-submissions...) 3) How much time did it take to manually label the training/test set before training the RF classifier? Even with topic modeling for extrapolating tags, accurate labeling for 20,000 submissions is a task.
- var_explained 9y agoOne of the devs here. 1. That's the way we were thinking about it :) 2. Oh, excellent! We hadn't found that or we'd have used it, and we'll start working with it. 3. Tomorrow I'm going to blog about how we approached the machine learning. Short version; we manually came up with regular expressions to classify a training set based on titles. The idea is that when we experimented with manual annotations on titles, the vast majority of the time we were looking for only a few key words. There's no question that this adds biases and will not be entirely accurate, but manual inspection convinced us it was a good enough approach for our hackathon, and most of the articles we identified with the resulting algorithm would not have been found by the title regex alone. You can see the table of regular expressions [here](https://github.com/dodger487/analyze_hn/blob/master/topics.csv https://github.com/dodger487/analyze_hn/blob/master/topics.c...) and a bunch of (pretty unstructured) analysis code [here](https://github.com/dodger487/analyze_hn/blob/master/hn-analysis.Rmd https://github.com/dodger487/analyze_hn/blob/master/hn-analy...).
- jibbolo 9y agoHey mate, you should follow this guide step by step when you deploy a django app: https://docs.djangoproject.com/en/1.11/howto/deployment/checklist/ https://docs.djangoproject.com/en/1.11/howto/deployment/chec... BTW, congrats for the projects, well done!
- rocky1138 9y agoStop trying to remake Hacker News. It's pretty much perfect the way it is.
- saganus 9y agoMaybe it's not perfect for everyone. I, for one, do like the idea of tagging stuff, since I might favorite a lot of stories but then years later it's hard to find a particular one, even if you do remember the general topic. Tags for this would be really helpful for me, ergo it's not perfect for me, ergo it's good someone else is trying to make it better. Since it's not affecting the original site, why would you want to stop them?
- gue5t 9y agoThis is the best joke I've ever read on HN.
- sergiotapia 9y agoRebuilding HN in some fashion if a yearly thing. I rebuilt it nearly 5 years ago, and even then people told me it's a yearly occurrence haha. https://news.ycombinator.com/item?id=4376184 https://news.ycombinator.com/item?id=4376184
- Markoff 9y agoit's difficult to find more mobile unfriendly page nowadays than Hacker News Materialistic on Android makes it decent, but would still prefer something like Boost for Reddit for comments
- shawkinaw 9y agoI think this is great, especially being able to click a tag and see a top 30 list of that tag. Obvious suggestions that would make it usable as a primary HN interface: • Login and voting (not sure the HN API supports this though) • Tag suggestions to feed into the model
- robertelder 9y agoI think the biggest value proposition of this is the ability to do sub-reddit like filtering on specific tags. As Hacker News grows I think dealing with the number of new submissions would become a bottleneck. During high-traffic times, new submissions sometimes drop of the first 'new' page in around 10 minutes. Of course there is more traffic during these times to upvote good content, but I'm not sure that is better than letting a smaller number of people have a longer period of time to filter a smaller collection of content.
- HiroshiSan 9y agoNot too mention a lot of the content has strayed away from technology and has gone towards mental health, personal growth, news, politics, etc.
- rfrey 9y agoI don't remember a time when all those topics were not common.
- HiroshiSan 9y agoMaybe now that I'm older I notice it more than I did when I first joined.
- staz 9y agosubreddit force are their communities Also in 10 minutes there is no time to read every articles (or any good long for that matter) so not sure what that leave on the front page
- j_s 9y agoIf you or anyone else is interested in a 10 minute audio summary of Hacker News, let me know. https://goo.gl/forms/En1iuoUcf1qVfSHr2 https://goo.gl/forms/En1iuoUcf1qVfSHr2 I've also posted this sign-up as a discussion if you want to leave any comments for everyone to see: https://news.ycombinator.com/item?id=14338456 https://news.ycombinator.com/item?id=14338456 There are also options like HN top 10: http://www.daemonology.net/hn-daily/ http://www.daemonology.net/hn-daily/ Or the ultra-cynical weekly antidote to HN: http://n-gate.com/ http://n-gate.com/
- pera 9y agoWow I was thinking to make something similar: an experimental HN fork where submissions are tagged (collaboratively) but without titles, as these are rarely useful to predict the content of an article. And of course there is also the convenience of categorization.
- sitkack 9y agoYou are describing lobste.rs
- faragon 9y agoUsing ML? Why not using just Bayesian filters?
- var_explained 9y agoYou mean Naive Bayes? Because it can't account for interactions between the effects of multiple words.
- faragon 9y agoJust add some "magic", e.g. per response analysis and inter-response rule-based system.
- var_explained 9y agoIf you keep adding "magic" and doing careful research on what magic works and what doesn't, you end up roughly with the modern field of machine learning. Random forests are a method that's often effective in taking into account many interactions among high dimensional data.
- faragon 9y agoExpert Systems "magic" predates neural networks by decades, being predictable and giving validable results (unlike most ML models).
- swetabhsuman8 9y agoREVEALED: THE 22-YEAR-OLD EXPERT SAVED THE WORLD FROM RANSOMWARE VIRUS http://hackernucleus.com/revealed-the-22-year-old-expert-saved-the-world-from-ransomware-virus/ http://hackernucleus.com/revealed-the-22-year-old-expert-sav...
- big_spammer 9y agoLink to try it out http://www.taggernews.com/ http://www.taggernews.com/
- bexp 9y agoI did similar thing + summarization a while ago with open sourcing the project http://hntop.org http://hntop.org https://github.com/bexp/textai https://github.com/bexp/textai
- hntop 9y agoI did similar project few months ago, it does automatic tagging + summarization of HN largely using scipy and numpy, you can see it in action: http://hntop.org http://hntop.org here github link https://github.com/bexp/textai https://github.com/bexp/textai
- egypturnash 9y agoPossibly off-topic, but: this was done at the "Disrupt NYC" hackathon, and somehow "add tags to HN" feels like about the least disruptive thing ever.
- Imagenuity 9y agoThis is what @twitter needs to do to make discovering worthwhile information better. Hashtag spam makes #hashtags mostly useless as a method of discovery.
- the_arun 9y agoTried http://www.taggernews.com/tags/aws/ http://www.taggernews.com/tags/aws/ and didn't find any results. Is it because it is listing only few tags for now?
- sasoon 9y agoHere is my take on it few years ago. I tried to make it more like magazine, and get article text and photo, and there is a section with only articles that reached top position. http://www.hnzine.com http://www.hnzine.com
- RichardHeart 9y agoThis article on book publishing: https://news.ycombinator.com/item?id=14334845 https://news.ycombinator.com/item?id=14334845 is tagged with "blockchain" only. Any idea why?
- salmonfamine 9y agoNot to hijack, but this is similar to a small ML project a friend and I built. It takes news headlines from a bunch of sources and classifies them by common topic. We took a lot longer than a day to build it, though. ;) It refreshes with new stories every few hours. You can check it out here: http://headlinr.herokuapp.com/ http://headlinr.herokuapp.com/ EDIT: click on the bubbles to see individual headlines. Also, here's the GitHub page: https://github.com/dgarrick/headliner https://github.com/dgarrick/headliner
- magicmikexxl 9y agoCan you also add a way to add TLDRs to everything, pls? :D