8 ms·
Show HN: Hacker News Comments
- nickwoodhams 14y agoGreat contribution Nathan, looking forward to using this to embed comments after my next blog post. Can you let me know how the comment system works? Are you scraping HN?
- nathancahill 14y agoThanks! I'm respectfully scraping. I've rate limited all calls to YC's servers and am caching all comments on my side.
- nickwoodhams 14y agoSeems respectable. Out of curiosity, why did none of the existing 3rd party API's suit your purpose?
- redslazer 14y agoProbably the same reason no one should you use this to power their comment section there is no revenue, and no real reason to keep it up. Things like this come and go.
- mootothemax 14y agoWow, I really like this, great work! I've previously thought that comment threads on blogs tend to die down fairly quickly, whereas discussions on HN tend to last a bit longer, so this is doubly cool. I guess the only downside is the near-impossibility of letting people comment directly from the page without having to log in again.
- nathancahill 14y agoNot a downside. I'm not allowing posting comments or upvotes through this system to maintain the integrity of the discussion on HN.
- sgdesign 14y agoI'd definitely use this on my blog (http://sachagreif.com http://sachagreif.com) if it was officially supported (or at least tolerated) by HN somehow. And the fact that people can only comment if they have an HN account makes it even better!
- jacobr 14y ago> Although many websites and blogs have a comment system, the quality of the discussion on Hacker News tends to be much better. Wouldn't this risk changing that?
- danmaz74 14y agoProbably not, unless your blog has an organic audience that is bigger (or at least comparable) than that of HN homepage...
- nathancahill 14y agoBlogs this big are unlikely to implement a third-party (and unsupported) discussion feed.
- ooobo 14y agoI would see that eventuating if this allowed commenting via the embedding blog, thankfully it doesn't. I don't have the same issue with this that others do, but I remember TheNextWeb getting a bit of slack from commenters for embedding HN comments (although that was mixed amongst other comment platforms, and is a little different) API-wise, http://api.ihackernews.com/ http://api.ihackernews.com/ , http://hndroidapi.appspot.com/ http://hndroidapi.appspot.com/ or http://www.hnsearch.com/api http://www.hnsearch.com/api didn't suit? [edit] what am I thinking, an API doesn't need an API... Oh, and simplicity of this seems pretty nice!
- danmaz74 14y agoVery, very interesting! So you already calculated that the cost of this will be insignificant for you even with millions of pageviews? Blogs have a long tail of visits... ;)
- irrationalfab 14y agoWhy isn't this open source?
- nickwoodhams 14y agoWhy would you want open source for this? Do you want to spend the time to deploy your own instance just to have the exact same result?
- sp332 14y agoTo learn from it; so that more than one person can work on improving it; just in case I don't trust the submitter to serve code to my visitors.
- nickwoodhams 14y agoYeah, that makes sense. What would give you the confidence necessary to have someone else/Nathan Cahill host the server?
- muellerwolfram 14y agoyes i would. to implement the whole thing, instead of only this little jquery snippet shouldn't take much longer. but then i'm not dependent on someone elses server being up/fast.
- nickwoodhams 14y agoThat makes sense. Maybe it will go open source in the future. It doesn't look like Nathan has a revenue model for this so he may be open to that.
- gcmartinelli 14y agoIf everyone could implement their own scrapper they would be able to follow robots.txt guidelines/limits, but HN servers might take a big blow it it gets popular.
- 14y ago
- unimpressive 14y agoIn my preferred model of Internet content, stuff like this would be available for every element without scraping. Good job.
- jerf 14y agoThe word for that is http://en.wikipedia.org/wiki/Transclusion http://en.wikipedia.org/wiki/Transclusion , and it's interesting that it was a core component of Xanadu (see the Wikipedia link there).
- unimpressive 14y agoYeah. I know. We also need a micropayment processor that can do sub-penny transactions. EDIT: Actually, I would be fine with penny transactions considering that the current state of things with say, paypal; is 30+ cents per transaction.
- beshrkayali 14y agoThis is very interesting! Great work.
- sp332 14y agoDo you think people are going to wait for someone to submit each post to HN? I think this will encourage writers to submit every single story they write to HN, just to get a comments page.
- nickwoodhams 14y agoThat's a good question, but it's probably a very small percentage. And they can always add it later.
- samjc 14y agoEven if someone put every single story they write on HN, that's not a guarantee that it will get comments, since it might not get up-voted. I think if I were to implement this on my own page, I would probably have two tabs that expand if clicked. Something like this: Show [your site's name] comments | Show HN comments And then for the HN comment tab, you would use the OP's awesome submission. Thanks, btw!
- sktrdie 14y agoThis would make more sense if it was entirely client-side code towards an official JSONP endpoint, such as the official hacker news API.
- mattacular 14y agoNot saying this is necessarily true for Hacker News, but unfortunately a lot of sites don't have the time/resources to develop and then maintain a public API for the majority of their content. And mostly see an RSS or XML feed of their news/articles/stories as good enough.
- Davertron 14y agoHow hard would it be to have your script look for the submission on HN by url instead of having to manually put in the id? Maybe it could use the one with the highest score if there's more than one?
- nathancahill 14y agoYes, that's the plan! A new API endpoint will use the referring page to capture the URL, then link it with the appropriate discussion on HN.
- progrock 14y agoThis idea is kind of alluring. But you can't really include discussions on your page that might be happening all over the Internet. Reddit etc. This is where the browser should really be smart enough to do a search engine lookup, and propose smart links alongside the page you're reading. Or even summarise and merge discussions found elsewhere.
- personlurking 14y agoI'm going off topic a little but... An element I really like from random blog posts I've seen off of HN is using the sidebars for additional, short commentary/details. Your comment reminded me of that, but sadly this might not work well if an article were all chopped up with tons of commentary (I'm thinking of used books where 'important' parts are highlighted and comments put beside them by a random reader, which I dislike). Well, then again, perhaps for people who like to dive into subjects and really digest an article, there could be a straight text version and a feature to see it marked up w/ smart commentary. This also reminds me of a small 'movement' a year or two ago for citing in-article links at the bottom instead of linking to them throughout the article itself.
- progrock 14y agoDoes it really matter where the links are listed - in the article or at the end? Opera has a side panel, one of which is the Links panel. That pulls links out of the page and lists them alongside. But this alone isn't that much use. The browser however could do something more interesting with them. I think a lot of pages seem dreadfully wasteful by placing in the same old accessory content on their pages. Buttonitis etc. I don't think this necessarily should live 'inside' the 'page' at all. The content should remain dumb. The inbound links, and chat around the content augments it. And this could be summised somehow by browser tools / other services. I understand the author's want for embedding HN comments on a page. There's a simple and elegant solution to the problem. Simply open up another browser window/tab and point it to the relevant HN comment page. To do so, the author could include a simple link in the page, leave it to the browser to infer, or just let the user discover it for themselves (pointers do help though.) Why the obsession of stuffing our pages with content? It also feels a little snooty doing something like this (embedding HN comments on your page that is.)
- jdrake3 14y agoI personally find this beneficial, and look forward to seeing it implemented on more websites.
- quesera 14y agoScraping comments on HN hits the robots.txt wall rather quickly. At a maximum 2 scrapes per minute, your cache will be 60 minutes old when you're watching 120 (single-page) comment threads. That might be tolerable. Now try 1200 or 12,000 comment threads. You can do smart cache refreshes, but you will miss bursty activity. You can ignore articles older than X days, but you'll miss the "dupe, comments here:" resurgences. HN gets about 150 posts per day that hit the top 60, at least briefly, as a point of reference. This is a neat thing, but if it gets popular you will have a problem. On the plus side, it's self-limiting. As the usage gets higher, the experience will degrade and usage will decline. No intervention required by you. :) I've wished for a policy-compliant way to cache comments in a timely manner, but there's just no way to stay current under the existing rules.
- rcavezza 14y agoWhat if you're scraping with ajax? Wouldn't each individual user's IP take the hit and not the domain's IP?
- dchuk 14y agoThe ajax request looks to be getting proxied through this guy's server. You have to do something like that because of cross domain ajax issues
- NathanKP 14y agoYou can't scrape with AJAX because of cross domain security restrictions. One potential solution to obey robots.txt might be to spawn multiple small EC2 instances with different IP's and have them coordinate with each other to share the crawling without individually running over the limits. (This is also useful for scraping from sites that have rate limits)
- typpo 14y agorobots.txt doesn't enforce itself so there is no IP limitation; this is still a violation and no better than simply lowering the delay on a single scraper.
- jere 14y ago>Although many websites and blogs have a comment system, the quality of the discussion on Hacker News tends to be much better. Probably because your random internet stranger has never heard of HN. It's a shame, for instance, that I was ever allowed to find this place.
- seagreen 14y agoIf that ever changes and HN is overrun (insert dramatic and sad Hans Zimmer score) what about starting a new HN just for programmers? I wouldn't be able to get in at the moment, but someday I'd hope to be able to.[1] [1] I'm assuming you'd have some kind of rotating or individualized Project Euler style problems that you have to solve to get in, and that they would be pretty tough. It occurs to me that you could also have temporary student accounts for people who are just getting into the software world as well though.
- jere 14y agoClever. It's like CAPTCHA for programmers. It wouldn't be much more difficult than the current registration process.
- mapleoin 14y agoPassing a programming test is no guarantee for having anything interesting to contribute to an online community, just look at your company's internal IT mailing lists.
- seagreen 14y agoTotally! That's why I'm not suggesting anyone do it now. There's a place for kludgy solutions, but not until you've tried everything else.
- pdeuchler 14y agoAn invitation based system, while not perfect, would probably be the most effective method. Plus, the owner of the site could restrict or encourage growth by the amount of invitations allocated... allowing for easy management of unwanted lulls or spikes in user activity. Invitation based communities are also very self-policing. If you invite someone and they happen to be an idiot then there is egg on your face as well.
- joshaidan 14y agoHere's a question, is it possible to put a Hacker News upvote button on your blog post? Or is that against the HN rules, or something that's frowned upon in the community?
- sbashyal 14y agoI created hnlike.com about a year ago that helps you to setup an upvote button for your blog. I have disabled the upvote feature as the community seemed worried. See the HN discussion on my story for details.
- nathancahill 14y agoI'm not supporting upvoting or posting comments. I like HN how it is now (or more so a year ago).
- danjessen 14y agolooks good :)
- swah 14y agoUpvoting gives the same result as "nice job!" comments like yours :)
- deleted 14y ago[deleted]
- mattjball 14y agoJust a heads up, I tried to test out your inline "Give it a try below!" but it's not working right now. In the browser console: "GET http://hncomments.nathancahill.com/comments/4285019/html/ http://hncomments.nathancahill.com/comments/4285019/html/ 500 (Internal Server Error)"
- nathancahill 14y agoThanks, there were a couple hiccups handling the large amount of traffic. Should be in working order now.
- sirbrad 14y agoNice work, this is pretty cool! I've been working on a similar approach but using Github (mainly because I host my blog there using Jekyll). I'll try and get it released tonight :)
- biot 14y agoHow do people opt-in to allow their comments to be shared on sites other than HN? Or is this strictly opt-out and, if so, what is the opt-out mechanism?
- bherms 14y agoI hope this doesn't further impact the quality of discussion on hacker news. It seems that every day I notice more and more people posting completely non-contributing crap in the comments that you'd expect from Reddit or something (one-liners, stupid jokes, etc). I go to Reddit for that, I come here for insight (which, granted, you can find a lot on Reddit also, but HN is supposed to be a bastion of intellectual thought and discussion).
- johnconroy 14y agonice idea, nicely implemented