7 ms·
My Hacker News firehose
- daxelrod 16y ago>Apparently that's because Hacker News has blocked his API. Does anyone know if this is actually the case? http://api.ihackernews.com http://api.ihackernews.com gives me a 404, but I'm posting this from ihackernews's browser interface, which obviously is getting updates.
- ronnier 16y agoI took down the site after YC blocked my IP address. I believe it was blocked as a result of increased usage from the explosion in traffic, resulting in some really heavy usage of the API. My guess is that YC's software did this automatically to prevent abuse. In addition, many folks weren't happy with me distributing the HN database. So there really wasn't a reason to keep it up. I'm going to rework the API such that it will work within the boundary of an acceptable scrapping rate. I'm not putting the database back up.
- ajg1977 16y agoThat's a real shame, I was thinking of building something on it too. I wish there was a way to get HN pages in simple JSON/XML.
- daxelrod 16y agoThank you for the confirmation. Will the ihackernews browser interface remain active? I hope so, because I love it and use it every day. Whether or not it will, thank you so much for creating it, as well as the API!
- deleted 16y ago[deleted]
- ig1 16y agoHow about putting the database back up without the comments ?
- deleted 16y ago[deleted]
- tptacek 16y agoHacker News isn't a potentially valuable Internet resource waiting to be mined for interesting new applications. That's Twitter. Hacker News is an actual community. I don't know or care why the API got blocked, but I've been as happy with as many HN add-on apps as I have been incredibly irritated by them (paging: guy who scraped all HN job postings and made his own job site with them). From what I can tell, it's not actually part of this site's philosophy to be a building block for other people's software ideas.
- bravura 16y agoAre you against http://searchyc.com http://searchyc.com? Are you against Gabriel Weinberg's Ask YC archive (http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive http://www.gabrielweinberg.com/startupswiki/Ask_YC_Archive)? Would you like it less if he built it automatically instead of manually (cf. http://metaoptimize.com/projects/autotag/hackernews/ http://metaoptimize.com/projects/autotag/hackernews/)? I agree that the job posting site was really tacky, but that guy entered all the data manually anyhow. I think in sum total that most add-ons are pointless. A few are useless. And only a very small fraction are bad, and should be dealt with on a case-by-case basis.
- tptacek 16y agoI like SearchYC. I've never used the Ask YC archive, but I trust Gabriel Weinberg. I'm not advocating an end to all add-on apps. I'm saying that there is no principle that animates Hacker News that requires Graham to bend over backwards to accomodate APIs. It's a simple, low-drama point.
- davewiner 16y agoYou misunderstood what I was saying. Start with the firehose. I don't want it to make money, I want it so I can link this flow in with other flows that I'm following without having to visit all the sites. Not trying to make a business, just scratching my own itch having no idea where it leads.
- mikeryan 16y ago
- nym 16y agoI think it's interesting that Dave Winer decided to do this as a blog post instead of a HN discussion. Instead of having a conversation with the HN community he decided to have a conversation with his readers.
- davewiner 16y agoThat's not what I "decided" at all. I do my writing on my blog. That's the way I work. Happy to participate in a discussion here.
- nym 16y agoThat's still a decision.
- kyro 16y agoWhat is your point?
- spage 16y agoI believe Dave Winer has always advocated writing on your own space and linking into the conversation. I feel this is very insightful advice and pro-web and really "getting it". I didn't get a chance yet to see what experiment Dave was building, his firehose, because scripting.com is down. But I'd give Dave the benefit of the doubt...
- daxelrod 16y agoHuh? He's also the one who posted this to HN. It seems to me like he wanted to have conversations with both the HN community and his readers.
- bryanlarsen 16y agoIt would be stranger if he didn't post it on his blog. After all his blog has been around for almost 10 years longer than hacker news, was around before they were called blogs. Dave was also the guy that invented RSS.
- ElbertF 16y agoThe page doesn't load for me, Google Cache got it: http://webcache.googleusercontent.com/search?q=cache:http://scripting.com/stories/2010/11/08/myHackerNewsFirehose.html&hl=en&strip=1 http://webcache.googleusercontent.com/search?q=cache:http://...
- elq 16y agoWhy is it that every time I visit scripting.com (about twice per year), I'm met with niggling little issues like reload loops or the lovely javascript link to the loopback address - "http://127.0.0.1:5337/scripting2/editor/controls?username=davewiner&url=http://scripting.com/stories/2010/11/08/myHackerNewsFirehose.html http://127.0.0.1:5337/scripting2/editor/controls?username=da... I think Dave needs a new tool.
- davewiner 16y agoJust curious, does that link cause you any problems in your browser? If so, what browser and on what platform?
- elq 16y agoYes. Caused both firefox 3.6.12 and safari 5.0.2 on osx to have fits. firefox on ubuntu and chrome seem to work without trouble.
- davewiner 16y agoSorry about that. What do the fits look like? How much can they penalize you for opening a web page that has (what amounts to) a broken link? BTW, I use Firefox on the Mac, on machines that don't have my CMS running on it (that's what the link connects to) without problems.
- mbeihoffer 16y agoOn my machine they look like a macro-lens closeup of a sunflower in full bloom, or alternately, like an Archimedean Spiral or whatever.
- itsnotvalid 16y agoAs we are seeing in here there are numerous APIs available (like the one made available via a torrent download, twitter bots, searching, or alternative look & feel) which they must have been doing their own parsing of this site. Would that actually be good to have a official API for all sorts of hacking needs?
- FluidDjango 16y agoRegarding the "API blocked"... Seems to be feeding fine as of Mon eve (Nov 8)
- scott_s 16y agoIn the past, clever things that others have done have been blocked because they placed a substantial burden on the servers.
- alttab 16y agoWhat all of these posts fail to consider is that philosophically pg may want you writing code instead of consuming his community 101 different ways. Just my 02. I visit hn daily, but I'm busy enough on my own work that the website suits my consumption needs. Anyone else like me out there?
- alttab 16y agoI almost expected to get down-voted. What I was trying to convey is managing your time wisely.
- lwhi 16y agoWhat incentive does Hacker News have to allow access via an API? At the moment, introducing an API would cost YC money because it would take time to develop and it would potentially increase server costs. However, allowing others to do the job seems less troublesome. Creating artificial restrictions to _prevent_ other people from doing so, seems like effort. I would imagine that the owners of a site would only seek to stem the wholesale flow of information to another property, when inaction is likely to lead to a loss in value. Where is the value in HN? It could be argued that the value lies in the posts and the related combined wisdom, but I think the true value is related to attention and focus. We come to HN to participate. If the content is allowed to spread to third-party applications (via an API), this primary source of value is lost.
- davewiner 16y agoHow many Ycombinator startups build on the APIs of other sites? Wouldn't it be reasonable to expect them to reciprocate?
- lwhi 16y agoIt would be nice to expect HN to reciprocate, but I don't think that it's likely. API access always needs to be part of a strategy, because granting it will change the landscape within which your business operates.
- nir 16y agoI'm curious about Giles Bowkett's comment on that story - is he really blocked on HN? If so, why?
- allenbrunson 16y agoyep, he really is blocked. based on his comments, i'm guessing the reason is because he wasn't able to remain civil and respectful. http://news.ycombinator.com/user?id=gilesgoatboy http://news.ycombinator.com/user?id=gilesgoatboy that's just the first one i found, i'm sure there are others.
- blasdel 16y agoThere were other accounts too that are still active but that he may be locked out of. That particular account got silently hellbanned for sarcasm: http://news.ycombinator.com/item?id=551664 http://news.ycombinator.com/item?id=551664 Some historical threads: http://news.ycombinator.com/item?id=196390 http://news.ycombinator.com/item?id=196390 http://news.ycombinator.com/item?id=1015591 http://news.ycombinator.com/item?id=1015591
- nir 16y agoHaving read the comments there, I don't see why they had him banned. Perhaps a few were offensive, but nothing the community can't handle. I prefer reading someone like this to safe comments that just recycle bits of PG's essays. HN could use some irreverence.
- adrianwaj 16y agoThe best way to get around the ban I think is to have low-latency distributed scraping. Basically, a plugin that submits to RR's site whenever a user is surfing on HN. RR's site then parses it as usual. The plugin could have some setting like max KB submitted per minute, and RR's site would only request the full page if it was needed at the time.
- jrockway 16y agoI think the idea behind HN is to have a place where you can go to chat when you are bored or feeling distractable. Having every story posted as its submitted is counterproductive -- it makes HN become a job (like reading Email) instead of something to do once in a while when you feel like it. I miss probably about 90% of the stories on HN. I enjoy it anyway. HN is a place to go, not a list of articles.
- aditya 16y agoAm I the only one who wants the firehose and the API to come back? As long as it can be done without taking the servers down and killing HN, that is :-)
- mike-cardwell 16y agoFWIW, either your feed, or the feed you're pulling from is not correctly handling encoding. "Announcing Brightbox Cloud â the UKu0027s first true cloud hosting platform"
- davewiner 16y agoUpdate: the firehose feed is working again. http://scripting.com/stories/2010/11/09/myHackerNewsFirehoseIsFlow.html http://scripting.com/stories/2010/11/09/myHackerNewsFirehose...