4 ms·
There's an official API that's much easier to work with and doesn't have aggressive precautions in place - http://hnsearch.com/api http://hnsearch.com/api
by benologist 13y ago
There's an official API that's much easier to work with and doesn't have aggressive precautions in place - http://hnsearch.com/api http://hnsearch.com/api
- minimaxir 13y agoBTW: there is a limit of 1000 results per search.
- benologist 13y agoYou can mitigate that by spidering users and searching domains to build a more complete database - http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][type]=submission&filter[fields][username]=minimaxir http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... http://api.thriftdb.com/api.hnsearch.com/items/_search?filter[fields][type]=submission&q=arstechnica.com http://api.thriftdb.com/api.hnsearch.com/items/_search?filte... I wrote a small NodeJS spider if anyone's interested - https://github.com/benlowry/hnsubmitterstats https://github.com/benlowry/hnsubmitterstats
- minimaxir 13y agoCoincidentally, I was looking into that option as well, and that seems perfect for my needs :) thanks!
- jkarneges 13y agoAh yes, I saw this. According to a post on their forums, the Thrift DB runs about 15 minutes behind HN itself. Not terribly great for my particular application (that checks for updates), but no doubt useful for other things.
- xauronx 13y agoI wouldn't necessarily say it's much easier to work with. Could not manage to pull down anything representing the front page, and from what I could tell from a lot of searching, no one else could either. So yes, you have access to the data but for 99% of uses it's either too old or not representative of what you would see in a browser. Awesome resource for querying + running analytics, but for an HN client, not as much.