4 ms·
HNSearch has a limitation of 1000 maximum results returned (see [1]). You can take a snapshot of the Front Page / New Posts every so often a la [2], but it's l
by minimaxir 13y ago
HNSearch has a limitation of 1000 maximum results returned (see [1]).
You can take a snapshot of the Front Page / New Posts every so often a la [2], but it's limited per the robots.txt. It's possible, but more time intensive (as in the linked post)
[1]: http://api.thriftdb.com/api.hnsearch.com/items/_search?q=medium.com&limit=100&filter[fields][domain]=medium.com&start=1000 http://api.thriftdb.com/api.hnsearch.com/items/_search?q=med...
[2]: http://mayank.lahiri.me/writing/hackernews/index.html http://mayank.lahiri.me/writing/hackernews/index.html
- jaredsohn 13y agoThanks for the information (including the links.) Based on other comments on this post, it looks like HNSearch and Reddit APIs have the same limitations, so doing something like this should be feasible for Hacker News. Also, another post indicates that the data for Reddit was done with only single 1000-limit queries, so the time-intensive method shouldn't be needed to produce equivalent results (although it could provide more accurate results.) Finally, for those looking for something like this for Hacker News right now, check out these sites: http://www.hntrends.com/ http://www.hntrends.com/, http://hntrends.jerodsanto.net/ http://hntrends.jerodsanto.net/, Hacker News hiring trends (https://github.com/adamw523/hackernewshires https://github.com/adamw523/hackernewshires)