3 ms·
I tried doing something like this a while ago. But HN kept blocking my IP after a few days of screen scraping. How did you get past that rate limit?
by jabo 14y ago
I tried doing something like this a while ago. But HN kept blocking my IP after a few days of screen scraping. How did you get past that rate limit?
- marcneuwirth 14y agoI am using the bigrss feed as described in the latest post: http://ycombinator.com/newsnews.html http://ycombinator.com/newsnews.html
- cooldeal 14y agoI wish it gave out the points the article had... does your data track that or just the position?
- marcneuwirth 14y agoJust the position unfortunately, I would need to scrape the site itself in order to get comments and points info. It would be awesome if that data was added to the RSS feed
- peteretep 14y agoWere you hitting it more than once every 30 seconds?
- jabo 14y agoNo, I hit it once in 15 minutes!