You can scrape HN as long as you respect the robots.txt[1] and don't retreive more than a couple of pages per minute. Have you considered just pulling the data

You can scrape HN as long as you respect the robots.txt[1] and don't retreive more than a couple of pages per minute. Have you considered just pulling the data from HNSearch's API[2] or the one by iHackerNews[3]? [1] https://news.ycombinator.com/robots.txt https://news.ycombinator.com/robots.txt [2] https://www.hnsearch.com/api https://www.hnsearch.com/api [3] http://api.ihackernews.com/ http://api.ihackernews.com/