3 ms·
Surprisingly it doesn't mention HN itself which is a treasure trove of data. I know there is APIs to download HN content but is there a permanent location for H
by sytelus 12y ago
Surprisingly it doesn't mention HN itself which is a treasure trove of data. I know there is APIs to download HN content but is there a permanent location for HN data dump (like StackOverflow do their data dump on Internet Archive)? This is a great article, BTW, anyone who wants some cool projects to do in data mining and machine learning.
- deleted 12y ago[deleted]
- j2kun 12y agoI currently have a subset of one year's worth of all HN stories and comment trees, organized by story, but it's on my local machine. Where is a good place to post it? It's quite big, on the order of multiple GB. The problem (if you want an easy scraper) is that the HN API limits you to 1k requests per hour. So it took me about 10 days of continuous running and restarting because of random crashes to get all the data.