3 ms·
You wouldn't even need to mine it - https://github.com/HackerNews/API https://github.com/HackerNews/API
by chipperyman573 9y ago
You wouldn't even need to mine it - https://github.com/HackerNews/API https://github.com/HackerNews/API
- applecrazy 9y agoSorry, I meant analyze. I actually am writing a blog post right now on this (this thread was very inspiring). Should be up in a day or two at https://applecrazy.github.io/blog https://applecrazy.github.io/blog
- Majora320 9y agoGood luck downloading 16 million records...
- minimaxir 9y agoYou could do that with the Algolia API: https://github.com/minimaxir/get-all-hacker-news-submissions-comments https://github.com/minimaxir/get-all-hacker-news-submissions... But as noted, BigQuery is more pragmatic.
- kuschku 9y agoWhy? 16 million isn’t much, maybe 10 to 20 gigabyte if compressed. I’ve seen IRC bouncers that have more messages stored for a single user (270 million, in fact).
- abhishekjha 9y agoI have a script running on aws right now.
- fhoffa 9y agoEven easier: All Hacker News posts are ready to be analyzed in BigQuery. - https://medium.com/@hoffa/hacker-news-on-bigquery-now-with-daily-updates-so-what-are-the-top-domains-963d3c68b2e2 https://medium.com/@hoffa/hacker-news-on-bigquery-now-with-d...