4 ms·
Good luck downloading 16 million records...
by Majora320 9y ago
Good luck downloading 16 million records...
- minimaxir 9y agoYou could do that with the Algolia API: https://github.com/minimaxir/get-all-hacker-news-submissions-comments https://github.com/minimaxir/get-all-hacker-news-submissions... But as noted, BigQuery is more pragmatic.
- kuschku 9y agoWhy? 16 million isn’t much, maybe 10 to 20 gigabyte if compressed. I’ve seen IRC bouncers that have more messages stored for a single user (270 million, in fact).
- abhishekjha 9y agoI have a script running on aws right now.