3 ms·
Ask HN: How to backup HN
A question provoked by Github: what's the best way to maintain a relatively current offline copy of all HN submissions and comments? How does Algolia get access to recent HN content?
Related question, is anyone pushing HN submisssion links to archive.org (or similar) so we have a record of the page at the time it was submitted to HN?
- minimaxir 8y agoAll HN data is unofficially archived on BigQuery: https://cloud.google.com/bigquery/public-data/hacker-news https://cloud.google.com/bigquery/public-data/hacker-news
- natch 8y agoThat's good information but I hope others have more complete answers to the OP's questions. There are a few problems with BigQuery being the only solution (you didn't say that's the case, but one has to wonder). The problems include cost, vulnerability to shutdown, potential rate limiting, to name a few.