4 ms·
"Perhaps one minor bonus to HackerSlide for now is anyone can take the data collected." Indeed, may I make a suggestion here: 1) Why not make a datadump so pe
by recuter 14y ago
"Perhaps one minor bonus to HackerSlide for now is anyone can take the data collected."
Indeed, may I make a suggestion here:
1) Why not make a datadump so people wouldn't need to scrape ~800*24 json files individually?
2) OP ought to load this data into his version so the timeline goes back further
3) It seems quite a few people get the urge to tinker like this with HN, I'm sure pg doesn't mind the scraping, but it strikes me as vastly more efficient if some sort of shared resource was setup and perhaps added to the footer, in the vain of HNsearch, so people don't waste time get crawling data setup.
I'm sure somebody else has a dataset just like yours that goes back further still. :)
Also, thank you for making this and OP for making his. Fun.
- petercooper 14y ago1) Why not make a datadump so people wouldn't need to scrape ~80024 json files individually?* Publicly available JSON files was just a side effect of the implementation. But it's easily to tar and gzip it up, so there's now such a file at http://secretshenanigans.s3.amazonaws.com/hnfrontpages.tar.gz http://secretshenanigans.s3.amazonaws.com/hnfrontpages.tar.g... (32MB).