5 ms·
If it becomes a big deal, and you can't get the data from ThriftDB for some reason, I've got a copy of HN data (submission & comment content, user, points, item
by thaumaturgy 13y ago
If it becomes a big deal, and you can't get the data from ThriftDB for some reason, I've got a copy of HN data (submission & comment content, user, points, item date) up to id 7018491, a comment by kashkhan, time-stamped 2014-01-05 23:56:34.
edit: Ack, I just realized that item_id got reset back to 7015126 on the reboot. My data matches HN up to 7015125, and then diverges after that.
- emillon 13y agoCan you make it available somewhere? I'd be interested in that data, and sure that others would be too. Thanks!
- thaumaturgy 13y agoSure. This is very temporary, I'll be removing the links sometime tomorrow: http://www.associatedtechs.com/tmp/hn_submissions_7015126.sql http://www.associatedtechs.com/tmp/hn_submissions_7015126.sq... http://www.associatedtechs.com/tmp/hn_comments_7015126.sql http://www.associatedtechs.com/tmp/hn_comments_7015126.sql After a semi-random sampling, the comments file appears to contain nothing but comments pre-crash. The submissions however got clobbered a little by the crawler at some point. There are some submissions in there pre-crash and some post-crash; I think everything's OK from 7015172 on, which only leaves 15 possibly damaged rows, and of those, I'd expect most of them didn't have id collisions. Sorting out the old stuff from the new stuff could be manually done. (Please let me know if there's anything I should be concerned about in those, or if they shouldn't be posted for some reason, or something. I'm recovering from flu and am still not entirely all here.)
- emillon 13y agoThanks!