4 ms·
Local archives are great, but they help only you. Please consider having your archiving scripts/services store their content in WARC format so you can submit b
by vitovito 13y ago
Local archives are great, but they help only you.
Please consider having your archiving scripts/services store their content in WARC format so you can submit bundles to the Internet Archive for integration into the Wayback Machine.
That's how Archive Team's downloads are able to be integrated. The latest versions of wget support storing their downloads in this fashion.
You could even regularly spider your own site and package it up into a WARC for submission.
- gwern 13y agoI don't know how to usefully create WARCs. Wget has a WARC option, but when I tried it out, it created weirdly named files that littered the www tree and looked like the names would collide and files be overwritten. Plus the IA live request should be handling getting webpages into IA.
- Asparagirl 13y agoHere, have a Gist for creating proper WARC's, ready for the Internet Archive: https://gist.github.com/Asparagirl/6202872 https://gist.github.com/Asparagirl/6202872 And a Gist for uploading the completed WARC's to the IA using their s3-like service: https://gist.github.com/Asparagirl/6206247 https://gist.github.com/Asparagirl/6206247 Final step is to e-mail someone at Archive Team with admin rights to move your IA upload into the proper "Archive Team" bucket, instead of "Community Texts". The awesome Mr. Jason Scott should be able to help you with that.