Also, please use a sensible format if you're crawling/archiving this. We're using WARC (Web Archive) which is an official ISO File Format standard - which the

Also, please use a sensible format if you're crawling/archiving this. We're using WARC (Web Archive) which is an official ISO File Format standard - which the Internet Archive's Wayback Machine can use. It's also a pretty good and nice format for archiving web pages in general.