4 ms·
Of course not. They have their own crawler (Heritrix, an open source Java crawler) and archive in WARC format. It‘s serious archiving, they want to preserve rep
by Tomte 2y ago
Of course not. They have their own crawler (Heritrix, an open source Java crawler) and archive in WARC format. It‘s serious archiving, they want to preserve reply codes, HTTP headers etc.