3 ms·
Does anyone know how much a text-only mirror of the IA is? I would like to keep such a mirror, but I don't want to store the 57 Petabytes it is right now https:
by renewiltord 3y ago
Does anyone know how much a text-only mirror of the IA is? I would like to keep such a mirror, but I don't want to store the 57 Petabytes it is right now https://wiki.archiveteam.org/index.php/Internet_Archive#Raw_Numbers https://wiki.archiveteam.org/index.php/Internet_Archive#Raw_...
The IA's rate limits are quite aggressive (rightly, I suppose) and so I would like to scan my own text-mirror for the precise point I want and then I can click through to the IA's full wayback machine.
$10/TiB is about what I'd expect for storage drives and perhaps with the right indexing scheme I can spend $10k or less and have a multi-level cache using overlayfs with goofys or some such equivalent.