4 ms·

I’m curious if anyone has any good articles on the Internet Archive’s backup procedure?

• dmd 3y ago
  There isn't a backup procedure. If something were to happen to the primary in San Francisco[0], that's it. There have been many attempts to crowdsource the backup (e.g. http://iabak.archiveteam.org/ http://iabak.archiveteam.org/, now defunct), but none have approached backing up even a tiny fraction. The IA has been approached several times by companies offering to provide a backup location, but has rebuffed them on the grounds that they want any backup to be community-owned-and-maintained. Which I understand, but this is classic "perfect is the enemy of the good" behavior. [0] Fortunately, SF isn't known for large scale natural disasters.
  • krackers 3y ago
    >SF isn't known for large scale natural disasters. Except earthquakes?
  • O1111OOO 3y ago
    > If something were to happen to the primary in San Francisco[0], that's it. Makes me sad, as I'm sure many others. I turned up the following about their Petabox (raw numbers thru Dec2021): * Wayback Machine: 57 PetaBytes * Books/Music/Video Collections: 42 PetaBytes * Unique data: 99 PetaBytes * Total used storage: 212 PetaBytes * Local computing to process the data (800 low-end PC's) * Co-location friendly * now houses *about* 3 Petaboxes Not 100% clear but I seems like each Petabox holds between 80PBs and 100PBs of data. Also... I'm seeing locations in Amsterdam and San Francisco (see History, bottom of sourced page)...? But this could just be old/original info (2004) and everything is in SF now. Source: https://archive.org/web/petabox.php https://archive.org/web/petabox.php