3 ms·

Sorry to hear your friend did not. Parity files are imo an wildly underrated tech for long-term storage and something that I currently use for all long-term sto

Sorry to hear your friend did not. Parity files are imo an wildly underrated tech for long-term storage and something that I currently use for all long-term storage. I’ve even been known to use 1% if I’m relying on ZFS/RAID and just want to “trust but verify”. Off-site and multi-device backups are still vital for anything that’s important enough but as you mention multiple copies does not remove the need to test the media. For anyone who doesn’t know already: they got popular (and probably created) for binary files posted to usenet. Usenet was a high-speed and low-completion platform where files got corrupted all the time. PARity files took the algorithm for RAID and created a new file that could live “next to” the binary files. If you had a 1GB file and 300MB of PAR/PAR2 files could could recover the complete original file even if 30% was missing or corrupted. It still works today on Windows/macOS/Linux.

dspillett 3y ago
> Parity files are imo an wildly underrated tech for long-term storage and something that I currently use for all long-term storage. As a not to others: if following this method for archive purposes, be careful how you are distributing these parity files and those that they relate to. If all your files are on the same physical disk/tape the one physical disk/tape being unreadable kills the whole lot so you are only protecting yourself against errors in individual blocks. To protect against more complete media failures (i.e. a CD/DVD being unreadable) you need to split the data over several disks/tapes. This matches how the system was used on usenet, where each post (of potentially many tens or hundreds making up a large binary post) was essentially a separate bit of storage media. At this point it may just be easier to have multiple copies, though note that even with multiple copies the parity information can be useful as you can detect bit and block level corruption and in many cases correct it (whereas with two copies you can detect corruption, as the copies no longer match, but have no reliable way of knowing which copy, if any, is correct).

dspillett 3y ago

> Parity files are imo an wildly underrated tech for long-term storage and something that I currently use for all long-term storage. As a not to others: if following this method for archive purposes, be careful how you are distributing these parity files and those that they relate to. If all your files are on the same physical disk/tape the one physical disk/tape being unreadable kills the whole lot so you are only protecting yourself against errors in individual blocks. To protect against more complete media failures (i.e. a CD/DVD being unreadable) you need to split the data over several disks/tapes. This matches how the system was used on usenet, where each post (of potentially many tens or hundreds making up a large binary post) was essentially a separate bit of storage media. At this point it may just be easier to have multiple copies, though note that even with multiple copies the parity information can be useful as you can detect bit and block level corruption and in many cases correct it (whereas with two copies you can detect corruption, as the copies no longer match, but have no reliable way of knowing which copy, if any, is correct).