3 ms·
I think preservation should be verifiable. ZFS, AWS S3, etc are great and should be used, but not relied upon to prevent data corruption. That should be a separ
by adultSwim 5y ago
I think preservation should be verifiable. ZFS, AWS S3, etc are great and should be used, but not relied upon to prevent data corruption. That should be a separate step on top of the storage layer.
We can also do better than hashing by using error correcting codes. If a file doesn't match a stored hash, you have to check other backups to see if the file went bad or if the hash went bad. ECC can do this without checking backups and has the ability to "heal" a damaged file (see Reed-Solomon, which is used in RAID 5/6).
Parchive/par2 is an off the shelf replacement/supplement for checksums.