3 ms·

Question: OK, so I know how to test — manually — that a few randomly chosen files are correctly backed up in my backup systems. But what if there are larger c

Question: OK, so I know how to test — manually — that a few randomly chosen files are correctly backed up in my backup systems. But what if there are larger classes of systemic error in the process I haven't thought of testing? What if some particular file type or directory tree has vanished from the backups, but since it isn't in my manual testing process, I never catch it? Are there best practices for validating that whole directory trees are correctly backed up across the board? Or any form of automated testing for backups (but, presumably, separate from the software that does the backup process)?

rhinoceraptor 7y ago
I would just use ZFS snapshots and replication. They're read-only, and they only use as much space as the difference between the current state and the snapshot. Plus, you'll have solid data integrity instead of relying on buggy firmware in your RAID controller.
- tracker1 7y ago
  And there's no way to compress/eliminate snapshots? Also, redundancy and snapshots are not a backup.
magashna 7y ago
Doing some DR testing is the only way to truly know
benjohnson 7y ago
For key servers, we've been known to clone the entire drive and put the clone in service. This helped us with several problems: Backup system didn't record the permissions. Backup didn't get 'resource forks' or 'alternate data streams' Backup wasn't complete enough to satisfy copy protection for software that was no longer supported. Backup process didn't see deep into the folder because there was a hardlink/symlink loop that caused it to stop Backup didn't get a coherent copy of a bunch of files that was acting like a database Drive had a CRC error that let it keep running but stopped the backup system from reading the file Windows program stored vale in the case of the file name while the Unix backup command assumed it could squish the Windows files to lowercase.

rhinoceraptor 7y ago

I would just use ZFS snapshots and replication. They're read-only, and they only use as much space as the difference between the current state and the snapshot. Plus, you'll have solid data integrity instead of relying on buggy firmware in your RAID controller.

tracker1 7y ago
And there's no way to compress/eliminate snapshots? Also, redundancy and snapshots are not a backup.

magashna 7y ago

Doing some DR testing is the only way to truly know

benjohnson 7y ago

For key servers, we've been known to clone the entire drive and put the clone in service. This helped us with several problems: Backup system didn't record the permissions. Backup didn't get 'resource forks' or 'alternate data streams' Backup wasn't complete enough to satisfy copy protection for software that was no longer supported. Backup process didn't see deep into the folder because there was a hardlink/symlink loop that caused it to stop Backup didn't get a coherent copy of a bunch of files that was acting like a database Drive had a CRC error that let it keep running but stopped the backup system from reading the file Windows program stored vale in the case of the file name while the Unix backup command assumed it could squish the Windows files to lowercase.