6 ms·
Ask HN: Theory of Backups
Most discussions of backups are focused on the consumerist matters.
There seems to exist, buried underneath the superficial & the common sense, theory on how to do backups well.
I've found two elements upon which better theory concerning rotatioms & other details (EG hash verification scheduling, amount of different devices) can be built.
The first is the Tower of Hanoi scheduling scheme, which we will abbreviate TOH.
The second is the Incremental-Differential-Full backups concept, which we will abbreviate IDF.
The best available resource seems to be the Acronis websites' illustrated docs: http://acronis-backup-recovery.helpmax.net/en/understanding-acronis-backup-recovery-10/tower-of-hanoi-backup-scheme/. I request that you in good faith ignore Acronis is a company selling commercial Windows software; you are free to post better links in comments were you to find better info elsewhere.
We end up with a scheme we can call IDF-TOH. In it we have three types of backups:
- Incremental, at L_0 ("Level A" in the linked resource), the most frequent level, capturing only changes made since the last backup.
- Differential, at each level that belongs to the closed interval L_0-L_n, capturing changes made since the last full backup.
- Full, at L_n, the least frequent level, capturing the whole system to be backed up.
So now, what can we do? At least the following directions could be taken in further developing a Theory of Backups:
0 (backup scheduling): The frequencies can be chosen in many ways, and I am not sure which one is most optimal. Tower of Hanoi is for every level L_a, where a belongs to closed interval 0 ... n, 2^a. Frame-Steward may or may not be of any use in this.
1 (rotations): IDF-TOH does not address the problems of rotations. IE: if you make a backup that corrupts your previous data, and then repeat the mistake, you get in trouble quick. It's ALSO noteworthy that certain mediums may better fit certain layers in IDF-TOH & the future schemes. At least, for example, adrian_b three days before this wrote:
"... Of all the optical discs that have been available commercially, those with the longest archival time were the pressed CD-ROM with gold mirrors, where the only degradation mechanism is the depolymerization of the polycarbonate, which could make them fragile, but when kept at reasonable temperatures and humidities that should require many centuries..."
Consequently these would be the best for the Full backups, while Solid State Drives may work for the Incremental ones.
2 (perfecting IDF): The IDF scheme may not be perfect either & can probably be refined more or less.
3 (hashing): Verifying the backups matters & should be a part of a complete scheme.
---
This may not be valuable for all businesses but most invididuals already using rsync or borg would probably prefer to use the best available scheme if reduces probability of incidental data loss at minimal effort. The task of translating the best possible scheme to a config program with humane interface is an undertaking of its own.
- eschneider 2y agoRule 0 for backups: Whatever scheme you use, periodically verify your backups by restoring some files off them. Before you have a problem. You'll thank yourself eventually.
- Xen9 2y agoThis is actually what I mean by that lots of "discussion" is repeating common sense hymns.
- randomlurking 2y agoI find your post quite interesting - at the same time I believe that one reason those hymns are repeated so often is that for the majority of businesses, 99% of the values lies in simply following them and not thinking too hard about theories etc. there will often be a relatively clear path forward prescribed by the software used, know how, or (where no software is used) ease of implementation.
- Xen9 2y agoThis is precisely the reason why a theory would be valuable. The businesses can have use Legendary Backup™ for Maximal Data Safety Certifications by Us.
- illuminant 2y agoBackups are like prayers, only if you make backups you won't need prayers.
- scrapheap 2y agoJust wait till your company depends on restoring that backup :D
- mikewarot 2y agoMy practically minded friend has an interesting scheme for backups. He uses clonezilla or similar products to clone the existing drives of a machine to new replacements. Then puts the old ones in a safe place. On a periodic basis. The other normal backups are usually managed by someone else, he just does the hardware, most of the time. His backups are tested by experience.