3 ms·
You can do much the same thing with less work using the Tarsnap tool and service. Rather than incremental backup, blocks are deduplicated on the client and onl
by loeg 9y ago
You can do much the same thing with less work using the Tarsnap tool and service. Rather than incremental backup, blocks are deduplicated on the client and only unique blocks are backed up. This has about the same storage efficiency as incremental backups, but has the benefit of not relying upon "full" backup plus incremental diffs to achieve the final snapshot contents.
(I have no affiliation with Tarsnap other than Colin seems like a nice guy and I am a customer.)
I have full backups configured every 12 hours. Just for example, on my last backup, the total logical size was 12 GB; the compressed size of the undeduplicated blocks was 7.2 GB; the total size of actual new data, uncompressed, was 181 MB; and the final sum uploaded for this new full backup was 72 MB. Logically, Tarsnap stores 2.9 TB of my backups, but after compression and deduplication the "physical" requirement is only 16 GB.
For this I pay about 17¢ USD/day, or $62/year. I could probably try to lower my use storage use somewhat (the largest component of that cost, 13.4¢/day) but it hasn't been worth my time yet.
- raphinou 9y agoTarsnap is really slow to restore big files. Just saying in the hope it can help some people.
- mey 9y agoIt's slow to restore smaller files as well. It is certainly not near-line disaster recovery. Conversely, it has already saved my bacon a few times and I trust it's design.
- Scaevolus 9y agoAs a tar-based backup system without a separate index, restoring a single file is approximately as expensive as restoring the entire backup. Systems that split metadata and data streams do a lot better (restic, attic, borg, bup).
- cperciva 9y agoErr... Tarsnap does split metadata and data.