3 ms·
One of the great things about gz archives is that the --rsyncable flag can be used to create archives that can be rsynced efficiently if they change only slight
by gizmo 11y ago
One of the great things about gz archives is that the --rsyncable flag can be used to create archives that can be rsynced efficiently if they change only slightly, such as sqldumps and logfiles. Basically the file is cut into a bunch of chunks, and each chunk is compressed independently of the other chunks. xz doesn't seem to have an equivalent feature because the standard implentation isn't deterministic[1].
Changing from one compression format to another seems harmless, but it always pays to think carefully about the implications.
[1]: https://www.freebsd.org/cgi/man.cgi?query=xz&sektion=1&manpath=FreeBSD+8.3-RELEASE https://www.freebsd.org/cgi/man.cgi?query=xz&sektion=1&manpa...
- mappu 11y agoThe `--rsyncable` patch never got upstreamed, and in recent debian the feature is totally broken (rsync needs to transmit ~100% of the file again). `pigz` has a similar flag that works reliably, though.
- kevinoid 11y agoAre you referring to https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=708423 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=708423 ? If so, it was fixed in December 2013 and doesn't affect the current Debian release (Jessie), although it does affect the previous release (Wheezy) and there is an open request to backport the fix https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=781496 https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=781496
- aktau 11y agoYep. I wrote a post about it, also doing some comparisons: http://www.aktau.be/2014/10/23/pg-dump-and-pigz-easy-rsyncable-backups-with-postgresql/ http://www.aktau.be/2014/10/23/pg-dump-and-pigz-easy-rsyncab...