3 ms·
The published results aren't consistent with the general understanding that xz is faster than bzip2 while compressing better even at the lowest settings. Xz be
by pengy 11y ago
The published results aren't consistent with the general understanding that xz is faster than bzip2 while compressing better even at the lowest settings.
Xz beyond the lowest compression setting quickly enters the realm of diminishing returns, but when even the lowest setting compresses better than bzip2 and is faster in both compression and decompression, there is no reason to use bzip2.
The opencpu post completely ignores the compression setting dimension, presenting an incomplete picture. Xz is shown as consistently the slowest by a wide margin, even relative to bzip2. This is unexpected, xz will be faster in both compression and decompression than bzip2 when using a compression setting appropriate for comparison with bzip2 compression ratios.
- rwmj 11y agoAlso xz upstream defaults to single threaded. However xz decompression scales pretty much linearly at least up to 4 cores - I wrote a parallel pxzcat to prove that: http://git.annexia.org/?p=pxzcat.git;a=summary http://git.annexia.org/?p=pxzcat.git;a=summary
- jacquesm 11y ago> Xz beyond the lowest compression setting quickly enters the realm of diminishing returns, but when even the lowest setting compresses better than bzip2 and is faster in both compression and decompression, there is no reason to use bzip2. Reasons may revolve around other things than compression quality and speed. For instance, it may be that a company has a history of storing archives using a particular archive format. Another would be a desire to communicate with other entities using a format both parties can use.
- datajeroen 11y agoThis blog is intended for the audience of the R programming language. The benchmarks are comparing compression techniques available in R with their default settings, rather than giving a comprehensive review of compression methodology.