4 ms·
Side note: "xz -9 -e" compresses the file to 3,296,864 where as "lz5 -15" only compresses the original file to 4,643,261 bytes. The xz compressed file is 29% sm
by jftuga 10y ago
Side note: "xz -9 -e" compresses the file to 3,296,864 where as "lz5 -15" only compresses the original file to 4,643,261 bytes. The xz compressed file is 29% smaller.
Even "gzip -9" compresses the file to 4,035,858.
So I wonder why lz5 was chosen for compression.
- jlgaddis 10y agoHow long did that "xz -9 -e" take compared to "lz5 -15" or "gzip -9"?
- mappu 10y agoNumbers on my machine, ordered by compression duration: Compressor Duration Size cat 0.0s 11385584 [*] gzip -1 0.2s 4918778 [*] bzip2 -1 0.9s 3334653 bzip2 -9 0.9s 3122347 [*] xz -1 1.1s 4222016 gzip -9 2.4s 4085818 lz5 -15 6.9s 4643261 xz -9 -e 10.7s 4033589 zpaq -m5 34.2s 2655834 [*] A [*] indicates that no better compressor was faster. Test methodology was `cat | $compressor > output` run 3-4 times to get an average. I'm surprised by how bzip2 turned out.
- mappu 10y agoThere's more low-hanging fruit before picking a different compressor, too. For instance, sorting the data: $ zcat openftp4_all_20160918.gz | sort | gzip -9 > sorted.gz $ ls -lha *.gz -rw-r--r--+ 1 mappu mappu 3.9M Sep 18 18:17 openftp4_all_20160918.gz -rw-r--r--+ 1 mappu mappu 2.2M Sep 18 18:19 sorted.gz `zpaq` should out-compress xz on the unsorted file, too, but i haven't tried it.