3 ms·
It does compress slightly faster and better than lz4 (when the output is written to disk on a SSD) using the default parameters: For an input text file alpha.t
by ogrisel 14y ago
It does compress slightly faster and better than lz4 (when the output is written to disk on a SSD) using the default parameters:
For an input text file alpha.txt whose size is 3.3GB:
- fastlz compresses in 1m6.446s (real time)
$ time ./6pack ~/Desktop/alpha/alpha.txt /tmp/out.fastlz
alpha.txt
[##################################################] 43.4% saved
real 1m6.446s
user 0m26.482s
sys 0m8.346s
- lz4 compresses in 1m15.681s (real time)
$ time ./LZ4Demo.exe ~/Desktop/alpha/alpha.txt /tmp/out.lz4
*** Compression CLI using LZ4 algorithm , by Yann Collet (Jul 25 2012) ***
Compressed 3590403790 bytes into 2418583396 bytes ==> 67.36%
Done in 35.99 s ==> 95.13 MB/s
real 1m15.681s
user 0m23.081s
sys 0m12.927s
Here is the size of the output:
1.9G /tmp/out.fastlz
2.3G /tmp/out.lz4
lz4 is here: http://code.google.com/p/lz4 http://code.google.com/p/lz4
alpha.txt was generated as advertised on this website: http://leon.bottou.org/projects/sgd http://leon.bottou.org/projects/sgd
Both projects were compiled from the trunk on a 2009 MacBook Pro running OSX 10.6 with gcc i686-apple-darwin10-gcc-4.2.1.
I will now make some room on my SSD to time uncompress.
Edit: I rerun the previous compression and the results are pretty stable: +/- 2s
- ogrisel 14y agoHere are the decompression timings: $ time ./6unpack /tmp/out.fastlz Archive: /tmp/out.fastlz alpha.txt [..................................................] real 2m1.514s user 0m12.038s sys 0m16.199s $ time ./LZ4Demo.exe -d /tmp/out.lz4 /tmp/out.lz4.txt *** Compression CLI using LZ4 algorithm , by Yann Collet (Jul 25 2012) *** Successfully decoded 3590403790 bytes Done in 24.54 s ==> 139.53 MB/s real 1m47.692s user 0m6.320s sys 0m18.227s Decompression is thus faster for LZ4 but probably very limited in both cases by writing the output file to the SSD.
- ogrisel 14y agoIndeed for in-memory decompression LZ4 is much faster as expected: $ time ./LZ4Demo.exe -d /tmp/out.lz4 /dev/null *** Compression CLI using LZ4 algorithm , by Yann Collet (Jul 25 2012) *** Successfully decoded 3590403790 bytes Done in 8.04 s ==> 426.03 MB/s real 0m17.829s user 0m5.837s sys 0m2.206s I cannot do the same bench using fastlz default 6unpack program as it does not seem to be able to skip the filesystem based decompression.