3 ms·
Bzip2 is a very slow compression algorithm, mostly due, from what I recall, to the Burrows-Wheeler transform that lies at its core. LZMA is pretty much superio
by DarkShikari 18y ago
Bzip2 is a very slow compression algorithm, mostly due, from what I recall, to the Burrows-Wheeler transform that lies at its core. LZMA is pretty much superior in every respect; it is (though quite slowly) on the road to replacing the aging Bzip2.
And as usual the comments on CodingHorror (at least the initial dozen or two) show a relative ignorance about the topic. 7zip (as can any compressor) can be trivially parallelized just by running it simultaneously on each solid block. The compression cost of a smaller solid block size is generally near-zero for the case where dictionary size << input data size.
The included Windows interface doesn't allow this kind of threading AFAIK, but it would be relatively simple to implement in an app using the LZMA libraries.
- Andys 18y agoThis is the exact approach taken by pigz (Parallel GZIP) - http://www.zlib.net/pigz/ http://www.zlib.net/pigz/
- alecco 18y ago> and uses the zlib and pthread libraries Er, no, thanks. What about a good STL C++ implementation with OpenMP (automagic on STL.) That's great for us, developers. We'll never run out of things to do :)
- Andys 18y agoWhats the problem? pigz works as advertised for me. Almost a 4x speed increase on a 4 core system, assuming your storage I/O can keep up.