3 ms·
pbzip2 only does parallel compression. lbzip2 can parallelize compression and decompression. With both pbzip2 and lbzip2 I got errors every now and then while
by treffer 10y ago
pbzip2 only does parallel compression. lbzip2 can parallelize compression and decompression.
With both pbzip2 and lbzip2 I got errors every now and then while everything worked with bzip2. YMMV.
Also note that bzip2 is block based (due to bwt) and thus does not compromise compression ratios like parallelized gzip implantations.
At 32 cores you can saturate Gbit links (even with good compression ratios!). I hope we will see some more bzip2 love in the future due to it's perfect fit for parallelism.
- 0x0 10y agoErrors sound deeply concerning. What kind of errors? Is it too risky to use [pl]bzip2 for archival reasons?
- treffer 10y agoWell, errors at a rate of one every few terabyte, and most of the time it was mainly decompression, and always caused program abort iirc. But those have been a few years ago, so my best advice would be: retest. You are testing your archives anyway, right? ;-)
- paxcoder 10y ago>You are testing your archives anyway, right? ;-) Umm no, I handle errors which I expect to be reported to me.