4 ms·
Multicore version of bzip2 compression utility
- wheels 17y agoTried it before and found it completely useless. The only place I cared about using parallel compression / decompression was on large files, and since it does everything in memory it dragged my system into a swapping hell every time that I used it.
- neilc 17y agoYou tried it, although it was just released today? since it does everything in memory it dragged my system into a swapping hell Well, it just keeps pending blocks in memory. It should be quite easy to bound the worst-case memory consumption (at some cost to parallelism), as they note in the article.
- wheels 17y agoI assumed it was the same as this: http://compression.ca/pbzip2/ http://compression.ca/pbzip2/ Though looking again, it seems like it's a separate implementation.
- evgen 17y ago> You tried it, although it was just released today? The version from clik was released today. Parallel bzip2 has been around for at least a couple of years. In fact, just doing a search for parallel bzip2 will lead you right to both an SMP/threaded version as well as one that uses MPI. I am guessing this is what was being referred to regarding the memory-consumption issues.
- Gompers 17y agoJeff Atwood (from Coding Horror) ran a comparison of bzip2 on 8 cores against 7-Zip on 2 cores in February. bzip2 ran faster but generated larger files. "The only reason to go bzip2 is if you need the file compressed as rapidly as possible to a reasonable size. On pure efficiency / energy, 7zip wins, despite bzip2 using 4x the CPU time." http://www.codinghorror.com/blog/archives/001231.html http://www.codinghorror.com/blog/archives/001231.html