3 ms·
Options for Parallel Compression
- mmt 17y agop7zip's LZMA can be parallelized, and it has plenty of tuning knobs. There are other LZMA implementations, too. You want Markoff chains for really repetitive data. The big disadvantage is that p7zip doesn't really do unix pipes, courtesy of its dosbox heritage. Shameless self-promotion, though weblogs are different than database dumps: http://news.ycombinator.com/item?id=1060821 http://news.ycombinator.com/item?id=1060821