You should consider zlib, it's compatible with everything and has very nice options. Also it's very low in memory usage (around 500KB vs *MB). Check out how pig

You should consider zlib, it's compatible with everything and has very nice options. Also it's very low in memory usage (around 500KB vs *MB). Check out how pigz implements it with pthreads (one thread per block).