3 ms·
From the bits of testing I've done today, it's phenomenally fast on x86. Much better than gzip (and pigz for that matter) in every metric I think I generally c
by Twirrim 10y ago
From the bits of testing I've done today, it's phenomenally fast on x86. Much better than gzip (and pigz for that matter) in every metric I think I generally care about: CPU Usage, Compression Speed, Decompression Speed, Compression Ratio.
On other architecture the picture gets a bit murky, it seems to get handily beaten by pigz through what at first blush I'd guess is just sheer parallelism. It's got solid performance, and without a shadow of doubt faster than vanilla gzip. If/as/when I get time, it'll be interesting to dig into why performance is worse there.
- Twirrim 10y agoDug in a bit further. On the non-x86 architecture I use, it looks like it's really just straight core performance that explains it. pigz's only advantage there really seems to be the brute force parallelism. In particular note the huge difference in branches between gzip and zstd on decompress: 8959780663 branches # 143.024 M/sec 2969481781 branches # 64.454 M/sec and on misses: 542158823 branch-misses # 6.05% of all branches 89060880 branch-misses # 3.00% of all branches
- Cyan4973 10y agoThere is now a pzstd implementation, if you wish to compare it to pigz
- Twirrim 10y agoHad a shot. It's slightly buggy, but holy crap is it fast. I'm not a C programmer, understanding what happened is a bit beyond me but: 1) to compile on linux it needs the -pthread flag passed to it, Makefile is missing that (compiles fine on OS X) 2) decompression over stdin appears to be effectively impossible, still demands in input file. Compression over stdin works fine.