4 ms·

Also try LZ4 and Snappy. Depending on your use case they can save your extra CPU too.

• croon 4y ago
  They are in the benchmarks on the page.
• wmf 4y ago
  Recent versions of zstd mostly obsolete LZ4 and Snappy so you can use one compressor to rule them all.
  • morelisp 4y ago
    The chart on the zstd site doesn't really back that up for lz4. zstd 1.4.5 -1 2.884 500 MB/s 1660 MB/s zstd 1.4.5 --fast=1 2.434 570 MB/s 2200 MB/s zstd 1.4.5 --fast=5 2.178 700 MB/s 2420 MB/s lz4 1.9.2 2.101 740 MB/s 4530 MB/s 3% larger size for a nearly 2x decompression speed is a no brainer for lots of uses. Anecdotally I have also gotten much better decomp speed on snappy on previous generations of CPUs staying within ~10% compression ratio, though I had not benchmarked/tuned extensively nor reconfirmed in the past ~2 years. (Probably this can also vary a lot by implementation, which you're often stuck with based on some other library choice.)
    • adgjlsfhk1 4y ago
      this can be a bit misleading because you only get the 2x decompression speed if you aren't bandwidth limited. this means that zstd will almost always be faster over a network.
      • morelisp 4y ago
        Even if you are bandwidth-limited you can rephrase this as "3% slower for half the CPU usage" which is still a good bargain in a lot of cases.
      • anonymoushn 4y ago
        A common use of zstd is for zfs, so decode speed matters quite a bit.
    • josephg 4y ago
      The lz4 algorithm is also tiny code-wise. Pulling in a lz4 compressor / decompressor library into my wasm project only added 11kb to the bundled size. Zstd is many times larger. For my use case, any extra benefits of zstd are far outweighed by the extra cost of downloading zstd itself in the browser. LZ4 is smaller, faster and much more simple.
  • rincebrain 4y ago
    Recent versions of zstd definitely don't obsolete LZ4, or else I don't think the author would still be contributing to both... zstd also seems to be optimizing much more, recently, for the large stream of data use cases, versus the fixed-size or smaller records you might get in, say, databases, or filesystems, so I think it's actually becoming less generally useful over time of late. And if you're going to play with Snappy, you might find S2, which was linked on HN relatively recently, interesting. [1] [1] - https://github.com/klauspost/compress/tree/master/s2 https://github.com/klauspost/compress/tree/master/s2