LZ4[1] intentionally compresses worse than Deflate—where Deflate is LZ77 with Huffman in a bitstream, LZ4 is just LZ77 with a good choice of octet-aligned encod

LZ4[1] intentionally compresses worse than Deflate—where Deflate is LZ77 with Huffman in a bitstream, LZ4 is just LZ77 with a good choice of octet-aligned encoding. It’s “we’re saturating Gigabit Ethernet and have some compute left over” compression, not “we need to squeeze our backups as much as possible” compression. If filtering+LZ4 does better than Deflate, then you’ve chosen the filter well. Curiously, for some kinds of serialized binary data varints or even zero-packing[2] can be a good enough filter[3]. [1] https://lz4.github.io/lz4/ https://lz4.github.io/lz4/ [2] https://capnproto.org/encoding.html#packing https://capnproto.org/encoding.html#packing [3] https://news.ycombinator.com/item?id=24073752 https://news.ycombinator.com/item?id=24073752