I often use a script I hacked together called memo [1] when exploring the commandline. Today I was running a pipeline like this: $ memo expensive-with-lots-o

I often use a script I hacked together called memo [1] when exploring the commandline. Today I was running a pipeline like this: $ memo expensive-with-lots-of-output | memo -s expensive | less I noticed that lz4 (the default (de)compressor memo uses) was the bottleneck. ("memo -s" absorbs standard input and calculates a checksum over it and the command arguments, if that checksum is in the cache, it knows it can just output the cached value instead of running the command). So I started looking around if there was nothing better around. From the Squish benchmarks posted elsewhere on this article [2], I noticed that zstd(1) seemed to make the right trade-offs. I required compression speed > 50MB/s, decompression speed at least as fast, and as high a compression ration as possible. zstd(1) seems to work brilliantly and is very fast, it was easy to compile too, just running make in its directory was sufficient for me. I love those kinds of projects. I've made it the default over lz4. What gave me even more confidence in this change is that zstd is by the author of lz4. [1]: https://github.com/aktau/dotfiles/blob/master/bin/memo https://github.com/aktau/dotfiles/blob/master/bin/memo (currently a version that doesn't prefer zstd(1) yet, I need to git push). [2]: https://quixdb.github.io/squash-benchmark/unstable/ https://quixdb.github.io/squash-benchmark/unstable/