3 ms·
I used to be a big fan of XZ(which uses lzma2), but Zstandard has good-enough compression at high levels with orders of magnitude faster decompression. It's als
by wongarsu 6y ago
I used to be a big fan of XZ(which uses lzma2), but Zstandard has good-enough compression at high levels with orders of magnitude faster decompression. It's also more versatile in its compression levels. It's now also my go-to for new projects.
- formerly_proven 6y agoXZ/LZMA2 has the problem that it slows down a lot on incompressible data and can even choke pretty badly on compressible data, which Yann Collet's algorithms (LZ4 and ze standard) don't do nearly as much. This means zstd can be used indiscriminately, while LZMA would frequently need some sort of qualifier to avoid processing incompressible data and the associated loss of performance. To put some numbers to this, on the machine (>10yo laptop) I'm typing this on: trivial random .wav file LZMA ~20 MB/s ~2 MB/s ~1 MB/s zstd ~500 MB/s ~250 MB/s ~70 MB/s zlib ~40 MB/s ~13 MB/s ~6 MB/s
- saurik 6y ago(xz is a compression container format that doesn't imply any particular compression algorithm. It has a concept of "filters" that are applied in "chains" over "blocks". One filter happens to be "lzma2", but to say xz "uses lzma2" is a bit of a confusing mental model, as xz could just as easily "use zstd" if someone allocated a filter ID for such.)
- wongarsu 6y agoIn principle yes, but in practice the only filters included in my version are lzma1, lzma2, a basic delta filter and a few filters for executable code. As far as I can tell all the presets that are reachable without explicitly specifying a filter chain are just lzma2 with various options set. I would love if xz had a zstd filter and a filter that adds/processes some error correction code. But with what's there right now I think it's fair to treat it as a pure lzma2 container format.