3 ms·
Last evaluation has shown that the size increase of zstd compared to xz warrants an introduction of delta debs first, so that people don't have to download more
by julian-klode 7y ago
Last evaluation has shown that the size increase of zstd compared to xz warrants an introduction of delta debs first, so that people don't have to download more.
But then with deltas, mirror sizes grows even more, which is a hard sell.
Arch Linux gets away with using zstd -21 but that's not practical for general purpose Linux distributions, as it has significant memory requirements compared to xz.
Also the speedup in practice is negligent - while zstd is much faster than xz, most of the time installing packages is actually spent in fsync(). So you only see huge speedups when run in eatmydata.
- julian-klode 7y agoLike, zstd -21 --ultra requires a whopping 365MB of RAM to compress a 16 MiB file, zstd -19 requires 12 MB, and xz requires 10 MB. Decompressing -21 requires 20MB vs 10MB for -19 and xz.
- m0zg 7y agozstd is _way_ faster at decompressing though, even from --ultra: some 13x faster in the case of Arch Linux. https://en.wikipedia.org/wiki/Zstandard https://en.wikipedia.org/wiki/Zstandard
- julian-klode 7y agoI know that, but compression time is not as relevant as you think it is. If you look back at the measurements we did 2 years ago: https://lists.ubuntu.com/archives/ubuntu-devel/2018-March/040211.html https://lists.ubuntu.com/archives/ubuntu-devel/2018-March/04... You can see that the performance gain from zstd under realistic scenarios is about 10%. you can see that switching from xz to zstd only improved firefox install time from 37s to 33s; running in eatmydata to avoid fsync from dpkg improved it to 12.5s (8.5s with zstd). I don't know what Arch linux does, but they likely do not correctly fsync() data after its been written to a temporary file, and then fsync the directory after files have been renamed to their final names, as is required to achieve consistent results in a crash.