3 ms·
Linux EFI Zboot Abandoning "Compression Library Museum", Focusing on Gzip, ZSTD
- fn-mote 2y agoThe article gives reasons, but it sounds to me like gzip was retained mainly because it is in use. I don’t have a horse in this race, but if only Zstd were left I would feel like even more simplification was accomplished. Everybody has their own tiny preference for one of the other but really… if it’s not embedded what is the impact? > - GZIP is tried and tested, and is still one of the fastest at decompression time, although the compression ratio is not very high; moreover, Fedora is already shipping EFI zboot kernels for arm64 that use GZIP, and QEMU implements direct support for it when booting a kernel without firmware loaded; > - ZSTD has a very high compression ratio (although not the highest), and is almost as fast as GZIP at decompression time.
- bawolff 2y ago> but it sounds to me like gzip was retained mainly because it is in use. Isn't that what the article is literally saying? They aren't trying to get rid of badly performing compression algos, just ones that are effectively unused.
- klysm 2y agoYeah I generally agree that leaving just zstd would simplify things, but I also see advantages in leaving 2. It seems plausible that a better thing will come along in the future. It’s nice to leave the abstraction in place that enables having more than one algo in place.
- walrus01 2y agozstandard with the correct flags and more cpu/time-intensive options, when creating the compressed file, can be almost as good in ratio as xz.
- alkh 2y agoAs far as I can tell, one of zstandard's main benefits is compression/decompression speed. Is the ratio still worse than xz though?
- AzzyHN 2y agoAt high levels (19), it gives similar ratios to xz and 7z. But you're correct that the main advantage is the comparatively fast compression and very fast decompression speeds.
- walrus01 2y agoIts default compression/decompression speed with no options passed is superior to gzip in almost every possible scenario and cpu type. It does have other options which are much more time intensive for creating an archive but which make it more competitive with bzip2, xz, lz4 and similar.
- deleted 2y ago[deleted]
- theandrewbailey 2y ago> - ZSTD has a very high compression ratio (although not the highest), and is almost as fast as GZIP at decompression time. If you're using zstd and its decompression speed isn't at least twice as fast as gzip's, you're using it wrong.
- consp 2y agoYes, everyone tunes their compression settings. /s It's a generalized statement of 'good enough'.
- orf 2y agoEven untuned, for every single workload I’ve tried I see more than 2x improvement in speed. Usually 5x to 10x
- mananaysiempre 2y agoNot actually true if gzip refers to libdeflate[1] or isa-l[2] instead of zlib. Does seem sus here, though,—I couldn’t quickly find which decompression code is used in zboot, but I’d expect it to be closer to zlib. Perhaps their implementation of Zstandard is simpler than the reference one? [1] https://github.com/ebiggers/libdeflate https://github.com/ebiggers/libdeflate [2] https://github.com/intel/isa-l https://github.com/intel/isa-l
- lousken 2y agowhy not keep lz4 as well, shouldn't it have the smallest performance hit compared to those two?
- Ufvasl 2y agoThis can't be answered with a simple yes/no without benchmarking this specific use case on specific hardware. For other use cases I've personally never really found lz4 very useful (neither gzip). Usually zstd on either low/medium can be both faster (or at least, fast enough) and better compressed.
- malkia 2y agoGame developers use LZ4 a lot, like a lot. It's fast on both ends.
- speed_spread 2y agoBooting only requires decompression which mighty explain why LZ4 doesn't shine there.
- SushiHippie 2y agolz4 is pretty awesome for ZFS compression. It has nearly no impact on read/write and writing/reading data is sometimes even faster than without compression. Though I use zstd on zfs datasets which I rarely read/write to i.e. backups.
- user20180120 2y agoLong Range Zip tends to give faster decompression. Anyone know which command line switches gives optimum results?