3 ms·
It also depends on your workload and the speed of your disks. If your disks are faster than your decompression algorithm when that algorithm is running alongsi
by microcolonel 6y ago
It also depends on your workload and the speed of your disks.
If your disks are faster than your decompression algorithm when that algorithm is running alongside the rest of your workload (generally not the case) then it can make sense to use the faster decompressor (lz4). In my understanding of the tradeoffs of zstd though, having used it recently in an application, chances are you have a free hardware thread that can saturate your disk without affecting your compute workload.
- jfkebwjsbx 6y agoConsidering that nowadays many people have an SSD, for boot files that would mean LZ4 is best. But perhaps for your data files that you don't open often, ZSTD is best because you save space on the SSD.
- microcolonel 6y ago> Considering that nowadays many people have an SSD, for boot files that would mean LZ4 is best. That depends on how concurrent boot is, and how fast your CPU and memory are. It may be true on a Celeron, but maybe not on a ThreadRipper. Furthermore, if you look at the performance testing, the sequential read performance was almost always better with zstd than with lz4, in ZFS; and the zstd-fast mode was about as fast as the lz4 mode in sequential writes. This may be a matter of their specific integration of lz4, but nonetheless it pays to look at the actual numbers before drawing conclusions.
- rakoo 6y agoIndeed in these cases lz4 makes sense. I'm thinking of what lz4 was created to "replace" namely snappy in SSTables: the pattern is mostly write once, read at most once or twice, and both of those are sequential to benefit of non-SSDs speeds. In these situations fast (de)compression schemes are definitely a gain.