3 ms·
The lz4 developers are the zstd developers :) In this case I suspect the IO reductions on the output are what matter -- while lz4 is faster, it also produces m
by ctur 4y ago
The lz4 developers are the zstd developers :) In this case I suspect the IO reductions on the output are what matter -- while lz4 is faster, it also produces more output (of course) which you then have to write to disk. This can make the wall time take longer... basically you become IO bound on output, be it network or disk.
zstd also can tune itself and adjust ratios to saturate output bandwidth, which is pretty cool (--adapt).
- marceloaltmann 4y agoThat is exactly it. The difference comes from compression ratio. ZSTD is compressing the data more, so you need to write less back to disk. Also when talking about streaming, the difference is even more as it has to go over wan to S3 (provider used on blog test).