4 ms·
It would have to be one really fast network... zstd compresses and decompresses at 5+ GB (bytes, not bits) per second.
by PhilipRoman 1y ago
It would have to be one really fast network... zstd compresses and decompresses at 5+ GB (bytes, not bits) per second.
- worldsavior 1y agoWhere are you getting this performance? On the average computer this is by far not the speed.
- o11c 1y agoI just tested on a ramdisk: tool cspeed size dspeed zstd 361 MB/s 16% 1321 MB/s lzop 512 MB/s 29% 539 MB/s lz4 555 MB/s 29% 1190 MB/s If working from files on disk that happen not to be cached, the speed differences are likely to disappear, even on many NVMe disks. (It just so happens that the concatenation of all text-looking .tar files I happen to have on this machine is roughly a gigabyte (though I did the math for the actual size)).
- PhilipRoman 1y agoLooks like it depends heavily on choice of file, but I see good performance on both compressible and uncompressible files. Small files tend to perform (relatively) bad though. Here is a sample of 3 large files with different compression ratios: zstd -b1 --fast -i10 some-rpi-os-image-idk.img -1#-os-image-idk.img :2097152000 -> 226798302 (x9.247), 6765.0 MB/s, 5897.3 MB/s zstd -b1 --fast -i10 jdk-11.0.8+10.tar -1#jdk-11.0.8+10.tar : 688844800 -> 142114709 (x4.847), 2660.7 MB/s, 2506.8 MB/s zstd -b1 --fast -i10 archlinux-2025.04.01-x86_64.iso -1#.04.01-x86_64.iso :1236303872 ->1221271223 (x1.012), 3643.5 MB/s, 7836.6 MB/s
- masklinn 1y agoAin't no way zstd compresses at 5+, even at -1. That's the sort of throughputs you see on lz4 running on a bunch of core (either half a dozen very fast, or 12~16 merely fast).