3 ms·
That doesn't seem to be the case. A relatively fast machine can't gzip anywhere near 120 MB/s unless the data ultra compressible. Zstd is multithreaded, but sin
by wang_li 7y ago
That doesn't seem to be the case. A relatively fast machine can't gzip anywhere near 120 MB/s unless the data ultra compressible. Zstd is multithreaded, but single threaded it's speed isn't that fast either.
$ lscpu | grep name
Model name: Intel(R) Xeon(R) Gold 6154 CPU @ 3.00GHz
$ time dd if=/dev/zero of=/dev/null bs=10240 count=102400
102400+0 records in
102400+0 records out
1048576000 bytes (1.0 GB) copied, 0.145168 s, 7.2 GB/s
real 0m0.147s
user 0m0.036s
sys 0m0.111s
$ time dd if=/dev/zero bs=10240 count=102400 | gzip > /dev/null
102400+0 records in
102400+0 records out
1048576000 bytes (1.0 GB) copied, 5.36061 s, 196 MB/s
real 0m5.364s
user 0m5.317s
sys 0m0.472s
$ time dd if=/dev/urandom bs=10240 count=102400 | gzip > /dev/null
102400+0 records in
102400+0 records out
1048576000 bytes (1.0 GB) copied, 27.3478 s, 38.3 MB/s
real 0m27.352s
user 0m27.169s
sys 0m10.119s
$ time dd if=biggestwords bs=10240 count=102400 | gzip > /dev/null
10642+1 records in
10642+1 records out
108981378 bytes (109 MB) copied, 6.66068 s, 16.4 MB/s
real 0m6.669s
user 0m6.667s
sys 0m0.249s
$ time dd if=biggestwords bs=10240 count=102400 | gzip -1 > /dev/null
10642+1 records in
10642+1 records out
108981378 bytes (109 MB) copied, 1.53539 s, 71.0 MB/s
real 0m1.540s
user 0m1.522s
sys 0m0.138s
$ time dd if=biggestwords of=/dev/null bs=10240 count=102400
10642+1 records in
10642+1 records out
108981378 bytes (109 MB) copied, 0.0407698 s, 2.7 GB/s
real 0m0.043s
user 0m0.007s
sys 0m0.036s
Biggestwords is simply /usr/share/dict/words concatenated to itself 22 times so as to provide ~100 MB of data.
If you can get good enough compression it might still be worthwhile to compress. But it's pretty rare in my experience.