2 ms·
Parent poster used the 5% value, so I did as well. But yes, to answer your question. Shown here on a Debian 11 system that's fairly old: $ hyperfine "export
by sgarland 3y ago
Parent poster used the 5% value, so I did as well. But yes, to answer your question. Shown here on a Debian 11 system that's fairly old:
$ hyperfine "export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort -S5% | uniq -c | sort -rn -S5% | head"
Benchmark 1: export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort -S5% | uniq -c | sort -rn -S5% | head
Time (mean ± σ): 1.504 s ± 0.318 s [User: 2.833 s, System: 0.474 s]
Range (min … max): 0.942 s … 1.937 s 10 runs
$ hyperfine "export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort | uniq -c | sort -rn | head"
Benchmark 1: export LC_ALL=C; cut -d' ' -f1 fake_log.txt | sort | uniq -c | sort -rn | head
Time (mean ± σ): 3.847 s ± 0.093 s [User: 4.165 s, System: 0.613 s]
Range (min … max): 3.591 s … 3.919 s 10 runs
Setting the buffer value to ~half the file size (100 MB) resulted in a mean time of 2.291 seconds. Setting it to the size of the file resulted in a mean time of 1.549 seconds, which is close enough to the 5% to call it equal - not like this server isn't busy with other stuff, so it's hardly a good place for perfect benchmarking.
Syscalls aren't free, nor are disk reads, so if you have the RAM to support slurping the entire file at once, it's sometimes faster.