4 ms·
I did the same test, but added a rust and bash version. My results: Rust: 21.9MiB/s Bash: 282KiB/s PHP: 2.35MiB/s Python: 2.30MiB/s Node: 943KiB/s In my c
by rascul 4y ago
I did the same test, but added a rust and bash version. My results:
Rust: 21.9MiB/s
Bash: 282KiB/s
PHP: 2.35MiB/s
Python: 2.30MiB/s
Node: 943KiB/s
In my case, node did not crash after about two minutes. I find it interesting that PHP and Python are comparable for me but not you, but I'm sure there's a plethora of reasons to explain that. I'm not surprised rust is vastly faster and bash vastly slower, I just thought it interesting to compare since I use those languages a lot.
Rust:
fn main() {
loop {
print!("1");
}
}
Bash (no discernible difference between echo and printf):
while :; do printf "1"; done | pv > /dev/null
- ur-whale 4y agofor the bash case, the cost of forking to write two chars is overwhelming compared to anything related to I/O.
- megous 4y agoThere's no forking and it's wrinting one character.
- mauvehaus 4y agoEcho and printf are shell built-ins in bash[0]. Does it have to fork to execute them? You could probably answer this by replacing printf with /bin/echo and comparing the results. I'm not in front of a Linux box, or I'd try. [0] https://www.gnu.org/software/bash/manual/html_node/Bash-Builtins.html https://www.gnu.org/software/bash/manual/html_node/Bash-Buil...
- ur-whale 4y ago> Echo and printf are shell built-ins in bash Ah, yeah, good point, I am wrong.
- megous 4y agoRust also cheats. https://megous.com/dl/tmp/1046458b5b450018.png https://megous.com/dl/tmp/1046458b5b450018.png
- cle 4y agoSeems like it's buffering output, which Python also does. Python is much slower if you flush every write (I get 2.6 MiB/s default, 600 KiB/s with flush=True). Interestingly, Go is very fast with a 8 KiB buffer (same as Python's), I get 218 MiB/s.
- deleted 4y ago[deleted]
- anon946 4y agoFor languages like C, C++, and Rust, the bottleneck is going to mainly be system calls. With a big buffer, on an old machine, I get about 1.5 GiB/s with C++. Writing 1 char at a time, I get less than 1 MiB/s. $ ./a.out 1000000 2000 | cat >/dev/null buffer size: 1000000, num syscalls: 2000, perf:1578.779593 MiB/s $ ./a.out 1 2000000 | cat >/dev/null buffer size: 1, num syscalls: 2000000, perf:0.832587 MiB/s Code is: #include <cstddef> #include <random> #include <chrono> #include <cassert> #include <array> #include <cstdio> #include <unistd.h> #include <cstring> #include <cstdlib> int main(int argc, char **argv) { int rv; assert(argc == 3); const unsigned int n = std::atoi(argv[1]); char *buf = new char[n]; std::memset(buf, '1', n); const unsigned int k = std::atoi(argv[2]); auto start = std::chrono::high_resolution_clock::now(); for (size_t i = 0; i < k; i++) { rv = write(1, buf, n); assert(rv == int(n)); } auto stop = std::chrono::high_resolution_clock::now(); auto duration = stop - start; std::chrono::duration<double> secs = duration; std::fprintf(stderr, "buffer size: %d, num syscalls: %d, perf:%f MiB/s\n", n, k, (double(n)*k)/(1024*1024)/secs.count()); } EDIT: Also note that a big write to a pipe (bigger than PIPE_BUF) may require multiple syscalls on the read side. EDIT 2: Also, it appears that the kernel is smart enough to not copy anything when it's clear that there is no need. When I don't go through cat, I get rates that are well above memory bandwidth, implying that it's not doing any actual work: $ ./a.out 1000000 1000 >/dev/null buffer size: 1000000, num syscalls: 1000, perf: 1827368.373827 MiB/s
- mortehu 4y agoThere's no special "no work" detection needed. a.out is calling the write function for the null device, which just returns without doing anything. No pipes are involved.
- zbentley 4y agoI suspect (but am not sure) that the shell may be doing something clever for a stream redirection (>) and giving your program a STDOUT file descriptor directly to /dev/null. I may be wrong, though. Check with lsof or similar.
- hderms 4y agowith Rust you could also avoid using a lock on STDOUT and get it even faster!
- skitter 4y agoTested it, seems to about double the speed (from 22.3mb/s to 47.6mb/s).