3 ms·
> If you look closely, you can see it's not quite one byte per write() but it's pretty close. Numbers go all at once for whatever reason, and those " : " string
by scottlamb 4y ago
> If you look closely, you can see it's not quite one byte per write() but it's pretty close. Numbers go all at once for whatever reason, and those " : " strings are another curiosity.
In many languages, there's an abstraction for writing (e.g. Rust's std::io::Write). There are implementations both for raw fds (e.g. std::io::File) and wrappers for buffering (e.g. std::io::BufWriter). Someone forgot to wrap the former in the latter, so write method calls and syscalls are 1:1. It's an easy mistake to make. It's also easy to fix once you've noticed it.
I think there should be a std::io::BufWrite interface that provides the flush method but more importantly just is a marker that the supplied Write is buffered. Stuff like a JSON library should use BufWrite to prevent this mistake. Rust's read side is like this already: there's std::io::Read vs std::io::BufRead (with std::io::BufReader as the most commonly used impl of BufRead but by no means the only one.) I'm using the Rust names, but the same idea applies to other languages.
The specific behavior quoted above is probably because whatever JSON library they're using writes as follows:
* whole numbers because it uses the std library's stuff that collects the numbers in a stack-allocated buffer before passing it on to the next layer's write call
* whole separators because that's a string literal in the JSON library somewhere
* strings a byte at a time because it's looping through looking for characters it needs to escape. (It'd probably be more efficient to scan ahead until the next character that needs to be escaped and do everything until then in one write call, but when you supply a buffering Write impl as the library author probably intended, maybe not enough more efficient that they noticed and thought it was worthwhile.)
> I wonder though, do people not strace programs any more?
I'm old-school, and even I rarely use unfiltered strace anymore. Too noisy on a complex/concurrent program, and not just because of dumb stuff. I filter it down to what I'm looking for or use another tool entirely.
- chrsig 4y ago>I'm old-school, and even I rarely use unfiltered strace anymore. Too noisy on a complex/concurrent program, and not just because of dumb stuff. I filter it down to what I'm looking for or use another tool entirely. This! Learning all of the strace flags is a very useful endeavor. Filtering is a must to see exactly the calls that I'm looking for. I don't think most developers appreciate how often syscalls happen. Without filtering, you're getting all: - file i/o (read, write, readv, writev, pread, pwrite, ...) - network i/o (select, poll, accept, epoll, ...) - memory allocations (mmap,munmap, sbrk, ...) - process querying (getenv/setenv, getpid, gettid,...) - time (gettimeofday, sleep, ...) - synchronization (futex, ...) I'm sure there's a bunch more that I'm neglecting. It produces quite a bit of noise!
- scottlamb 4y agoYeah, excellent description of why the number of syscalls is so high, and when you're tracing a multithreaded program there's the additional noise from the way it reports concurrent syscalls from the various threads. Besides filtering, I forgot to mention that I like seeing some kind of aggregation/summary. strace flags are useful; even better to write a little program with eBPF (on Linux) or DTrace (on macOS/BSD/Solaris). I like being able to do stuff like this: https://github.com/scottlamb/retina/issues/5#issue-933945792 https://github.com/scottlamb/retina/issues/5#issue-933945792