33 ms·
Why does an extraneous build step make my Zig app 10x faster?
- 0x000xca0xfe 3y agoAlso a great reminder to always benchmark with different data set sizes.
- Rygian 3y agoThe TL:DR; is that the build step masks the wait for input from a shell pipe. With a side dish of "do buffered input" and then a small "avoid memory allocation for fun."
- FreeFull 3y agoYou can easily hit a similar problem in other languages too. For example, in Rust, std::fs::File isn't buffered, so reading single bytes from it will also be rather slow.
- aforty 3y agoThere is general wisdom about bash pipelines here that I think most people will miss simply because of the title. Interesting though, my mental model of bash piping was wrong too.
- alas44 3y agoWe are two!
- Joker_vD 3y agoThere were several reasons why pipes were added to Unix, and the ability to run producer/consumer processes concurrently was one of them. Before that (and for many years after on non-Unix systems) indeed the most prevalent paradigm were to run multi-stage pipelines with the moral equivalent of the following: stage1.exe /in:input.dat /out:stage1.dat stage2.exe /in:stage1.dat /out:stage2.dat del stage1.dat stage3.exe /in:stage2.dat /out:result.dat del stage2.dat
- jakogut 3y agoPipes are so useful. I find myself more and more using shell script and pipes for complex multi-stage tasks. This also simplifies any non-shell code I must write, as there are already high quality, performant implementations of hashing and compression algorithms I can just pipe to.
- number6 3y ago"The programmer scoffed at Master Foo and rose to depart. But Master Foo nodded to his student Nubi, who wrote a line of shell script on a nearby whiteboard, and said: “Master programmer, consider this pipeline. Implemented in pure C, would it not span ten thousand lines?”" http://catb.org/~esr/writings/unix-koans/ten-thousand.html http://catb.org/~esr/writings/unix-koans/ten-thousand.html
- keybored 3y agoUgh. I don’t feel that the spirit of those satirical Zen Koans is to be so self-congratulatory.
- capitol_ 3y agoWhat programming language do you use where there isn't performant hashing/compression algorithms implemented as libraries?
- jvanderbot 3y agoWell they all do, but in terms of ease of use, tar and zip are much simpler to implement in a cli pipeline than to write bespoke code. At least that has been my experience.
- jerf 3y agoIt is hard to compete with "| gzip" in any programming language. Just importing a library and you're already well past that. Just typing "import" and you're tied! Overbudget if I drop the space in "| gzip". This is one of the reasons why, for all its faults, shell just isn't going anywhere any time soon.
- adql 3y ago...how ? It's called pipe, not "infinitely large buffer that will wait indefintely till the command ends to pass its output further"
- shiomiru 3y agoDOS also has a "pipe", which works exactly like that. (Obviously, since DOS can't run multiple programs in parallel.)
- zamfi 3y agoCan’t speak for OP, but one might reasonably expect later stages to only start execution once at least some data is available—rather than immediately, before any data is available for them to consume. Of course, there many reasons you wouldn’t want this—processes can take time to start up, for example—but it’s not an unreasonable mental model.
- OJFord 3y agoNot even that they might be particularly slow to start in absolute terms, but just that they might be slow relative to how fast the previous stage starts cranking out some input for it. (Since, as GP said, not an infinite buffer.)
- Joker_vD 3y agoWell, it could be implemented like this, it's just more cumbersome than "create N-1 anonymous pipes, fork N processes, wait for the last process to finish": at the very least you'll need to select() on the last unattached pipe, and when job control comes into the picture, you'd really would like the "setting up the pipeline" and "monitoring the pipeline's execution" parts to be disentangled.
- hawski 3y agoI know this about Unix pipes from a very long time. Whenever they are introduced it is always said, but I guess people can miss it. Though now I will break your mind as my mind was broken not a long time ago. Powershell, which is often said to be a better shell, works like that. It doesn't run things in parallel. I think the same is to be said about Windows cmd/batch, but don't cite me on that. That one thing makes Powershell insufficient to ever be a full replacement of a proper shell.
- Symmetry 3y agoMy first guess involved caching but I was thinking about whether the binary itself had to be read from disk or was already cached in RAM. Great linux-fu post.
- OJFord 3y agoI was so confused about why this mattered/made such a difference - until I went back and re-read from the top: OP does the benchmark timing in `main`, in the Zig app under test. If you don't do that, if you use the `time` CLI for example, this wouldn't have been a problem in the first place. Though sure you couldn't have compared to compiling fresh & running anyway, and at least on small inputs would've wanted to do the input prep first anyway. But I think if you put the benchmark code inside the DUT you're setting yourself up for all kinds of gotchas like this.
- hawski 3y agoI don't want to belittle the author, but I am surprised, that people using a low-level language on Linux wouldn't know how Unix pipelines work or that reading one byte per syscall is quite inefficient. I understand that the author is still learning (aren't we all?), but I just felt it is a pretty fundamental knowledge. At the same time author managed to have better performance that the official thing had. I guess many things feel fundamental in the retrospect.
- mtlynch 3y agoAuthor here. Thanks for reading! >I am surprised, that people using a low-level language on Linux wouldn't know ... that reading one byte per syscall is quite inefficient. In my defense, it wasn't that I didn't realize one byte per syscall was inefficient; it was that I didn't realize that I was doing one syscall per byte read. I'm coming back to low-level programming after 8ish years of Go/Python/JS, so I wasn't really registering that I'd forgotten to layer in a buffered reader on top of stdin's reader. Alex Kladov (matklad) made an interesting point on the Ziggit thread[0] that the Zig standard library could adjust the API to make this kind of mistake less likely: >I’d say readByte is a flawed API to have on a Reader. While you technically can read a byte-at-time from something like TCP socket, it just doesn’t make sense. The reader should only allow reading into a slice. >Byte-oriented API belongs to a buffered reader. [0] https://ziggit.dev/t/zig-build-run-is-10x-faster-than-compiled-binary/3446/9 https://ziggit.dev/t/zig-build-run-is-10x-faster-than-compil...
- joelfried 3y agoThank you for sharing it! Articles like this are how one learns the nuances of such things, and it's good for people to keep putting them out there.
- hawski 3y agoZig certainly needs more work. That part is more on familiarity with Zig and how intuitive it is or isn't. In any case I would recommend anyone investigating things like that to run things through strace. It is often my first step in trying to understand what happens with anything - like a cryptic error "No such file or directory" without telling me what a thing tried to access. You would run: $ strace -f sh -c 'your | pipeline | here' -o strace.log You could then track things easily and see what is really happening. Cheers!
- masto 3y agoI realize one needs a catchy title and some storytelling to get people to read a blog article, but for a summary of the main points: * This is not about a build step that makes the app perform better * The app isn't 10x faster (or faster at all; it's the same binary) * The author ran a benchmark two ways, one of which inadvertently included the time taken to generate sample input data, because it was coming from a pipe * Generating the data before starting the program under test fixes the measurement
- meowface 3y agoAnother semi-summary of the core part of the article: >"echo '60016000526001601ff3' | xxd -r -p | zig build run -Doptimize=ReleaseFast" is much faster than "echo '60016000526001601ff3' | xxd -r -p | ./zig-out/bin/count-bytes" (compiling + running the program is faster than just running an already-compiled program) >When you execute the program directly, xxd and count-bytes start at the same time, so the pipe buffer is empty when count-bytes first tries to read from stdin, requiring it to wait until xxd fills it. But when you use zig build run, xxd gets a head start while the program is compiling, so by the time count-bytes reads from stdin, the pipe buffer has been filled. >Imagine a simple bash pipeline like the following: "./jobA | ./jobB". My mental model was that jobA would start and run to completion and then jobB would start with jobA’s output as its input. It turns out that all commands in a bash pipeline start at the same time.
- anonymous-panda 3y agoThat doesn’t make sense unless you have only 1 or 2 physical CPUs with contention. In a modern CPU the latter should be faster and I’m left unsatisfied by the correctness of the explanation. Am I just being thick or is there a more plausible explanation?
- DougBTX 3y agoIt depends on where the timing code is. If the timer starts after all the data has already been loaded, the time recorded will be lower (even if the total time for the whole process is higher).
- underdeserver 3y agoIf I were trying to optimize my code, I would start with loading the entire benchmark bytecode to memory, then start the counter. Otherwise I can't be sure how much time is spent reading from a pipe/file to memory, and how much time is spent in my code. Then I would try to benchmark what happens if it all fits in L1 cache, L2, L3, and main memory. Of course, if the common use case is reading from a file, network, or pipe, maybe you can optimize that, but I would take it step by step.
- jedisct1 3y agoThis is exactly the thing that feels obvious once you realize it, but that can be puzzling until you don't.
- john-tells-all 3y agoThis is an excellent writeup, with interesting ideas and clear description of actions taken. My idea of pipelines, also, was flawed. Well done! Nothing to do with Zig. Just a nice debugging story.
- WalterBright 3y agoBack in college, a friend of mine decided to learn how to program. He had never programmed before. He picked up the DEC FORTRAN-10 manual and read it cover to cover. He then wrote a program that generated some large amount of data and wrote it to a file. Being much smarter than I am, his first program worked the first time. But it ran terribly slow. Baffled, he showed it to his friend, who exclaimed why are you, in a loop, opening the file, appending one character, and closing the file? That's going to run incredibly slowly. Instead, open the file, write all the data, then close it! The reply was "the manual didn't say anything about that or how to do I/O efficiently."
- TylerE 3y agoI firmly believe that teaching how to idiomatically do both character and line oriented file IO should be the first thing any language tutorial teaches, almost. Just as soon as you’ve introduced enough syntax.
- PoignardAzur 3y agoFWIW, the Epitech cursus starts your very first C programming lesson by making you write a "my_putchar" function that does a "write" syscall with a single character. Then you spend the next few days learning how to create my_putnbr, my_putstr, etc, using that single building block. I think that's the right choice, by the way. Baby developers don't need to learn efficient I/O, they need to learn how you make the pile of sand do smart things. And if you've spent weeks doing I/O one syscall per character, getting to the point you write hundreds of lines that way, the moment some classmate shows you that you can 100x your program's performance by batching I/O gets burned in your memory forever, in a way "I'm doing it because the manual said so" doesn't.
- TylerE 3y agoI said idiomatic because that’s the form they’re going to encounter it in the wild, in library doc, in stack overflow answers. I do t think rhe sort of low level bit banging you propose is a worthwhile use of a students time, given the vast amount they have to learn that won’t be immediately obsolete.
- michael1999 3y agoIf you want create something like the pipe behaviour the author expected (buffer all output before sending to the next command), the sponge command from moreutils can help.
- styfle 3y ago> By adding a benchmarking script to my continuous integration and archiving the results, it was easy for me to identify when my measurements changed. This assumes CI runs on the same machine with same hardware every time, but most CI doesn’t do that.
- boesboes 3y agoAnd that the hardware is not overbooked. I found that my ci/cd runs would vary between 8 and 14 minutes (for a specific task in the pipeline, no cache involved) between reruns. And it seemed correlated to time of day. So pretty sure they had some contention there. Edit: and that was with all the same cpu’s reported to the os atleast
- mcguire 3y agoThere seems to be a small misunderstanding on the behavior of pipes here. All the commands in a bash pipeline do start at the same time, but output goes into the pipeline buffer whenever the writing process writes it. There is no specific point where the "output from jobA is ready". The author's example code, "jobA starts, sleeps for three seconds, prints to stdout, sleeps for two more seconds, then exits" and "jobB starts, waits for input on stdin, then prints everything it can read from stdin until stdin closes" is measuring 5 seconds not because the input to jobB is not ready until jobA terminates but because jobB is waiting for the pipe to close which doesn't happen until jobA ends. That explains the timing of the output: $ ./jobA | ./jobB 09:11:53.326 jobA is starting 09:11:53.326 jobB is starting 09:11:53.328 jobB is waiting on input 09:11:56.330 jobB read 'result of jobA is...' from input 09:11:58.331 jobA is terminating 09:11:58.331 jobB read '42' from input 09:11:58.333 jobB is done reading input 09:11:58.335 jobB is terminating The bottom line is that it's important to actually measure what you want to measure.
- mtlynch 3y agoAuthor here. Thanks for reading! >All the commands in a bash pipeline do start at the same time, but output goes into the pipeline buffer whenever the writing process writes it. There is no specific point where the "output from jobA is ready". Right, I didn't mean to give the impression that there's a time at which all input from jobA is ready at once. But there is a time when jobB can start reading stdin, and there's a time when jobA closes the handle to its stdout. The reason I split jobA's output into two commands is to show that jobB starts reading 3 seconds after the command begins, and jobB finishes reading 2 seconds after reading the first output from jobA.
- dsm9000 3y agoThis post is another example of why I like zig so much. It seems to get people talking about performance in a way which helps them learn how things work below today’s heavily abstracted veneer