7 ms·
Yes, yes, yes, yes, yes. I'm going to make the kind of prediction that will set me up to be the laughingstock of HN in a few years. I think in-place binary pa
by benesch 6y ago
Yes, yes, yes, yes, yes.
I'm going to make the kind of prediction that will set me up to be the laughingstock of HN in a few years.
I think in-place binary patching will be the single most consequential development in build toolchains in the last twenty years; the most consequential development since a graduate student at the University of Illinois named Chris Lattner decided to embark on LLVM. If Zig is successful in this endeavor, by the end of this decade I expect that in-place binary patching will be a feature that comes standard with most build toolchains.
When you reason about build systems from first principles, it becomes abundantly clear that our
current approach to linking is nuts. In both my current job and my last job, linking was by far the bottleneck in the development cycle, and a nearly unfixable one. Nothing puts a damper on getting into the zone like a sixty second link. Change a string literal somewhere? Wait sixty seconds. Fix a one-line bug in the test you just wrote? Wait sixty seconds. Change the guts of a function, but not its public interface? Wait sixty seconds.
Compilation is massively parallel, so if your compiler is slow, you have options. You can buy a bigger computer with more cores. You can outsource your compilation to the cloud. You can arrange your code cleverly to minimize dependencies, so changing one package doesn't result in invalidating too many downstream packages.
But if linking is slow, you're pretty stuck. Linking always happens, linking always happens last, and linking always depends on the output of all previous steps. Linking is rarely parallel and rarely incremental. You can switch from macOS to Linux, because Apple skimps on their linker. You can switch from the standard GNU linker (bfd) to the new GNU linker (gold) [0],
which has both built-in parallelism and an incremental mode. But gold is only 20-30% faster.
I'm not excited by a 30% speedup. Saving 30% on a 60s link still means you have a brutal 42s link; still plenty long to kick you out of the flow. A 30% speedup is the kind of speedup that's erased in a year when your project is 30% larger. What I'm excited about is a speedup the size of several orders of magnitude! That's the magic that Zig is striving for here. Change a string literal? Bam! Your binary has already been relinked, before your finger even rolled off the return key.
This is the kind of thing that has the potential to change our relationship with compiled languages forever. The differences between a dynamic language like Python and a compiled language like Zig become mightly slim when recompiling a large Zig program becomes as fast as restarting a Python program.
And the only way to get there is to redesign the compiler toolchain from start to finish. You need everything from the design of the programming language down to a custom linker to cooperate. Binary formats like ELF are not designed to be hot-swappable like this; they're designed to be write-once. The same goes for debugging information like DWARF. They've developed some very clever tricks to make this work. I wouldn't be surprised if one day there's a custom, non-ELF binary format specifically for the use of incremental toolchains.
Huge props to Andy Kelley and the Zig team for having the vision and the guts to embark on this quest. (That reminds me: I am long overdue on setting up a sponsorship for the Zig Software Foundation [1] to fund this sort of work!)
[0]: https://en.wikipedia.org/wiki/Gold_(linker) https://en.wikipedia.org/wiki/Gold_(linker)
[1]: https://ziglang.org/zsf/ https://ziglang.org/zsf/
- zokier 6y agoThis might be stupid question but would it make sense to forgoe linking a monolithic binary altogether and just load the different bits dynamically at startup from various object files or something like that? On the other hand your way of thinking sounds a lot like what I understand Smalltalk image format to be like
- benesch 6y agoThat's not a stupid question. That's extremely insightful! That is exactly my model of the world, in fact. When linking incrementally, you don't want a tightly packed object file where the functions are stacked right on top of one another. You want something more akin to a key–value database, where you can swap out key–value pairs without corrupting the entire file. Whether that's an object file that "symlinks" a bunch of other object files, or is one large file with easily swappable blocks (like a SQLite database) probably involves evaluating tradeoffs that I don't have a lot of context on. (For example, one benefit of keeping it all in one file is that you can send the binary to someone else or move it to another path without corrupting the file.)
- jorangreef 6y agoI'm with you on the mental model, but from a hardware point of view what you also don't want is hundreds of random disk seeks. I know we're almost in an SSD-only world but HDDs are still a thing and large sequential reads are always important at any level of the memory hierarchy.
- benesch 6y agoI wouldn't expect that to matter much during the debug cycle. The whole binary is going to be sitting in the buffer cache.
- jorangreef 6y agoSure, but the buffer cache is also part of the memory hierarchy. At every level, it's always good not to go wild with cache misses. Not that that's what you're proposing of course.