4 ms·

I wish more C/C++ projects used "unity builds" -- there's only one translation unit containing all the code, usually by #include-ing all the .c and .cpp files.

I wish more C/C++ projects used "unity builds" -- there's only one translation unit containing all the code, usually by #include-ing all the .c and .cpp files. It makes "build configuration" incredibly easy by largely doing away with it. And C/C++ compilers just copy paste your code in via the preprocessor when doing #include of headers, etc., anyway, so it's not a wacky concept at all. - It's just one compiler invocation. If you want to support multiple compilers, just write that one invocation for each one. Usually clang and gcc will be the exact same. cl and clang-cl similarly. Even if you support multiple platforms, writing N command lines for N platforms is usually less characters than any build system out there, and simpler, and less opaque as well. - You can guarantee exactly the flags you want are being set. No more problems where you set -fwrapv or -march or whatever on one sub-project, but forgot on another, etc., or trouble figuring out what's the "CMake way" of specifying something. Getting these types of flags working with build tools becomes quite frustrating and error prone the more specific your requirements are. - Not something I dealt with working at trading companies, but distro package maintainers can very easily patch in their own flags, or more often can get by without even that if your unity build can just put $CFLAGS or $CXXFLAGS at the end of the command line. - Wouldn't tout this as a major benefit as it has saved me maybe once, but it basically forces a compiler error for certain violations of the One Definition Rule, because the compiler can see the duplicate and conflicting definition at compile time, instead of the linker silently picking one of the conflicting definitions and masking the problem. I've found that there are ample ways to keep compilation speed pretty reasonable (we're talking <10 seconds), even compiling all code at once on large, complicated projects. The general solution is to not let your compile times get crazy in the first place, which means not using a lot of general purpose external dependencies or templates. Another technique is, for dev, split the unity build in two. Unity build of files you're actively working on or of files you need to use -O0 or -0g on for a while, and unity of the rest, and link. Games people often carry that idea of a split unity build further. Split the structure of the code itself, and have an "infrastructure" unity build, a "business logic" unity build, and the infra code actually dynamically loads the logic .so/.dll, and can reload it on demand (so you can iterate fast on logic, without recompiling or even linking your infra code). I don't work on games so this hasn't been worth it for my projects, but I have definitely worked on projects structured with a control process (slow, smart, size not a critical issue), and a worker process (fast, dumb, size critical), which gets you pretty much the same structure by different means. You can easily get 1s turnaround for builds of the worker. I never used this structure specifically in order to help dev turnaround, rather it was intrinsic to the project requirements and faster dev was a happy knock-on effect. I'd be a happy man if I saw a post like this in the future that at least started with: If you see build.sh or build.ps1/bat, just run that and you're done. Edit, PS: "unity build" obviously has nothing to do with Unity the game engine. In fact, I never even called them "unity builds" at work, I had never heard that term. I always called it a blob build, or a STU (single translation unit) build, or whatever analogy was funny that day.

mid-kid 5y ago
The main issue with unity builds is the simple fact that you get very slow incremental builds. When working on a project you generally want to keep a cache of already-built files to avoid wasting time on a rebuild, while not sacrificing runtime speed in the form of disabling optimizations. With C unity builds don't quickly get very bad, but with C++ it grows fast. This goes hand-in-hand with the fact that for clean builds, parallelizing compilation can improve its speed by a significant amount. And for bigger projects (especially when we're talking C++), unity builds are simply infeasible due to their RAM requirements. Compiling a single file in the firefox code can take up to 3GB at times, and while you may consider it an extreme example, it can be deceptively easy to reach such numbers. All of this is of course not considering the namespace collision issues, where in C++, the "anonymous" namespace is shared across the entire file, and similarly, in C, functions and variables marked as "static" are only visible in the file they're defined in. Both of these are very useful features to reduce verbosity and reduce namespace clutter.
- dundarious 5y ago
  Implicit in my post was an admonition to avoid the kind of code that causes compilation time explosion. I do acknowledge that nowadays it's very uncommon for people to actually do this -- and I count my past self as one of those people failing to manage compile times properly. Just looking at some cons, with small units you have: - recompilation of the same headers over and over (not really "incremental" then) - must pay increased build system costs for each unit (see https://apenwarr.ca/log/20181113 https://apenwarr.ca/log/20181113) -- something of an open problem - reduced ability to optimize across units (LTO was a bit finicky 10 years ago and it still is today) - reduced developer will to keep compilation times reasonable (workarounds exist like "cloud build systems" such as bazel, which retrieve build artifacts from the cloud, because processing some text files locally is too taxing now) I do agree that there is some project size where a single unity build is too much. But leaving unity/incremental aside, I think there's a strong argument to avoid code that's slow to compile. And if and only if that is agreed upon for a project, then even for very large projects, you can have a unity build for each of the 10 or so modules that make up the system. And if your build is so big, you can use a simple build system like redo for that if you wish, the cost in specification and time is relatively low. However, I think that should be quite rare. We really don't have to pretend we're "optimizing" things by acting like every project we build is the size of firefox or the linux kernel. Let those projects have build systems, and let the remaining 99% of projects just call gcc one or two times. There's very little in the ecosystem of open source software to make doing unity builds very easy, so I'm not chiding anyone for making a tradeoff and using a behemoth like Qt instead of Dear ImGui, because they need assistive tech to work and there's nothing "unity buildable" that supports it.

mid-kid 5y ago

The main issue with unity builds is the simple fact that you get very slow incremental builds. When working on a project you generally want to keep a cache of already-built files to avoid wasting time on a rebuild, while not sacrificing runtime speed in the form of disabling optimizations. With C unity builds don't quickly get very bad, but with C++ it grows fast. This goes hand-in-hand with the fact that for clean builds, parallelizing compilation can improve its speed by a significant amount. And for bigger projects (especially when we're talking C++), unity builds are simply infeasible due to their RAM requirements. Compiling a single file in the firefox code can take up to 3GB at times, and while you may consider it an extreme example, it can be deceptively easy to reach such numbers. All of this is of course not considering the namespace collision issues, where in C++, the "anonymous" namespace is shared across the entire file, and similarly, in C, functions and variables marked as "static" are only visible in the file they're defined in. Both of these are very useful features to reduce verbosity and reduce namespace clutter.

dundarious 5y ago
Implicit in my post was an admonition to avoid the kind of code that causes compilation time explosion. I do acknowledge that nowadays it's very uncommon for people to actually do this -- and I count my past self as one of those people failing to manage compile times properly. Just looking at some cons, with small units you have: - recompilation of the same headers over and over (not really "incremental" then) - must pay increased build system costs for each unit (see https://apenwarr.ca/log/20181113 https://apenwarr.ca/log/20181113) -- something of an open problem - reduced ability to optimize across units (LTO was a bit finicky 10 years ago and it still is today) - reduced developer will to keep compilation times reasonable (workarounds exist like "cloud build systems" such as bazel, which retrieve build artifacts from the cloud, because processing some text files locally is too taxing now) I do agree that there is some project size where a single unity build is too much. But leaving unity/incremental aside, I think there's a strong argument to avoid code that's slow to compile. And if and only if that is agreed upon for a project, then even for very large projects, you can have a unity build for each of the 10 or so modules that make up the system. And if your build is so big, you can use a simple build system like redo for that if you wish, the cost in specification and time is relatively low. However, I think that should be quite rare. We really don't have to pretend we're "optimizing" things by acting like every project we build is the size of firefox or the linux kernel. Let those projects have build systems, and let the remaining 99% of projects just call gcc one or two times. There's very little in the ecosystem of open source software to make doing unity builds very easy, so I'm not chiding anyone for making a tradeoff and using a behemoth like Qt instead of Dear ImGui, because they need assistive tech to work and there's nothing "unity buildable" that supports it.