9 ms·
The Little Things: Speeding up C++ compilation
- adembudak 6y agoGreat write up. Thanks for sharing.
- zabzonk 6y agoOne of (many) other ways to speed up C++ (or C) builds is to use multi-processing in your make file - https://stackoverflow.com/questions/414714/compiling-with-g-using-multiple-cores https://stackoverflow.com/questions/414714/compiling-with-g-... - I typically get about a 20% speed up, but obviously YMMV.
- saagarjha 6y ago> Remember that std::vector consists of three pointers to a chunk of dynamically allocated memory. Note that this is not required, and most std::string implementations inline small strings anyways.
- nikki93 6y agoWas your `std::string` SBO comment meant to relate to the `std::vector` quote? `std::vector` can't use SBO bc. `std::swap` on `std::vector`s can't invalidate pointers to elements.
- saagarjha 6y agoI swapped std::string and std::vector when writing that comment :/ But still, there's no need for a vector to necessarily keep three pointers, it could use a pointer and two sizes for example.
- ReactiveJelly 6y agoIn Rust they call it "three pointers" because a size and a pointer are either both 32 bits or both 64 bits
- steveklabnik 6y agoI’ve never heard this term used, to be honest. The type name is “usize.”
- plorkyeran 6y agoWhether std::vector holds three pointers or a pointer and two sizes is entirely irrelevant. Either way sizeof(vector<T>) does not vary depending on T.
- saagarjha 6y agoIt is extremely relevant when responding to the claim "std::vector consists of three pointers".
- gjvc 6y agoThis is an excellent and thorough review of the details. An almost-should-be-mandatory tool on projects of any decent size is https://ccache.dev/ https://ccache.dev/
- modeless 6y agoI wish someone would pick up the zapcc project. Compilers do an insane amount of duplicated and redundant work and there's tremendous potential for speeding up C++ builds if you are willing to rethink how the compiler works a little bit. https://github.com/yrnkrn/zapcc https://github.com/yrnkrn/zapcc
- wheybags 6y agoI work on the video game factorio, which is a c++ project. On a 9900k a rebuild takes about a minute, so it's pretty sizeable but not something ridiculous like oracle db or unreal engine. I tried using zapcc on it, and it was a complete failure. I don't have measurements to hand, but iirc it was actually slower than stock clang. I tested it on a threadripper 2950x with 64gb of ram, running Debian.
- modeless 6y agoI've had good results on some of my own projects. The current implementation certainly has limitations, but it's really early and the potential is huge. For example, because it can cache at a sub-compilation-unit granularity it can make rebuilds of a single file faster, which no distributed build tool or cache can do.
- MontagFTB 6y agoThank you for Factorio!
- jiggawatts 6y agoI just installed the game again to play the v1.0 release. It looks and sounds very good! Congratulations on making it to the release milestone after many years of hard work. As a developer, I must say that I'm deeply impressed by the simulation complexity and incredible scalability. I keep telling people that Factorio is the most complex train simulator game that I've ever played. But it isn't a train simulator. It's a factory game. The trains are optional. That's crazy to me, that a subset of a game can be more complex than entire games that "specialise" in the topic.
- stephc_int13 6y agoFrom my experience, unity builds (single compilation unit with #include of all source files into a single main file) and use of orthodox C++ (avoidance of templates and fancy features) is giving good results.
- ladberg 6y agoThat sounds like it breaks all incremental builds (correct me if I'm wrong) for relatively little benefit (skipping linking? less header parsing?). Are there scenarios where unity builds are faster than incremental builds? Or is the intended use-case only for from-scratch builds?
- keldaris 6y agoIn my experience, you can just do full from-scratch builds in under 10 seconds for any sanely sized codebase (say, under 1M LoC conservatively). This obviously assumes the "orthodox C++" part, so YMMV wildly depending on how far you stray from that. I usually have compile times under 0.5s for all personal projects that don't use CUDA, but I basically write C with a bit of function/operator overloading, absolutely no STL or templates at all. Consequently, I have no use for incremental (or parallel, for that matter) builds whatsoever.
- asdfasgasdgasdg 6y agoOnly an extraordinarily ambitious personal project would experience a significant amount of build latency on modern compilers and CPUs, no matter what approach is used. I don't think you can reason from "it works for personal projects" to "it's fast." One thing I can tell you immediately about this unity build idea that is suboptimal for many codebases: modern compilers do not have much intra translation unit parallelism. If there's more in your project than one CPU core can compile quickly (i.e. most shipping products), it's going to be a serious bottleneck.
- stephc_int13 6y agoI am not talking about hobby projects here, but codebases with more than 100K lines of code. I agree with your objection, scaling up to 2-8 compilation units to use several cores is something I also plan to test/benchmark. And to be honest, I don't completely understand why the classic way of compiling (makefiles + dozens separate files) is so slow but there is clearly something fishy as it is very often unbearable. Slow compilation time can be very detrimental for productivity as it creates a way for the mind to wander and lose focus.
- tekknolagi 6y agoHow does the <algorithm> header compare in that list? That's the one I always try and avoid in my projects.
- jart 6y agoWhat I've found wreaks the most havoc on build times is having too many -iquote and -isystem flags. Last time I profiled TensorFlow compiles using strace, about half the gcc wall time was spent inside all the stat() system calls those flags generate.
- gumby 6y agoModules are supposed to make much of this now quite useful info irrelevant. Unfortunately modules are several years in the future, both due to lack of implementations yet and the fact that few projects are being written in C++20 yet. I also harbour a suspicion that tricks like these will still be useful with modules.
- tomovo 6y agoLast time I checked somebody working on modules said something along the lines of "don't expect modules to speed things up". The goals were different, apparently (stop #defines from leaking everywhere, large-project management, libraries). Has this changed, are there any benchmarks available?
- zarkov99 6y agoIf its true that speeding up compilation was not a goal then the committee is out of its mind. Speeding up compilation would have more impact on productivity and the environment than almost anything else that could be done with the language.
- higerordermap 6y agoThe committee was always out of the mind. In the past years they have neglected compile times, debug build performance, error messages or sometimes even performance.
- will4274 6y agoModules contain an optimization (less include leaking) and a pessimization (more serialized build graph). Only the former applies to CPU time, so modules should lead to less cpu time, but it's unclear exactly how that translates to wall time.
- mhh__ 6y agoI still haven't seen a module in the wild yet, they should've been standardized 20 years ago now (D has had modules from day 1, and there are more people on the C++ committee than working on D at all) . I hope they get some traction but I just can't see it helping for all but the newest codebases.
- m0zg 6y agoJust use a build system in which reliable incremental builds are possible and are the default. Your compilation will take seconds most of the time.
- jeffbee 6y agoBuilds are just like any other software: if it takes too long, profile it. Use bazel build --profile or whatever equivalent your build system offers.
- deleted 6y ago[deleted]
- cgrealy 6y agoOn thing about forward declarations: they violate DRY. Now, most of the time, the trade off of repeating class Thing; is more than offset by faster build times. But if class Thing; is actually namespace Bob { namespace Dave { template <typename T> class ThingTrait; template <typename T> class Thing : BaseClass<T, allocator<T>, trait<ThingTrait<T> >; } } you really want to put that in it's own header, typically "Thing_fwd.h" (apologies for any syntax errors, been a while since I've written C++)
- asdfasgasdgasdg 6y agoYou're either gonna write #include "thing.h" or class Thing; Either way, as a practical matter, you're saying, "There's something called Thing I'm going to use." C++ forces you to say that before use in each translation unit. It's not really "repeating" yourself to write it, since you haven't "said" it yet in this TU.
- cgrealy 6y agoYou are repeating yourself if you write class Thing; in every file that uses a Thing. And as I said, that's generally fine. The issue becomes when the declaration of Thing includes namespaces, template params, etc. Rewriting that is tedious and error prone. Look at something like std::string. That's actually namespace std { template< class T > struct allocator; template<class charT> struct char_traits; template<> struct char_traits<char>; template<class CharT, class Traits = char_traits<CharT>, class Allocator = allocator<CharT>> class basic_string; using string = basic_string<char>; } No-one is forward declaring that.
- steerablesafe 6y agoI prefer _fwd headers for a different reason: it's easier to modify without breaking dependencies. If you change Thing to an alias to a class template specialization then all of the "class Thing;" forward declarations break.
- ephimetheus 6y agoAnother tip to keep compilation fast: don’t use Eigen. Non-trivial usage absolutely obliterates compile times and blows up the memory consumption by an insane amount.
- barumi 6y agoEigen is an absurdly template-intensive library, and people use it because so far there is nothing better or even remotely comparable to it. If you use Eigen but somehow feel that compilation time is your focus instead of actually doing linear algebra or solving systems of linear equations then you have a few best practices that you can follow and techniques at your disposal to lower compilation times.
- ephimetheus 6y agoSuch as? (genuinely curious) We've been trying hard to get memory consumption down because we get OOMs in our CI frequently. (Our largest compilation unit compiles for 3 minutes and peaks at just under 6GB memory)
- happyweasel 6y agoHmm... What about the pImpl Idiom? That should also help (and also brings in it's own costs).
- flohofwoe 6y agoBruce Dawson's Chromium build time investigation is very enlightening: https://randomascii.wordpress.com/2020/03/30/big-project-build-times-chromium/ https://randomascii.wordpress.com/2020/03/30/big-project-bui... The main takeaway: > That is, the main source files represent just 0.32% of the lines of code being processed. Chromium itself has about 12 million lines of code, but the compiler needs to process 3.6 billion lines of code because it needs to parse the same headers over and over again.
- secondcoming 6y agoThere's a lot of money to be made by someone who write a tool that takes in a C++ codebase and suggests all the improvements that can be made to speed the build up.
- grandinj 6y agoIf you can get it to work (not guaranteed) icecream can make a very large difference.
- kevincox 6y agoThis appears to be referring to a distcc derivative https://github.com/icecc/icecream https://github.com/icecc/icecream Note that discc doesn't really solve the header problem, it just throws more compute at it.
- grandinj 6y agoYup, some people use spare machines lying around the office, some people buy a big-ass server that can be shared between the developers, some people (like me) buy a bunch of used desktop-class quad-core machines and run a little build cluster. Very useful if you are working on very large codebases (I hack on LibreOffice, 10M LOC, from time to time).