4 ms·
Imagine you have one source file with 100 of functions which you want to compile. Traditionally it could be compiled only using one core, processing function af
by acqq 6y ago
Imagine you have one source file with 100 of functions which you want to compile. Traditionally it could be compiled only using one core, processing function after function. Assuming we are processing already preprocessed information, there's no theoretical reason why more cores couldn't be used to compile the functions: e.g. with 4 cores, each could get 25 functions. The practical reason why not is not only that it increases code complexity in the trivial case (all the functions take the same time) but that in non-trivial case you can end up trying to do a lot of what the OS and make system together do with separate processes doing the separate compilation of many files, dynamically deciding what to do at which point.
This experiment tries to make a case that something in that direction is doable and worth doing. I missed the good proof for the later, based on the analysis of state of the existing projects, however. I would personally rather consider the completely opposite direction:
In the big C++ projects, most of the functions are already present in small compilation units, and the total of the build process mostly spends time in processing the same huge set of headers for every compilation unit. Often, the whole process would be faster if more compilation units would be compiled as one(!) when the headers aren't used in a way that the semantic of them changes depending on the way in which they are used (I've heard that, of all, LLVM actually does that unfortunate thing, but I haven't spend time analyzing that myself).
- yxhuvud 6y agoOne complication is inlining, whre some of those 25 methods may want to inline methods handled by some other process.
- thebiss 6y agoCompiling entire units as one was standard practice dealing with the (defunct) IBM VisualAge C++ compiler. On one of our larger projects, updating the makefile to concatenate all .cpp's in each component, then compiling those .cpp's, reduced build times from 15-20 hours down to 1-2 hours.