4 ms·
I work with a project that takes an hour to build because it’s thousands of C++ files. What would you suggest I do to improve my build times?
by saagarjha 6y ago
I work with a project that takes an hour to build because it’s thousands of C++ files. What would you suggest I do to improve my build times?
- coding_unit_1 6y agoDo you need to recompile all multi-thousand files every time you make a change? Modularising the build would be my first step.
- magicalhippo 6y agoA buddy slashed his CI build times by using a RAM disk. But yeah, usually faster to avoid compiling things.
- froj 6y agoWe have a similar situation and use ccache and distcc is setup on each dev's workstation. It also helps to use forward declarations instead of including header files whenever possible.
- rkangel 6y agoA good build system and caching. By "a good build system" I mean a hermetic one like Bazel that can be confident of knowing the inputs to each build step so that they can be run or not as appropriate. Sometimes you'll touch a core header file and a lot of things will rebuild, but it should still be a lot better.
- pjc50 6y agoSurprisingly, "concatenative builds" where you chuck all the files in a single file and compile that, tend to be a lot faster. distcc/ccache also works. Years ago we had a rack full of machines devoted to this. Requires a little tuning but lets you do -j 50. Of course, that was in the days of dual-core machines and these days you can buy a lot of cores. Bottleneck is probably writing intermediate files to disk and flushing them. Untangling the inter-dependencies that cause all the rebuilding is very hard and often forces design changes you don't want, such as use of the "pimpl" idiom.
- rob74 6y agoEven without having programmed in C/C++ for years, it doesn't sound very surprising that "concatenative builds" are faster - I assume it has something to do with the preprocessor having exponentially less work the fewer files you have?
- rlaanemets 6y agoOur QT project had about 400 compilation units (header+source, compiled to a separate object file). Many of them were 100-200 lines of code only. Full build used to take 40 minutes. Each compilation unit pulls in about 40k lines of code from Qt through headers. So the compiler has to process at least 400*40k = 16M lines of code. Concatenative build only processes app code (about 50k) plus about 60k of cumulated framework code. This gives about 100 times speedup. Our concatenative build took indeed less than 1 minute.
- saagarjha 6y agoThis is with unified builds :(