4 ms·

Bit of a side-question, but somewhat related. Is anyone working on "whole program compilation"? I don't mean whole program optimisation, I mean an attempt to re

Bit of a side-question, but somewhat related. Is anyone working on "whole program compilation"? I don't mean whole program optimisation, I mean an attempt to read all files for a given target in memory at the same time and then generate all translation units in one go (all in memory? and maybe linking them in memory too?). Clearly, there would be caveats (strange header inclusion techniques relying on macros to modify text of include files would break, gigantic use of memory and so forth), but for those willing to take the risk, presumably this should result in faster builds right? In fact, ideally you'd even generate all binaries for a project in one go but that may be taking it a step too far :-) At any rate, I searched google scholar for any experience reports on this and found nothing. It must have a more technical name I guess...

Khoth 8y ago
The terms you're looking for are "single compilation unit" or "unity build". It's used sometimes, I think mostly to help the compiler optimise better. Build times for a full rebuild may be faster, but may not, since traditional builds can use many CPU cores. However, it stops incremental builds from working - if you modify one source file, you have to recompile everything.
- eequah9L 8y ago
  Or a compile server / incremental compilation. Tom Tromey worked on supporting this in GCC years ago, and blogged about the roadblocks that he met on the way. I don't remember the details, but eventually the project was abandoned. It might still be interesting to read through this stuff--throw "tromey gcc compile server" at a search engine and see what comes up.
deleted 8y ago
[deleted]
archi42 8y ago
We do "kind of that": Our source is split across various cpp files to keep it well organized. Our build scripts then generates a single cpp file with ~30 #includes. Compilation of the 11Mb object file takes about 30s and requires at most 1GB memory (the lib is not that huge, 20kloc), but I think when we started on it, the build took a few minutes for the lib alone. So we save some dev time, but building all unit test executables still takes an additional 1m30s. So that's only a minor improvement. But I think the real gain is a much better optimization (the architecture of the lib is great to maintain and bugs are at least critical or might even have a lethal impact; there is a lot of potential for inlining/LTO).

Khoth 8y ago

The terms you're looking for are "single compilation unit" or "unity build". It's used sometimes, I think mostly to help the compiler optimise better. Build times for a full rebuild may be faster, but may not, since traditional builds can use many CPU cores. However, it stops incremental builds from working - if you modify one source file, you have to recompile everything.

eequah9L 8y ago
Or a compile server / incremental compilation. Tom Tromey worked on supporting this in GCC years ago, and blogged about the roadblocks that he met on the way. I don't remember the details, but eventually the project was abandoned. It might still be interesting to read through this stuff--throw "tromey gcc compile server" at a search engine and see what comes up.

deleted 8y ago

[deleted]

archi42 8y ago

We do "kind of that": Our source is split across various cpp files to keep it well organized. Our build scripts then generates a single cpp file with ~30 #includes. Compilation of the 11Mb object file takes about 30s and requires at most 1GB memory (the lib is not that huge, 20kloc), but I think when we started on it, the build took a few minutes for the lib alone. So we save some dev time, but building all unit test executables still takes an additional 1m30s. So that's only a minor improvement. But I think the real gain is a much better optimization (the architecture of the lib is great to maintain and bugs are at least critical or might even have a lethal impact; there is a lot of potential for inlining/LTO).