3 ms·

The debian build system already runs `make -j$(nproc)` whenever the package supports it. So in principle sometimes I'm running 1024 (32*32) concurrent compilati

by moyix 5y ago

The debian build system already runs `make -j$(nproc)` whenever the package supports it. So in principle sometimes I'm running 1024 (32*32) concurrent compilations, but it's rare for that many to actually be active at once (most builds have large portions that can't be parallelized, like running ./configure or tarring up the build artifacts at the end) – right now I have 280 processes running in 32 containers. Not using ccache/distcc since I'm just running this on my own workstation. I'm not sure ccache would help here because each build runs in an isolated container environment, to prevent cross-contamination with other builds. I'm also instrumenting each build using bear to generate a compilation database, and I think distcc would mess with that? (I can't imagine distcc knows how to transfer over the bear wrapper binaries / shared libs?)

0xbadcafebee 5y ago
Well the idea with ccache/distcc is to store the cache on the host or a shared volume and reference it from your containers. Your container builds remain isolated but don't waste time rebuilding objects that were already compiled (and re-compiling obvs gets much faster). Assuming same architecture, compiler, linker, etc I think it won't affect the isolated-build-ness. Here's some examples: https://cinaq.com/blog/2020/05/10/speed-up-docker-builds-with-distcc-ccache-and-kubernetes/ https://cinaq.com/blog/2020/05/10/speed-up-docker-builds-wit... https://stackoverflow.com/questions/39650056/using-ccache-when-building-inside-of-docker https://stackoverflow.com/questions/39650056/using-ccache-wh... To be honest it's been... a very long time... since I've used these. But the theory is that it all "just works", in that if it can compile something with cache or remotely, it does, but if not, it just works as normal. So I think it would work for your use case? Maybe a Gentoo user can correct me :)

by moyix 5y ago

0xbadcafebee 5y ago

Well the idea with ccache/distcc is to store the cache on the host or a shared volume and reference it from your containers. Your container builds remain isolated but don't waste time rebuilding objects that were already compiled (and re-compiling obvs gets much faster). Assuming same architecture, compiler, linker, etc I think it won't affect the isolated-build-ness. Here's some examples: https://cinaq.com/blog/2020/05/10/speed-up-docker-builds-with-distcc-ccache-and-kubernetes/ https://cinaq.com/blog/2020/05/10/speed-up-docker-builds-wit... https://stackoverflow.com/questions/39650056/using-ccache-when-building-inside-of-docker https://stackoverflow.com/questions/39650056/using-ccache-wh... To be honest it's been... a very long time... since I've used these. But the theory is that it all "just works", in that if it can compile something with cache or remotely, it does, but if not, it just works as normal. So I think it would work for your use case? Maybe a Gentoo user can correct me :)