5 ms·
Several hour builds? I have to ask what kind of app this is for, because I haven’t ever had a build take more than 2 minutes. I’ve been using docker heavily in
by cutety 8y ago
Several hour builds? I have to ask what kind of app this is for, because I haven’t ever had a build take more than 2 minutes. I’ve been using docker heavily in CI/dev for about 2ish years and the past couple months been getting some production stuff going.
My first thought since you say you’re having issues with build caching, you might want to look into multi-stage builds[1], if you haven’t already. I haven’t gotten any “heavy” use from these since I mostly do web stuff in non compiled languages, so I can’t benefit from this feature as much as a compiled language than can just copy binaries around. But, it has been useful for like compiling JS, since it’s done in a separate stage, I a) don’t ever have to contaminate my final image with a node_modules at any point and b) can rebuild the image and used the cached stage if there are no changes in the front end, and that’s usually the longest parts of my builds anyway.
However, if you’re in a compiled language (which from the sounds of it, I have a feeling you might be — long builds and multiple dependent components that have to be rebuilt), you can compile some core component in the first stage (A), then have multiple stages (B, C) that depend on that component’s compiled files, and then maybe a final stage (D) that depends on B & C. So then, if you make changes to just C, you’ll only have to rebuild the C & D stages, while the other two are cached. Each stage itself is just an intermediary image, so you can use them as you would the final, and you can even tell Docker to just build up to stage B, if you only need that image.
[1] https://docs.docker.com/develop/develop-images/multistage-build/ https://docs.docker.com/develop/develop-images/multistage-bu...
- etaioinshrdlu 8y agoAhh yes -- some of our dependencies are huge compiled C++ projects that need some customization. We're starting to use multi stage builds here and there. Seems like an essential building block. The main pain point seems to be knowing when an image needs to be rebuilt and to ensure any dependent images also get rebuilt.