5 ms·
Unfortunately this messes with caching and causes the builder step to always rebuild if you’re using the default inline cache, until registries start supporting
by orf 4y ago
Unfortunately this messes with caching and causes the builder step to always rebuild if you’re using the default inline cache, until registries start supporting cache manifests.
- vidarh 4y agoHow so? I just tested a build, and it used the cache for every layer including the builder layers.
- orf 4y agoYou did this on the same machine, right? In a CI setting with no shared cache you need to rely on an OCI cache. The last build image is cached with the inline cache, but prior images are not
- claytonjy 4y agoI never got around to implementing it but I wonder how this plays with cross-runner caches in e.g. Gitlab, where the cache goes to S3; there's a cost to pulling the cache, so it'll never be as fast as same-machine, but should be way faster for most builds, right?
- orf 4y agoIf you're running the CI inside AWS, and assuming the code isn't doing anything stupid, it will be fast enough for nobody to notice.
- maaarghk 4y agothe cache is small but if you have a `docker buildx build --cache-from --push` type command it will always pull the image at the end and try to push it again (although it'll get layer already exists responses), for ~250mb images on gitlab I find this do-nothing job takes about 2.5 mins in total (vs a 10 min build if the entire cache were to be invalidated by a new base image version). I'd very much like it if I could say "if the entire build was cached don't bother pulling it at the end", maybe buildkit is the tool for that job
- claytonjy 4y agoI mostly love Docker and container-based CI but wow what a great reminder that even common-seeming workflows still have plenty of sharp edges!
- vidarh 4y agoAh, sorry I misunderstood you. Yes, I don't tend to care about whether or not the steps are cached in my CI setups as most of the Docker containers I work on build fast enough that it doesn't really matter to me, but that will of course matter for some.
- korijn 4y agoYou can build the first stage separately as a first step using `--target` and store the cache that way. No problem.
- orf 4y agoHow would you do this in a generic, reusable way company-wide for any Dockerfile? Given that you don't know the targets beforehand, the names, or even the number of stages. It is of course possible to do for a single project with a bit of effort: build each stage with a remote OCI cache source, push the cash there after. But... that sucks. What you want is the `max` cache type in buildkit[1]. Except... not much supports that yet. The native S3 cache would also be good once it stabalizes. 1. https://github.com/moby/buildkit#export-cache https://github.com/moby/buildkit#export-cache
- korijn 4y agoStandardize the stages and names. We use `dev` and `latest`. It worked wonders for us, on a cache hit the build time is reduced from 10 to 1,5 minutes.
- sequoia 4y agoA tiny bit more on this topic: https://sequoia.makes.software/reducing-docker-image-size-particularly-for-kubernetes-environments/#which-one-to-use-a-note-on-layer-caching https://sequoia.makes.software/reducing-docker-image-size-pa...
- iyn 4y agoThanks for sharing, very useful blog post (not just the linked section). Reference to https://github.com/wagoodman/dive https://github.com/wagoodman/dive will help a lot today.