3 ms·
Yeah, I am actually aware— I've been following it keenly via the buildah issue tracker, and I should have put that in my original post. But the current reality
by mikepurvis 5y ago
Yeah, I am actually aware— I've been following it keenly via the buildah issue tracker, and I should have put that in my original post. But the current reality is still that building images on Kubernetes is a choice between several not-great options, especially if you're on managed k8s and can't use privileged mode as a stopgap.
Anyway, even once this stuff all lands, there's still actually no way to do what I would consider to be the actual gold standard of k8s image building, which would be a method where you build the image starting from any base layers already on the kubelet. Because currently, whether it's kaniko, buildah, docker-in-docker, etc, you're basically always either downloading everything every time, or you're having to manually manage some scheme with a long-lived cache container that you volume-in each time and purge periodically, for example: https://github.com/GoogleContainerTools/kaniko#caching-base-images https://github.com/GoogleContainerTools/kaniko#caching-base-...
In principle this should be possible with a Kaniko-like workflow, but you'd need a separate control pod / build pod setup, where the control pod would compute all the layer hashes and then repeatedly try to spawn from the bottom-up using `imagePullPolicy: never` until one of them succeeded, and then build the remainder of the container from there.
- znpy 5y agoI feel your pain, as it's my pain too. Right now we have a pool of gitlab runners running as dedicated virtual machines but on the to-do list I have to test whether rootless podman + gitlab runner cache + $DOCKER_HOST pointing to podman's sock file can let developers use plain old docker-compose and the general docker tooling... Within a dumb pod in a kubernetes clusters, with all the bells and whistles (especially cluster autoscaling). A man can dream. Edit: we are a bit advantaged because we run on openstack, run our own registry (harbor) and our cloud provider doesn't charge us for bandwidth...
- mikepurvis 5y agoI'm a huge fan of the kubernetes runner for GitLab CI, but I really wish there was a way to just to `image: path/in/repo/to/Dockerfile` and have it do the right thing as far as creating/using the supplied image.
- deastman 5y agoCan you open an issue in the gitlab-runner project? It would be great if one of our engineers that focuses on the Kubernetes executor can weigh in. https://gitlab.com/gitlab-org/gitlab-runner/-/issues https://gitlab.com/gitlab-org/gitlab-runner/-/issues
- mikepurvis 5y agoSure, done: https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28195 https://gitlab.com/gitlab-org/gitlab-runner/-/issues/28195
- rhatdan 5y agoYou can volume mount in the host container storage into the container using additional stores. In this article, note sections on /var/lib/shared https://developers.redhat.com/blog/2019/08/14/best-practices-for-running-buildah-in-a-container# https://developers.redhat.com/blog/2019/08/14/best-practices...