3 ms·

and its "oh, you want multi-arch, do you?" friend. While prosecuting this <https://gitlab.com/gitlab-org/gitlab/-/issues/339567 https://gitlab.com/gitlab-org/gi

and its "oh, you want multi-arch, do you?" friend. While prosecuting this <https://gitlab.com/gitlab-org/gitlab/-/issues/339567 https://gitlab.com/gitlab-org/gitlab/-/issues/339567> I learned that they bundle binfmt into the buildkit docker image <https://github.com/moby/buildkit/blob/v0.12.5/Dockerfile#L110 https://github.com/moby/buildkit/blob/v0.12.5/Dockerfile#L11...> which itself is compiled against a static copy of qemu <https://github.com/tonistiigi/binfmt/blob/deploy/v6.2.0-26/Dockerfile#L8 https://github.com/tonistiigi/binfmt/blob/deploy/v6.2.0-26/D...> with various /proc/sys/fs/binfmt_misc wizardry <https://github.com/tonistiigi/binfmt/blob/deploy/v6.2.0-26/cmd/binfmt/main.go https://github.com/tonistiigi/binfmt/blob/deploy/v6.2.0-26/c...> We ended up switching to using multiarch/qemu-user-static:7.2.0-1 <https://hub.docker.com/layers/multiarch/qemu-user-static/7.2.0-1/images/sha256-fe60359c92e86a43cc87b3d906006245f77bfc0565676b80004cc666e4feb9f0?context=explore https://hub.docker.com/layers/multiarch/qemu-user-static/7.2...> which also mutates binfmt_misc in buildx's context in order to exec the static copy of qemu in it but I find their shell script much more legible about what's going on: <https://github.com/multiarch/qemu-user-static/blob/v7.2.0-1/containers/latest/register.sh#L14 https://github.com/multiarch/qemu-user-static/blob/v7.2.0-1/...> not to mention the fact that it more clearly represents the recent qemu version versus secreting it away inside the moby/buildkit:heheh-good-luck-friend They have an entire GH label for that: https://github.com/docker/buildx/labels/area%2Fqemu https://github.com/docker/buildx/labels/area%2Fqemu and my journey into the buildx sewers started from https://github.com/docker/buildx/issues/1170 https://github.com/docker/buildx/issues/1170

zamalek 3y ago
I see you're trying to do multi-arch builds. At reasonable (on bare metal) speeds you say? Delete the entire docker stack. It's utter shite. Then install Podman and Buildah, and rejoice! For all the evil buildx magic can now be done explicitly. This solution was born when trying to do native builds on CircleCI - where you do not have a buildx cluster with multiple architectures. You have to build the archs as independent build steps, then pull them together afterwards. This is something that Docker never considered, and Docker only functions on the happy path. Buildah has multiarch manipulation commands: https://danmanners.com/posts/2022-01-buildah-multi-arch/ https://danmanners.com/posts/2022-01-buildah-multi-arch/ (not a plug, first blog on search) We migrated to GHA, but still use separate builders to avoid the Docker trite. We also ditched DCT (Docker signing) for Cosign because - again - DCT is home-grown garbage.

zamalek 3y ago

I see you're trying to do multi-arch builds. At reasonable (on bare metal) speeds you say? Delete the entire docker stack. It's utter shite. Then install Podman and Buildah, and rejoice! For all the evil buildx magic can now be done explicitly. This solution was born when trying to do native builds on CircleCI - where you do not have a buildx cluster with multiple architectures. You have to build the archs as independent build steps, then pull them together afterwards. This is something that Docker never considered, and Docker only functions on the happy path. Buildah has multiarch manipulation commands: https://danmanners.com/posts/2022-01-buildah-multi-arch/ https://danmanners.com/posts/2022-01-buildah-multi-arch/ (not a plug, first blog on search) We migrated to GHA, but still use separate builders to avoid the Docker trite. We also ditched DCT (Docker signing) for Cosign because - again - DCT is home-grown garbage.