5 ms·

Here’s a little known fact: “docker build” can trivially be extended to build buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles are

by zapita 8y ago

Here’s a little known fact: “docker build” can trivially be extended to build buildpacks or CNB. Now that the buildkit refactoring is complete, Dockerfiles are just the default frontend. There’s already a buildpack frontend in the community repo, and it works great. Writing your own frontend is real straightforward. Honestly after years of stagnation, the most exciting work on container building is now coming out of Docker. Buildkit is amazing, a real hidden gem. See https://github.com/moby/buildkit https://github.com/moby/buildkit

ImJasonH 8y ago
Link to the buildpack frontend for docker: https://github.com/tonistiigi/buildkit-pack https://github.com/tonistiigi/buildkit-pack
molszanski 8y ago
Thank you for sharing this!
bshipp 8y ago
What the heck, how is it that I haven't seen these yet? Maybe one of the downfalls of becoming comfortable enough with the syntax that I'm not having to look up documentation on the docker website anymore is that I'm more and more out of the loop on this stuff. Thanks for posting that!
meowface 8y ago
My favorite Docker BuildKit feature is SSH agent forwarding. Add "--mount=type=ssh" after RUN commands in Dockerfiles and the command will use your host machine's SSH agent. I've been able to greatly simplify a lot of Dockerfiles and CI build processes using it. There's a good introduction here: https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 https://medium.com/@tonistiigi/build-secrets-and-ssh-forward...
- theptip 8y ago
  Oh, that's nice -- that's been a pain point for a long time, and it seemed like there was a philosophical argument against making the Dockerfile's behaviour context-dependent like this.
  - meowface 8y ago
    You do have to pass an extra "--ssh default" argument to "docker build", so it's not totally automagical or anything.
- bsilvereagle 8y ago
  This feature currently does not work with the OS X ssh agent: https://github.com/docker/for-mac/issues/410 https://github.com/docker/for-mac/issues/410
  - auvrw 8y ago
    ssh access at build time via buildkit works on os x https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 https://medium.com/@tonistiigi/build-secrets-and-ssh-forward... the link you've supplied regards ssh access when running docker images, not when building them. ... i did notice that for lots of downloads, it is somewhat slower, or perhaps more prone to lag, than authentication from inside the running image.
sclevine 8y ago
While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVEs). It will stop working with new buildpack versions entirely when Cloud Foundry drops support for Trusty. Implementing CNBs as a buildkit frontend would break key security and performance features. For instance, CNBs can build images in unprivileged containers without any extra capabilities, which buildkit cannot do. CNBs can also patch images by manipulating their manifests directly on a remote Docker registry. This means that image rebuilds in a fresh VM or container can reuse layers from a previous build without downloading them (just metadata about them), and base images can be patched for many images simultaneously with near-zero data transfer (as long as a copy of the new base image is available on the registry). As far as I know, buildkit can't do any of that yet. That said, we do plan on using buildkit (once it ships with Docker by default) to optimize the CNB pack CLI when you build images without publishing them to a Docker registry. It's a huge improvement over the current Docker daemon implementation for sure! [1] https://github.com/buildpack/packs https://github.com/buildpack/packs
- zapita 8y ago
  Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t support it since it’s based on containerd. I think it’s a mistake not to jump on the buildkit/docker-build bandwagon. You would get a 10x larger ecosystem overnight, basically for free. Instead it seems like you’re betting on CNB as a way to “kill” Dockerfiles. But users don’t actually want to kill anything, they want their stuff to continue working. Without a good interop story, you’re pretty much guaranteeing that CNB will not get traction outside of the Pivotal ecosystem. Seems like a shame to me.
  - 8y ago
jacques_chester 8y ago
When we started, buildkit was still experimental. Personally I'd like for `pack` to be able to jettison all the code that has to deal with the Docker daemon, it's a bit of a PITA, and buildkit looks to have significant improvements in at least that area. I'd note that the frontend there is for "v2b" buildpacks -- the previous generation of Cloud Foundry buildpacks. The CNB lifecycle has changed a fair amount from the v2a (Heroku) and v2b designs.
whalesalad 8y ago
At the end of the day a docker image (or qemu, firecracker, etc...) is just a tarballed root filesystem. The funny thing to me is how seemingly overly complex the ecosystem has become for turning a blueprint for a Linux system into a tarball of files and folders. What am I missing?
- bboreham 8y ago
  Layers?
  - guiriduro 8y ago
    But aren't they just an implementation detail, albeit a useful one? You'd find copy-on-write deltas in modern snapshotting filesystems too.
    - bboreham 8y ago
      Parent was comparing against a tarball, not a snapshotting filesystem.
    - jacques_chester 8y ago
      > But aren't they just an implementation detail, albeit a useful one? Part of what we do in Cloud Native Buildpacks is to use the layer abstraction more aggressively to make image updating more efficient. That requires taking care with the ordering and contents of each layer, so that they can be replaced individually. Putting it another way: we don't see the image as the unit of work and distribution. We're focused on layers as the central concept.
  - whalesalad 8y ago
    Layers are each a tarball, extracted one by one on top of one another to get you to the end result.
    - RantyDave 8y ago
      So docker untars a filesystem which contains a file "foo" and calls this layer "1". Then it encounters a "RUN some_installer_thing" which removes file "foo" and calls this layer "2". If you just untar the layers on top of each other, foo will still be there. This is a problem, no?

by zapita 8y ago

ImJasonH 8y ago

Link to the buildpack frontend for docker: https://github.com/tonistiigi/buildkit-pack https://github.com/tonistiigi/buildkit-pack

molszanski 8y ago

Thank you for sharing this!

bshipp 8y ago

What the heck, how is it that I haven't seen these yet? Maybe one of the downfalls of becoming comfortable enough with the syntax that I'm not having to look up documentation on the docker website anymore is that I'm more and more out of the loop on this stuff. Thanks for posting that!

meowface 8y ago

My favorite Docker BuildKit feature is SSH agent forwarding. Add "--mount=type=ssh" after RUN commands in Dockerfiles and the command will use your host machine's SSH agent. I've been able to greatly simplify a lot of Dockerfiles and CI build processes using it. There's a good introduction here: https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 https://medium.com/@tonistiigi/build-secrets-and-ssh-forward...

theptip 8y ago
Oh, that's nice -- that's been a pain point for a long time, and it seemed like there was a philosophical argument against making the Dockerfile's behaviour context-dependent like this.
- meowface 8y ago
  You do have to pass an extra "--ssh default" argument to "docker build", so it's not totally automagical or anything.
bsilvereagle 8y ago
This feature currently does not work with the OS X ssh agent: https://github.com/docker/for-mac/issues/410 https://github.com/docker/for-mac/issues/410
- auvrw 8y ago
  ssh access at build time via buildkit works on os x https://medium.com/@tonistiigi/build-secrets-and-ssh-forwarding-in-docker-18-09-ae8161d066 https://medium.com/@tonistiigi/build-secrets-and-ssh-forward... the link you've supplied regards ssh access when running docker images, not when building them. ... i did notice that for lots of downloads, it is somewhat slower, or perhaps more prone to lag, than authentication from inside the running image.

sclevine 8y ago

While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVEs). It will stop working with new buildpack versions entirely when Cloud Foundry drops support for Trusty. Implementing CNBs as a buildkit frontend would break key security and performance features. For instance, CNBs can build images in unprivileged containers without any extra capabilities, which buildkit cannot do. CNBs can also patch images by manipulating their manifests directly on a remote Docker registry. This means that image rebuilds in a fresh VM or container can reuse layers from a previous build without downloading them (just metadata about them), and base images can be patched for many images simultaneously with near-zero data transfer (as long as a copy of the new base image is available on the registry). As far as I know, buildkit can't do any of that yet. That said, we do plan on using buildkit (once it ships with Docker by default) to optimize the CNB pack CLI when you build images without publishing them to a Docker registry. It's a huge improvement over the current Docker daemon implementation for sure! [1] https://github.com/buildpack/packs https://github.com/buildpack/packs

zapita 8y ago
Your answer makes sense, but it actually makes me less excited about CNB. It sounds like CNB will break compatibility with the massive Dockerfile ecosystem, in exchange for... sometimes not downloading a layer? That is not appealing to me at all, because Dockerfiles are too embedded in my workflow, losing support for them is simply not an option. As for unprivileged builds, I don’t see any reason buildkit can’t support it since it’s based on containerd. I think it’s a mistake not to jump on the buildkit/docker-build bandwagon. You would get a 10x larger ecosystem overnight, basically for free. Instead it seems like you’re betting on CNB as a way to “kill” Dockerfiles. But users don’t actually want to kill anything, they want their stuff to continue working. Without a good interop story, you’re pretty much guaranteeing that CNB will not get traction outside of the Pivotal ecosystem. Seems like a shame to me.
- 8y ago

jacques_chester 8y ago

When we started, buildkit was still experimental. Personally I'd like for `pack` to be able to jettison all the code that has to deal with the Docker daemon, it's a bit of a PITA, and buildkit looks to have significant improvements in at least that area. I'd note that the frontend there is for "v2b" buildpacks -- the previous generation of Cloud Foundry buildpacks. The CNB lifecycle has changed a fair amount from the v2a (Heroku) and v2b designs.

whalesalad 8y ago

At the end of the day a docker image (or qemu, firecracker, etc...) is just a tarballed root filesystem. The funny thing to me is how seemingly overly complex the ecosystem has become for turning a blueprint for a Linux system into a tarball of files and folders. What am I missing?

bboreham 8y ago
Layers?
- guiriduro 8y ago
  But aren't they just an implementation detail, albeit a useful one? You'd find copy-on-write deltas in modern snapshotting filesystems too.
  - bboreham 8y ago
    Parent was comparing against a tarball, not a snapshotting filesystem.
  - jacques_chester 8y ago
    > But aren't they just an implementation detail, albeit a useful one? Part of what we do in Cloud Native Buildpacks is to use the layer abstraction more aggressively to make image updating more efficient. That requires taking care with the ordering and contents of each layer, so that they can be replaced individually. Putting it another way: we don't see the image as the unit of work and distribution. We're focused on layers as the central concept.
- whalesalad 8y ago
  Layers are each a tarball, extracted one by one on top of one another to get you to the end result.
  - RantyDave 8y ago
    So docker untars a filesystem which contains a file "foo" and calls this layer "1". Then it encounters a "RUN some_installer_thing" which removes file "foo" and calls this layer "2". If you just untar the layers on top of each other, foo will still be there. This is a problem, no?