4 ms·

During the last 3 years I've had the pleasure of using Bazel's rules_docker to generate all my container images (https://github.com/bazelbuild/rules_docker http

During the last 3 years I've had the pleasure of using Bazel's rules_docker to generate all my container images (https://github.com/bazelbuild/rules_docker https://github.com/bazelbuild/rules_docker). In a nutshell, rules_docker is a set of build rules for the Bazel build system (https://bazel.build https://bazel.build). What's pretty nice about these rules is that they don't rely on a Docker daemon. They are rules that directly construct image tarballs that you can either load into your local Docker daemon or push to a registry. What's nice about this approach is that image generation works on any operating system. For example, even on a Mac or Windows system that doesn't have Docker installed, you're able to build Linux containers. They are also fully reproducible, meaning that you often don't need to upload layers when pushing (either because they haven't changed, or because some colleague/CI job already pushed those layers). I guess rules_docker works fine for a variety of programming languages. I've mainly used it with Go, though.

kitd 6y ago
Sounds similar to podman [1] which also doesn't require a daemon (though you can run it with one if you like). [1] https://podman.io/getting-started/ https://podman.io/getting-started/ Back to the original article, I prefer dockerfile-based builds as it allows me to run my own tests easily on the image locally before uploading. Can this be done with buildkits?
- dfrey1 6y ago
  There is ongoing work with Buildpacks (specifically `pack`, the project CLI), to integrate better with podman, and enable a daemon-less workflow. We most often build it with the local docker daemon (though you can with a remote docker daemon), test it, and then publish the image.
tikkabhuna 6y ago
We've used Google Jib for Docker for the same reasons. It also means we don't have to think about the layout of our images. I know that if we went with Dockerfiles each team would put application code in different directories. There would be no consistency. Jib hides that away from us.
- jpgvm 6y ago
  Jib is also great because it integrates into your existing build system (Bazel is a big lift for most projects, especially JVM ones) and takes care of the important parts of building a good JVM image like separating the dependencies into a different layer etc. Also it doesn't require a Docker daemon to build and push the image which is a huge win.
- e12e 6y ago
  https://github.com/GoogleContainerTools/jib#what-is-jib https://github.com/GoogleContainerTools/jib#what-is-jib > What is Jib? > Jib builds optimized Docker and OCI images for your Java applications Funny. Here I thought you could just build and deploy a jar if you were committed to writing Java... I suppose it does make sense that "all things come to docker/containers", though. Allowing both golang and Java services to be deployed in the same manner.
  - jacques_chester 6y ago
    My view is that if you're an all-Java shop and have mature infrastructure based on JARs and WARs, stick with it. That technology is time-tested and reliable. But if you have a mixture of technologies, the equation begins to shift towards using container images for everything.
    - tikkabhuna 6y ago
      100%. Coming up with a common pattern for deploying multiple languages is a nightmare for us. Different groups also have different patterns, all internal. Using Docker, it doesn't matter if its Go, Python, Java, you start it with the same commands. Running in a Docker container also means you have the same platform for local development and each environment. It also means that you can incrementally upgrade the Java version each application runs on. Not so bad for Java, but for something like Python it makes it easy and consistent.
kevincox 6y ago
I use Nix to do something similar: https://gitlab.com/kevincox/feed-test/-/blob/8b217add5fc11bbb8ee6aae480fca3b501abcbae/default.nix#L13-24 https://gitlab.com/kevincox/feed-test/-/blob/8b217add5fc11bb...
- bfrog 6y ago
  Same, with cross compiles! Its wonderful
an_d_rew 6y ago
We use Bazel’s rules_docker as well, and I would caution someone evaluating it with a note from out experience. What Bazel does well (and as well as Bazel fits your use-case) Bazel does extremely well and is a reproducible joy to use. But if you stray off that path even a tiny bit, you’re often in for a surprisingly inexplicable, unavoidable, far-reaching pain. For example, rules_docker is amazing at laying down files in a known base image. Everything is timestamped to the 1970 unix epoch, for reproducibility, but hey, it’s a bit-perfect reproduction. Need to run one teensy executable to do even the smallest thing that’s trivial with a Dockerfile? Bazel mist create the image transfer it to a docker daemon, run the command, transfer it back... your 1 kb change just took 5 minutes and 36 gb of data being tarred, gzipped, and flung around (hopefully-the-local) network. It may not be a dealbreaker, and you may not care, but be forewarned that these little surprises creep up fairly ofen from unexpected quarters!
- MuffinFlavored 6y ago
  > But if you stray off that path even a tiny bit, you’re often in for a surprisingly inexplicable, unavoidable, far-reaching pain. Not to go off on a tangent, but I feel like this is true for most packages/alternatives to mainstream solutions (at least in node.js)
kkapelon 6y ago
> What's pretty nice about these rules is that they don't rely on a Docker daemon You can also use podman, img, kaniko, jib and several other tools that don't need a docker daemon No need to adopt a whole build system (Bazel) just to simplify the way you create artifacts.
- EdSchouten 6y ago
  Note that I didn't just adopt the build system to build container images. There are also lots of other build steps that I wanted to do (compile Protobuf & gRPC definitions, build a frontend web application and build it into the binary, etc.). If I already want to do stuff like that and Bazel does that well for me, then at least in my case there was no incentive to use any of the tools you listed.
mikepurvis 6y ago
I'm a huge fan of offline generation of these kinds of things across the board— one that I'm found surprisingly tricky is OVAs. You can create your rootfs in a loopback, and use qemu-img to turn that into a vmdk, but there's not a lot of documentation or helpers/validators out there to assist with producing the XML ovf metadata, and if you get it wrong in any way, vmware just coughs up a generic error message. One thing that is quite helpful as an example to pick through is the 40MB yVM from here: https://cloudarchitectblog.wordpress.com/2015/11/11/yvm-download-page/ https://cloudarchitectblog.wordpress.com/2015/11/11/yvm-down... Anyone else got tips or resources for this?
jacques_chester 6y ago
> What's pretty nice about these rules is that they don't rely on a Docker daemon. Neither do Cloud Native Buildpacks. The lifecycle can run as a regular process inside a container. The `pack` CLI tool uses docker as a convenient way to run those containers on a local environment. But if something else creates the containers, that's fine, the lifecycle works exactly the same. FWIW there is a lot of cross-pollination of ideas between ko, Jib, rules_docker and CNB folks.

kitd 6y ago

Sounds similar to podman [1] which also doesn't require a daemon (though you can run it with one if you like). [1] https://podman.io/getting-started/ https://podman.io/getting-started/ Back to the original article, I prefer dockerfile-based builds as it allows me to run my own tests easily on the image locally before uploading. Can this be done with buildkits?

dfrey1 6y ago
There is ongoing work with Buildpacks (specifically `pack`, the project CLI), to integrate better with podman, and enable a daemon-less workflow. We most often build it with the local docker daemon (though you can with a remote docker daemon), test it, and then publish the image.

tikkabhuna 6y ago

We've used Google Jib for Docker for the same reasons. It also means we don't have to think about the layout of our images. I know that if we went with Dockerfiles each team would put application code in different directories. There would be no consistency. Jib hides that away from us.

jpgvm 6y ago
Jib is also great because it integrates into your existing build system (Bazel is a big lift for most projects, especially JVM ones) and takes care of the important parts of building a good JVM image like separating the dependencies into a different layer etc. Also it doesn't require a Docker daemon to build and push the image which is a huge win.
e12e 6y ago
https://github.com/GoogleContainerTools/jib#what-is-jib https://github.com/GoogleContainerTools/jib#what-is-jib > What is Jib? > Jib builds optimized Docker and OCI images for your Java applications Funny. Here I thought you could just build and deploy a jar if you were committed to writing Java... I suppose it does make sense that "all things come to docker/containers", though. Allowing both golang and Java services to be deployed in the same manner.
- jacques_chester 6y ago
  My view is that if you're an all-Java shop and have mature infrastructure based on JARs and WARs, stick with it. That technology is time-tested and reliable. But if you have a mixture of technologies, the equation begins to shift towards using container images for everything.
  - tikkabhuna 6y ago
    100%. Coming up with a common pattern for deploying multiple languages is a nightmare for us. Different groups also have different patterns, all internal. Using Docker, it doesn't matter if its Go, Python, Java, you start it with the same commands. Running in a Docker container also means you have the same platform for local development and each environment. It also means that you can incrementally upgrade the Java version each application runs on. Not so bad for Java, but for something like Python it makes it easy and consistent.

kevincox 6y ago

I use Nix to do something similar: https://gitlab.com/kevincox/feed-test/-/blob/8b217add5fc11bbb8ee6aae480fca3b501abcbae/default.nix#L13-24 https://gitlab.com/kevincox/feed-test/-/blob/8b217add5fc11bb...

bfrog 6y ago
Same, with cross compiles! Its wonderful

an_d_rew 6y ago

We use Bazel’s rules_docker as well, and I would caution someone evaluating it with a note from out experience. What Bazel does well (and as well as Bazel fits your use-case) Bazel does extremely well and is a reproducible joy to use. But if you stray off that path even a tiny bit, you’re often in for a surprisingly inexplicable, unavoidable, far-reaching pain. For example, rules_docker is amazing at laying down files in a known base image. Everything is timestamped to the 1970 unix epoch, for reproducibility, but hey, it’s a bit-perfect reproduction. Need to run one teensy executable to do even the smallest thing that’s trivial with a Dockerfile? Bazel mist create the image transfer it to a docker daemon, run the command, transfer it back... your 1 kb change just took 5 minutes and 36 gb of data being tarred, gzipped, and flung around (hopefully-the-local) network. It may not be a dealbreaker, and you may not care, but be forewarned that these little surprises creep up fairly ofen from unexpected quarters!

MuffinFlavored 6y ago
> But if you stray off that path even a tiny bit, you’re often in for a surprisingly inexplicable, unavoidable, far-reaching pain. Not to go off on a tangent, but I feel like this is true for most packages/alternatives to mainstream solutions (at least in node.js)

kkapelon 6y ago

> What's pretty nice about these rules is that they don't rely on a Docker daemon You can also use podman, img, kaniko, jib and several other tools that don't need a docker daemon No need to adopt a whole build system (Bazel) just to simplify the way you create artifacts.

EdSchouten 6y ago
Note that I didn't just adopt the build system to build container images. There are also lots of other build steps that I wanted to do (compile Protobuf & gRPC definitions, build a frontend web application and build it into the binary, etc.). If I already want to do stuff like that and Bazel does that well for me, then at least in my case there was no incentive to use any of the tools you listed.

mikepurvis 6y ago

I'm a huge fan of offline generation of these kinds of things across the board— one that I'm found surprisingly tricky is OVAs. You can create your rootfs in a loopback, and use qemu-img to turn that into a vmdk, but there's not a lot of documentation or helpers/validators out there to assist with producing the XML ovf metadata, and if you get it wrong in any way, vmware just coughs up a generic error message. One thing that is quite helpful as an example to pick through is the 40MB yVM from here: https://cloudarchitectblog.wordpress.com/2015/11/11/yvm-download-page/ https://cloudarchitectblog.wordpress.com/2015/11/11/yvm-down... Anyone else got tips or resources for this?

jacques_chester 6y ago

> What's pretty nice about these rules is that they don't rely on a Docker daemon. Neither do Cloud Native Buildpacks. The lifecycle can run as a regular process inside a container. The `pack` CLI tool uses docker as a convenient way to run those containers on a local environment. But if something else creates the containers, that's fine, the lifecycle works exactly the same. FWIW there is a lot of cross-pollination of ideas between ko, Jib, rules_docker and CNB folks.