5 ms·

Maybe someone can help me out on a question I've not got a clear answer on: is it better to build your artefact (jar file in this case) and put that in the dock

Maybe someone can help me out on a question I've not got a clear answer on: is it better to build your artefact (jar file in this case) and put that in the docker image, or is it better to put everything you need in the image (source files, dependencies etc) then build the artefact from within the container? Seems to me like the first approach (compile outside then put artefact in container) is the most sane/sensible approach and the one used here in Jib, but I have seen people advocating the build-in-container approach as better (self-contained, repeatable and so on). Any benefits to the build-in-container approach that are compelling?

organsnyder 8y ago
We run our builds within containers (using the Jenkins k8s plugin) for repeatability, but the generated artifacts (jars and Docker images) are pushed to our artifact repository (Artifactory) as part of the build process. Our final deployable artifact is our base image with a single layer added to include our app.jar.
paulddraper 8y ago
Building in a container gives you the portability benefit of having tools installed in a container. But not building in your productiom container makes it smaller. Dockerfiles can actually do both, via "mutli-stage builds". You can build in one container and in the same Dockerfiles copy files from that build container to your final container. Some take a really extreme stance where the production container has nothing superfluous, not even a shell. https://github.com/GoogleContainerTools/distroless https://github.com/GoogleContainerTools/distroless
bzajax 8y ago
Docker recently has support for multi stage builds. So you can use one Dockerfile to define how to build your image (produce your jar and necessary other resources) and then copy them into a fresh container running off a more lightweight image that might not contain all the intermediates and your build tools. https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds https://docs.docker.com/develop/develop-images/multistage-bu...
- zokier 8y ago
  Multi-stage builds is definitely my preferred method of building stuff, but the problem I've found is that caching dependencies is bit of a pain, either you need to do some tricks with volumes or your builds will be very slow.
  - kinghajj 8y ago
    At work, I setup the build system to use Apache Archiva to cache Java dependencies, along with other services like apt-cacher, devpi, and squid, for Debian, Python, and general HTTP caching, respectively. We build all of our images atop the "bitnami/minideb" base, and split the Dockerfiles into "partials" that source a dynamic parent image via a build argument. For example, the partial that sets up Debian package and HTTP caching: ARG base FROM ${base} ARG apt_cacher_host ARG http_proxy ARG https_proxy ARG no_proxy ENV HTTP_PROXY="${http_proxy}" ENV HTTPS_PROXY="${https_proxy}" ENV NO_PROXY="${no_proxy}" ENV http_proxy="${http_proxy}" ENV https_proxy="${https_proxy}" ENV no_proxy="${no_proxy}" RUN set -ex; \ if [ ! -z "${apt_cacher_host}" ]; then \ echo "Acquire::http::Proxy \"${apt_cacher_host}\";Acquire::https::Proxy \"false\";" >/etc/apt/apt.conf.d/01proxy; \ fi; \ For Java, another partial creates a Maven settings file with the Apache Archiva URL: ARG base FROM ${base} ARG maven_mirror COPY devops/image-partials/java-artifacts/settings.xml /tmp/settings.xml RUN set -ex; \ env; \ if [ ! -z "${maven_mirror}" ]; then \ sed -r "s!%%MAVEN_MIRROR%%!${maven_mirror}!g;" /tmp/settings.xml >/usr/share/maven/conf/settings.xml; \ fi; The important part of settings.xml is this section: <mirrors> <mirror> <id>archiva.default</id> <url>%%MAVEN_MIRROR%%</url> <mirrorOf>external:*</mirrorOf> </mirror> </mirrors> And voila, Maven will go through Archiva for all its dependencies, which are transparently cached so that subsequent builds don't need to re-download everything again.
- jacques_chester 8y ago
  > Docker recently has support for multi stage builds. I view these as a future antipattern. They bring what is properly CI's work inside the Docker daemon, which makes it a larger attack surface (many folks run PRs sight-unseen) and obscures it from outside tooling. They only exist because of the limits of the Dockerfile format. With something like Jib, that disappears.
humbleMouse 8y ago
Everything your jar contains can be stored in a git-hub repo with the project. Then you build a fat jar and put it in the image.
- zokier 8y ago
  I would advice against using fat jars with docker. They do not layer nicely, so your layer sizes become much bigger than necessary, slowing everything down. If you have multiple applications then with fat jars you lose the ability to share the common bits in the applications, which could also have impact on maintainability.
  - humbleMouse 8y ago
    I would argue that fat jars make everything simple and portable. Especially if you do performance testing frequently on hardware that isn't running docker. You can compare performance baremetal vs. VM's this way. Furthermore, as log as you use gradle/maven with artifactory then maintainability of fat jar artifacts is pretty simple.
peterwwillis 8y ago
I would do both. Make a build container that pulls in the Git hash that you want to build/test against. You can keep your dependencies, build process, unit and func tests in the container. Your dev team can use the same container image to build and test in the same way that you do, pulling in the latest code from Git and running a docker build. Now nobody needs to set up their environment because it's all just in the Docker image in the same way. You don't need all those build and test files in production, so you can build a new "deploy container" with just the artifacts from your build process. I would also record the versions of your dependencies, as well as the SHA hash of the build/test container you used, and the SHA hash of the Git repos you pulled from. You can also set things like flyway migrations to run each time the container runs. Now you can deploy this second container somewhere. You can then use the same build/test container you used before to run func tests against the deployed container, to double check it works in your deployed environment.
- hinkley 8y ago
  The former can be handily taken care of in your CI toolchain. You need a build agent with all of the CLI tools you need for your build. Many support Docker these days. Some can even spin up images on demand.
  - peterwwillis 8y ago
    This is (for me) the best part of keeping your build process in a container: reducing the complexity of the build agent. The only software dependencies on your build agent are Docker and Git. Your CI/CD job pulls a Git repo with a Dockerfile, and runs 'docker build', and that pulls every dependency it needs.
    - hinkley 8y ago
      Agreed, and I can’t wait to work someplace like that again. We have a build agent with 9 versions of Node in it. It was 6 when I started here. Once a month I get to help someone figure out weird errors because one task has the wrong value in a drop down. It’s such busy work I can’t even. When I run the CI system, shit gets done and people are expected to be able to recreate build failures by themselves. Give a man a tool, make it reliable, then expect him to use it.
rifung 8y ago
> Any benefits to the build-in-container approach that are compelling? The build in container approach makes it so that in theory you have reproducible builds too from any Docker supported machine. I personally like this approach because it makes it so that you don't run into the case where "It works on my machine" when testing locally.
adrianmsmith 8y ago
I think Java has such good build tooling that you don’t need to build in a container for reproducible builds. Other languages might be different. Install the JDK, install Maven, check out the project, “mvn package” and it’ll download what it needs and produce the JAR to be copied into the Docker image by the Dockerfile.

organsnyder 8y ago

We run our builds within containers (using the Jenkins k8s plugin) for repeatability, but the generated artifacts (jars and Docker images) are pushed to our artifact repository (Artifactory) as part of the build process. Our final deployable artifact is our base image with a single layer added to include our app.jar.

paulddraper 8y ago

Building in a container gives you the portability benefit of having tools installed in a container. But not building in your productiom container makes it smaller. Dockerfiles can actually do both, via "mutli-stage builds". You can build in one container and in the same Dockerfiles copy files from that build container to your final container. Some take a really extreme stance where the production container has nothing superfluous, not even a shell. https://github.com/GoogleContainerTools/distroless https://github.com/GoogleContainerTools/distroless

bzajax 8y ago

Docker recently has support for multi stage builds. So you can use one Dockerfile to define how to build your image (produce your jar and necessary other resources) and then copy them into a fresh container running off a more lightweight image that might not contain all the intermediates and your build tools. https://docs.docker.com/develop/develop-images/multistage-build/#use-multi-stage-builds https://docs.docker.com/develop/develop-images/multistage-bu...

zokier 8y ago
Multi-stage builds is definitely my preferred method of building stuff, but the problem I've found is that caching dependencies is bit of a pain, either you need to do some tricks with volumes or your builds will be very slow.
- kinghajj 8y ago
  At work, I setup the build system to use Apache Archiva to cache Java dependencies, along with other services like apt-cacher, devpi, and squid, for Debian, Python, and general HTTP caching, respectively. We build all of our images atop the "bitnami/minideb" base, and split the Dockerfiles into "partials" that source a dynamic parent image via a build argument. For example, the partial that sets up Debian package and HTTP caching: ARG base FROM ${base} ARG apt_cacher_host ARG http_proxy ARG https_proxy ARG no_proxy ENV HTTP_PROXY="${http_proxy}" ENV HTTPS_PROXY="${https_proxy}" ENV NO_PROXY="${no_proxy}" ENV http_proxy="${http_proxy}" ENV https_proxy="${https_proxy}" ENV no_proxy="${no_proxy}" RUN set -ex; \ if [ ! -z "${apt_cacher_host}" ]; then \ echo "Acquire::http::Proxy \"${apt_cacher_host}\";Acquire::https::Proxy \"false\";" >/etc/apt/apt.conf.d/01proxy; \ fi; \ For Java, another partial creates a Maven settings file with the Apache Archiva URL: ARG base FROM ${base} ARG maven_mirror COPY devops/image-partials/java-artifacts/settings.xml /tmp/settings.xml RUN set -ex; \ env; \ if [ ! -z "${maven_mirror}" ]; then \ sed -r "s!%%MAVEN_MIRROR%%!${maven_mirror}!g;" /tmp/settings.xml >/usr/share/maven/conf/settings.xml; \ fi; The important part of settings.xml is this section: <mirrors> <mirror> <id>archiva.default</id> <url>%%MAVEN_MIRROR%%</url> <mirrorOf>external:*</mirrorOf> </mirror> </mirrors> And voila, Maven will go through Archiva for all its dependencies, which are transparently cached so that subsequent builds don't need to re-download everything again.
jacques_chester 8y ago
> Docker recently has support for multi stage builds. I view these as a future antipattern. They bring what is properly CI's work inside the Docker daemon, which makes it a larger attack surface (many folks run PRs sight-unseen) and obscures it from outside tooling. They only exist because of the limits of the Dockerfile format. With something like Jib, that disappears.

humbleMouse 8y ago

Everything your jar contains can be stored in a git-hub repo with the project. Then you build a fat jar and put it in the image.

zokier 8y ago
I would advice against using fat jars with docker. They do not layer nicely, so your layer sizes become much bigger than necessary, slowing everything down. If you have multiple applications then with fat jars you lose the ability to share the common bits in the applications, which could also have impact on maintainability.
- humbleMouse 8y ago
  I would argue that fat jars make everything simple and portable. Especially if you do performance testing frequently on hardware that isn't running docker. You can compare performance baremetal vs. VM's this way. Furthermore, as log as you use gradle/maven with artifactory then maintainability of fat jar artifacts is pretty simple.

peterwwillis 8y ago

I would do both. Make a build container that pulls in the Git hash that you want to build/test against. You can keep your dependencies, build process, unit and func tests in the container. Your dev team can use the same container image to build and test in the same way that you do, pulling in the latest code from Git and running a docker build. Now nobody needs to set up their environment because it's all just in the Docker image in the same way. You don't need all those build and test files in production, so you can build a new "deploy container" with just the artifacts from your build process. I would also record the versions of your dependencies, as well as the SHA hash of the build/test container you used, and the SHA hash of the Git repos you pulled from. You can also set things like flyway migrations to run each time the container runs. Now you can deploy this second container somewhere. You can then use the same build/test container you used before to run func tests against the deployed container, to double check it works in your deployed environment.

hinkley 8y ago
The former can be handily taken care of in your CI toolchain. You need a build agent with all of the CLI tools you need for your build. Many support Docker these days. Some can even spin up images on demand.
- peterwwillis 8y ago
  This is (for me) the best part of keeping your build process in a container: reducing the complexity of the build agent. The only software dependencies on your build agent are Docker and Git. Your CI/CD job pulls a Git repo with a Dockerfile, and runs 'docker build', and that pulls every dependency it needs.
  - hinkley 8y ago
    Agreed, and I can’t wait to work someplace like that again. We have a build agent with 9 versions of Node in it. It was 6 when I started here. Once a month I get to help someone figure out weird errors because one task has the wrong value in a drop down. It’s such busy work I can’t even. When I run the CI system, shit gets done and people are expected to be able to recreate build failures by themselves. Give a man a tool, make it reliable, then expect him to use it.

rifung 8y ago

> Any benefits to the build-in-container approach that are compelling? The build in container approach makes it so that in theory you have reproducible builds too from any Docker supported machine. I personally like this approach because it makes it so that you don't run into the case where "It works on my machine" when testing locally.

adrianmsmith 8y ago

I think Java has such good build tooling that you don’t need to build in a container for reproducible builds. Other languages might be different. Install the JDK, install Maven, check out the project, “mvn package” and it’ll download what it needs and produce the JAR to be copied into the Docker image by the Dockerfile.