5 ms·

Which advantages does buildpacks offer vs Dockerfiles? > Familiarity with Docker and Dockerfile syntax is not universal. Even those with experience may not be

Which advantages does buildpacks offer vs Dockerfiles? > Familiarity with Docker and Dockerfile syntax is not universal. Even those with experience may not be equipped to write a Dockerfile that can quickly build (and rebuild) a small and secure image. Are there any others besides writing a dockerfile? (which can be non-trivial, I'm not underestimating it)

jillesvangurp 6y ago
This sounds like just moving the problem. The author makes it seem like writing a Dockerfile is somehow such a burden on development teams when in reality it's a rounding error on the overall development time spent on a project. Ours is about five lines long. Copy the files, define some environment variables and a port, done. Not exactly rocket science. Part of the problem stems from trying to roll CI into the Dockerfile. Not a thing if you use, Travis-CI, Github Actions, the Gitlab equivalent of that, Google Cloudbuild, AWS Build, etc. Because it's all externalized there. Creating good pipelines with those is a bit of work but also nothing major. They all follow the same pattern: build pipelines defined in json or yaml that define steps that are effectively dockerized build tools. The only problem your project Dockerfile should solve is copying the output of that pipeline to the container. I guess a build pack would try to "standardize" CI/CD in the context of a very specific deploy environment and CI environment. Standardizing something that specific has limited value though. What would make sense at this point are dockerized build steps that work in most or all of the above CI systems. They all solve more or less the same problems in a very similar way. But of course where it gets complicated is the deep integration with the stuff outside the build system (cloud specific components, secret management, networking, deployment clusters, etc.).
- flavioheleno 6y ago
  I think that Buildpacks help when your Dockerfile is actually complex due to application's dependencies or build process. I worked at a company where we used a multi-stage build Dockerfile that was quite long and tedious to maintain and developers would avoid keeping it up-to-date. There is also another point when developers are used to PaaS such as Heroku that already handles the "build burden" for them and thus Buildpacks may ease the transition to using Dockerfile-based builds (if needed of course).
- robin21 6y ago
  Performance is the challenge. Caching node_modules in Docker is a little tricky. The alternative is adding minutes to your build and deploy vs seconds.
- frafra 6y ago
  There is plenty of terrible Dockerfile out there. I would vastly prefer to see images or Dockerfile generated by some well written buildpacks. I like to think about buildpacks as reasonably written Docker templates. Yours is 5 lines long, but many are others are more complex and could benefit from having a multistage (base stage with common dependencies, build phase with build dependencies, run time with the result of the previous plus some extra runtime dependency), setting up a custom user/group for extra security, optimizing execution order which vastly affect the time to build the image, etc. I am considering using them to make easier for my colleagues which are not used to Docker to containerize their application as well as using them as template system for Dockerfile handled centrally, which do not need any kind of manual merge/alignment afterwards.
- ojhughes 6y ago
  Another advantage of buildpacks is the ability to easily “rebase” an image layer to update dependencies (eg JDK) without rebuilding the whole image. https://buildpacks.io/docs/concepts/operations/rebase/ https://buildpacks.io/docs/concepts/operations/rebase/ The Java buildpack also calculates optimal JDK memory flags based on available resources
- jacques_chester 6y ago
  > This sounds like just moving the problem. The author makes it seem like writing a Dockerfile is somehow such a burden on development teams when in reality it's a rounding error on the overall development time spent on a project. Ours is about five lines long. Copy the files, define some environment variables and a port, done. Not exactly rocket science. Dockerfiles appear trivial and they usually start trivial, but they don't remain trivial for long[0]. In particular it is very, very easy to inadvertently write a Dockerfile that is insecure, or wasteful of space, or promotes build churn, or most frequently: all three. All the tips and tricks that folks need to know in order to write production grade Dockerfiles are given for free in a buildpack. You don't need to think about the efficient ordering and contents of layers, the buildpack does it. You don't need to worry about mystery dependencies arriving via `wget`, buildpacks control what dials out to the outside world. Build churn is dropped to the minimum necessary changes, rather than the minimum possible changes. Disk space and network traffic is saved because far more layers are identical. [0] See "Problem" of this document I wrote a few years ago: https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit#heading=h.5p7j3s6v6hk5 https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU...
mads 6y ago
I think there is rarely a reason to choose this over just a standard Dockerfile, which is usually not that complicated and probably wont change much over time once it has been implemented. If generating Dockerfiles needs to be scaled up somehow, I can understand why this would be useful. Sounds exactly like a problem someone like Heroku would have.
- ojhughes 6y ago
  Heroku actually invented buildpacks

jillesvangurp 6y ago

This sounds like just moving the problem. The author makes it seem like writing a Dockerfile is somehow such a burden on development teams when in reality it's a rounding error on the overall development time spent on a project. Ours is about five lines long. Copy the files, define some environment variables and a port, done. Not exactly rocket science. Part of the problem stems from trying to roll CI into the Dockerfile. Not a thing if you use, Travis-CI, Github Actions, the Gitlab equivalent of that, Google Cloudbuild, AWS Build, etc. Because it's all externalized there. Creating good pipelines with those is a bit of work but also nothing major. They all follow the same pattern: build pipelines defined in json or yaml that define steps that are effectively dockerized build tools. The only problem your project Dockerfile should solve is copying the output of that pipeline to the container. I guess a build pack would try to "standardize" CI/CD in the context of a very specific deploy environment and CI environment. Standardizing something that specific has limited value though. What would make sense at this point are dockerized build steps that work in most or all of the above CI systems. They all solve more or less the same problems in a very similar way. But of course where it gets complicated is the deep integration with the stuff outside the build system (cloud specific components, secret management, networking, deployment clusters, etc.).

flavioheleno 6y ago
I think that Buildpacks help when your Dockerfile is actually complex due to application's dependencies or build process. I worked at a company where we used a multi-stage build Dockerfile that was quite long and tedious to maintain and developers would avoid keeping it up-to-date. There is also another point when developers are used to PaaS such as Heroku that already handles the "build burden" for them and thus Buildpacks may ease the transition to using Dockerfile-based builds (if needed of course).
robin21 6y ago
Performance is the challenge. Caching node_modules in Docker is a little tricky. The alternative is adding minutes to your build and deploy vs seconds.
frafra 6y ago
There is plenty of terrible Dockerfile out there. I would vastly prefer to see images or Dockerfile generated by some well written buildpacks. I like to think about buildpacks as reasonably written Docker templates. Yours is 5 lines long, but many are others are more complex and could benefit from having a multistage (base stage with common dependencies, build phase with build dependencies, run time with the result of the previous plus some extra runtime dependency), setting up a custom user/group for extra security, optimizing execution order which vastly affect the time to build the image, etc. I am considering using them to make easier for my colleagues which are not used to Docker to containerize their application as well as using them as template system for Dockerfile handled centrally, which do not need any kind of manual merge/alignment afterwards.
ojhughes 6y ago
Another advantage of buildpacks is the ability to easily “rebase” an image layer to update dependencies (eg JDK) without rebuilding the whole image. https://buildpacks.io/docs/concepts/operations/rebase/ https://buildpacks.io/docs/concepts/operations/rebase/ The Java buildpack also calculates optimal JDK memory flags based on available resources
jacques_chester 6y ago
> This sounds like just moving the problem. The author makes it seem like writing a Dockerfile is somehow such a burden on development teams when in reality it's a rounding error on the overall development time spent on a project. Ours is about five lines long. Copy the files, define some environment variables and a port, done. Not exactly rocket science. Dockerfiles appear trivial and they usually start trivial, but they don't remain trivial for long[0]. In particular it is very, very easy to inadvertently write a Dockerfile that is insecure, or wasteful of space, or promotes build churn, or most frequently: all three. All the tips and tricks that folks need to know in order to write production grade Dockerfiles are given for free in a buildpack. You don't need to think about the efficient ordering and contents of layers, the buildpack does it. You don't need to worry about mystery dependencies arriving via `wget`, buildpacks control what dials out to the outside world. Build churn is dropped to the minimum necessary changes, rather than the minimum possible changes. Disk space and network traffic is saved because far more layers are identical. [0] See "Problem" of this document I wrote a few years ago: https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit#heading=h.5p7j3s6v6hk5 https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU...

mads 6y ago

I think there is rarely a reason to choose this over just a standard Dockerfile, which is usually not that complicated and probably wont change much over time once it has been implemented. If generating Dockerfiles needs to be scaled up somehow, I can understand why this would be useful. Sounds exactly like a problem someone like Heroku would have.

ojhughes 6y ago
Heroku actually invented buildpacks