5 ms·

I wish we could replace Dockerfiles already with a format that allows for fine grained layer control.

• enos_feedler 6y ago
  What kind of control are you looking for?
  • yjftsjthsd-h 6y ago
    I'd like explicit layers (transactions, I guess?) - so instead of every command making a layer, I could write FROM alpine STARTLAYER RUN apk update RUN apk upgrade RUN apk add foo ENDLAYER and end up with a 2-layer image (alpine + my stuff) rather than the 4-layer image that docker would produce today.
    • hiram112 6y ago
      Isn't this the same? RUN apk update && apk upgrade && apk add foo
      • kelnos 6y ago
        It is, but the syntax is uglier. Also imagine that you might want to do more than just RUN in a single layer. Maybe COPY some file into the container and then RUN something that uses it.
      • yjftsjthsd-h 6y ago
        In the simple case, yes it is, but it scales poorly; consider ex. https://github.com/docker-library/postgres/blob/b9c080857b880202ebd23c59d33fe86d7a70fea3/13/alpine/Dockerfile#L24 https://github.com/docker-library/postgres/blob/b9c080857b88... which is 112 lines run together like that to keep things in a single layer. This is pretty bad for readability. Also, as sibling comment notes, that only works for RUN commands, not other or mixed groups of commands.
      • topspin 6y ago
        Spend some time on docker hub and you'll find thousands of Dockerfiles with RUN <150 lines of && concatentated scripts>. It's dysfunctional.
        • wildpeaks 6y ago
          Good to know, I often wondered why people produced these giant RUN statements instead of making multiple RUN.
    • nobleach 6y ago
      I too have wanted a solution for the massive amount of layers that are generated just by doing normal things. When you reach a point where you're trying to be so clever just to avoid the creation of a useless layer, it's the tooling's fault, not yours.
    • m463 6y ago
      I have always wanted that too. the way to create small docker image is: RUN apt-get install foo && \ do bar && \ do more && \ remove all of the crap you don't want It should be replaced with actual first-order dockerfile statements You can use multi-stage builds, but they are sort of hacky and do not match your thinking. You can also use --squash but it is a hack too and usually is not available where needed because of experimental. Also, ADD is stupid. You can add a tar.gz file and it will expand it (but you can't use -k) You can add the same file using http/https, except it WON'T expand it so back to the RUN dilemma you get: RUN apt-get install wget && \ wget foo.tar.gz && \ tar xvkf foo.tar.gz -C dir && \ rm foo.tar.gz
    • topspin 6y ago
      You might investigate buildah. You can achieve precise control over layers. Also, buildah mount is awesome; direct access to a container file system from the host; COPY/ADD et al. are obviated.
    • macinjosh 6y ago
      The appropriate solution for this is to put your shell code in a shell script and RUN that.
      • eropple 6y ago
        Then you have to create a layer for sticking the shell script into the container and another to clean up after yourself. If the Dockerfile is canonical, I should have as much as I can reasonably put into it.
    • ecliptik 6y ago
      Have your tried multi-stage builds? We use them almost everywhere to slim down images to only have what they need in production. https://docs.docker.com/develop/develop-images/multistage-build/ https://docs.docker.com/develop/develop-images/multistage-bu...
      • hinkley 6y ago
        Yeah we asked for transactions and they gave us this... thing. It’s cute, and useful, but in the end it brings back the primary problem that Docker actually solves: absolutely getting all of the files your code needs to run into the package, and the right version. Multistage builds require you to airlift files out of one image into another. Accurately. If cherry-picking is the best option on offer by docker, there are plenty of other tools that can do that. Collapsing layers is about seeing that an image provides a set of services at point A, and another set at point B, and that nothing in between represents an interim state of any substantial value. For most images, this is one or two layers that relate to either the main payload, or one particularly volatile dependency.
        • jsmith45 6y ago
          The real purpose for multi-stage builds is for compiled languages. If you have some service written in say C++/Java/C#/Go, then building it in one container (that has the build tools) and copying the build result to another that only has the stuff actually needed at runtime makes good sense. That is much easier and saner than trying to uninstall all the build time components after using them. It is not like for desktop software we install the build tool, build, and then uninstall build tools. Instead we grab the relevant build outputs and package them into an archive or installer. But as a solution for clearing out cruft like package manager caches, etc, yeah multi-stage build is not really optimal. Being able to write the executed steps in a natural way, and then specify at the end to compare everything to come specified previous layer, and create a layer that just has the differences would be really useful. They could even still cache the intermediate layers as part of the build caching process if they wanted, so long as they are not included in final image as pushed to a repository. Docker does have a super cut down version of this with its experimental `--squash` option, but that only makes sure there is a single new layer for the output of the whole dockerfile run. You cannot use this if you for some reason want exactly 2 new layers. This limitation is probably acceptable in a lot of cases. However this option is not supported in the buildkit based backend yet.
    • cpuguy83 6y ago
      RUN --mount=source=my_script.sh,target=/tmp/script.sh /tmp/script.sh This will adds the script without adding a layer for it.