8 ms·

docker image definitions are idempotent as a matter of principle. creating an idempotent shell script is non trivial IMO - e.g. what return code is returned fro

docker image definitions are idempotent as a matter of principle. creating an idempotent shell script is non trivial IMO - e.g. what return code is returned from package manager XY when something is already installed etc.

TheDong 8y ago
Really? What do your dockerfiles look like? Most of the ones I've seen in the wild look something like: FROM debian:jessie RUN apt-get install -y somepkg ... What happens when the "debian:jessie" image changes (as it does weekly on dockerhub)? What about "somepkg" in debian's repositories? The answer is that `docker build` will produce a different image. In fact, very few docker image builds I've seen are idempotent. They're not declarative, they're not reproducible, merely the output (the docker image itself) is itself able to be run reproducibly. The actual image definition, not so much. Creating idempotent shell scripts is no harder than creating an idempotent dockerfile. Both are the same problem. A dockerfile is almost entirely the same as a shell script; it copies files around, it runs commands in an environment, and that's all.
- devhead 8y ago
  that's why you version pin and vet new updates before you let them in.
  - TheDong 8y ago
    Okay, so walk me through how to do this in a dockerfile? My first line becomes: FROM debian@sha256:14e15b63bf3c26dac4f6e782dbb4c9877fb88d7d5978d202cb64065b1e01a88b Okay, that's easy. Now, what about older versions of packages in debian's apt repos that have been deleted? How do I get those? I run my own apt mirror I guess which I update in lock-step with my dockerfile and thus don't let the Dockerfile reach out to the network. Is this any different from what you do in a shell script on a server? You use btrfs/zfs/whatever to snapshot the initial version and back it up, you run an apt repository so you can pin package versions, you snapshot before and after updating... I don't see how a docker image definition makes any of this easy. There's not even a flag to disallow network access during "docker build". The claim I'm responding to is "docker image definitions are idempotent as a matter of principle". The large majority of dockerfiles I've seen are not idempotent. Yes, it's possible to make them idempotent, but they do not make it easy.
    - johnmaguire2013 8y ago
      A Dockerfile is an input which produces an image as an output. That image should not suffer from the bit rot examples you gave (e.g. "what about older versions of packages in debian's apt repos that have been deleted?") However, when security patches are released, your image obviously will not contain them.
      - TheDong 8y ago
        I am not arguing that the docker image output is mutably changing. It is a good artifact that can be reproducibly run. The comment I am originally replying to is 'docker image definitions are idempotent'. Note, 'image definitions', not 'images'. My point has nothing to do with the image, but with the image definition itself.
        johnmaguire2013 8y ago
        Understood, just trying to point out there is still a flaw with the image (in that updates are actually important!) FWIW at my work, we don't use apt for installing packages. We compile the packages as a part of the Docker build. This generates mostly idempotent builds.
    - devhead 8y ago
      > Now, what about older versions of packages in debian's apt repos that have been deleted? How do I get those? you can version pin your apt packages if you need to, i personally prefer the minor patches so i get my security updates. my build tool will catch if there's a bug affecting my software. > Is this any different from what you do in a shell script on a server? yes, because I can take that built image and deploy it to any host and all my developers get to all use the same one in their development. but hey, if you like building with shell, you could try out packer and run that shell script to create an image; which can safely be used on any host that supports docker or kub? > I don't see how a docker image definition makes any of this easy. There's not even a flag to disallow network access during "docker build". Easier depends on your goals and perspectives. For me, it's easier to write a docker file that installs what I need to run a service. Where bash doesn't have that, it's just a script that needs an environment to run. where do you run bash? is it locally on your OS with your packages with your settings and your needs? what happens when I run that bash script in a different OS? Who's going to debug that? Are you going to track your changes made in version control? how do you update the other servers/users who use your script? I got other fun things to do than worry about that.

TheDong 8y ago

Really? What do your dockerfiles look like? Most of the ones I've seen in the wild look something like: FROM debian:jessie RUN apt-get install -y somepkg ... What happens when the "debian:jessie" image changes (as it does weekly on dockerhub)? What about "somepkg" in debian's repositories? The answer is that `docker build` will produce a different image. In fact, very few docker image builds I've seen are idempotent. They're not declarative, they're not reproducible, merely the output (the docker image itself) is itself able to be run reproducibly. The actual image definition, not so much. Creating idempotent shell scripts is no harder than creating an idempotent dockerfile. Both are the same problem. A dockerfile is almost entirely the same as a shell script; it copies files around, it runs commands in an environment, and that's all.

devhead 8y ago
that's why you version pin and vet new updates before you let them in.
- TheDong 8y ago
  Okay, so walk me through how to do this in a dockerfile? My first line becomes: FROM debian@sha256:14e15b63bf3c26dac4f6e782dbb4c9877fb88d7d5978d202cb64065b1e01a88b Okay, that's easy. Now, what about older versions of packages in debian's apt repos that have been deleted? How do I get those? I run my own apt mirror I guess which I update in lock-step with my dockerfile and thus don't let the Dockerfile reach out to the network. Is this any different from what you do in a shell script on a server? You use btrfs/zfs/whatever to snapshot the initial version and back it up, you run an apt repository so you can pin package versions, you snapshot before and after updating... I don't see how a docker image definition makes any of this easy. There's not even a flag to disallow network access during "docker build". The claim I'm responding to is "docker image definitions are idempotent as a matter of principle". The large majority of dockerfiles I've seen are not idempotent. Yes, it's possible to make them idempotent, but they do not make it easy.
  - johnmaguire2013 8y ago
    A Dockerfile is an input which produces an image as an output. That image should not suffer from the bit rot examples you gave (e.g. "what about older versions of packages in debian's apt repos that have been deleted?") However, when security patches are released, your image obviously will not contain them.
    - TheDong 8y ago
      I am not arguing that the docker image output is mutably changing. It is a good artifact that can be reproducibly run. The comment I am originally replying to is 'docker image definitions are idempotent'. Note, 'image definitions', not 'images'. My point has nothing to do with the image, but with the image definition itself.
      - johnmaguire2013 8y ago
        Understood, just trying to point out there is still a flaw with the image (in that updates are actually important!) FWIW at my work, we don't use apt for installing packages. We compile the packages as a part of the Docker build. This generates mostly idempotent builds.
  - devhead 8y ago
    > Now, what about older versions of packages in debian's apt repos that have been deleted? How do I get those? you can version pin your apt packages if you need to, i personally prefer the minor patches so i get my security updates. my build tool will catch if there's a bug affecting my software. > Is this any different from what you do in a shell script on a server? yes, because I can take that built image and deploy it to any host and all my developers get to all use the same one in their development. but hey, if you like building with shell, you could try out packer and run that shell script to create an image; which can safely be used on any host that supports docker or kub? > I don't see how a docker image definition makes any of this easy. There's not even a flag to disallow network access during "docker build". Easier depends on your goals and perspectives. For me, it's easier to write a docker file that installs what I need to run a service. Where bash doesn't have that, it's just a script that needs an environment to run. where do you run bash? is it locally on your OS with your packages with your settings and your needs? what happens when I run that bash script in a different OS? Who's going to debug that? Are you going to track your changes made in version control? how do you update the other servers/users who use your script? I got other fun things to do than worry about that.