4 ms·

Being caught in the containerization craze myself I'd love to hear whether the story is exaggerating or is painfully accurate. So far I've been bitten by the i

by FuNe 10y ago

Being caught in the containerization craze myself I'd love to hear whether the story is exaggerating or is painfully accurate. So far I've been bitten by the inability to clean up images of certain age. UPDATE: Another really annoying thing is the inability to tag an image directly into a registry (AFAIK). You need to pull, tag and push back again. Given that images can be GBs long you end up with really heavy net traffic for a simple task.

mobiuscog 10y ago
I'm confused by this. You obviously have a process to build new images... why doesn't the same process prune old ones ?
- n0us 10y ago
  No, there is no simple command to remove old images because the creators of docker want you to compose other existing commands to delete them and it would be unnecessary to add a command. IMO this is a terrible decision but that was apparently the rationalization.
  - thaJeztah 10y ago
    A new "docker prune" command is added to docker 1.13 to address this, see https://github.com/docker/docker/pull/26108 https://github.com/docker/docker/pull/26108
- FuNe 10y ago
  That's what I do but given that the process is run in parallel being part of a complex CI system I needed to write quite a bit of logic to do the pruning safely myself. Main problem being that I couldn't rely on docker daemon not deleting images that might actually be still needed by other processes building in parallel.
mgliwka 10y ago
The team at spotify has built this (see above): https://github.com/spotify/docker-gc https://github.com/spotify/docker-gc
- FuNe 10y ago
  Thanks! That looks promising.
- bostonvaulter2 10y ago
  Why isn't that just built-in?
amouat 10y ago
Why do you want to retag images? I'd suggest simply not doing this and adding all the tags and labels you need at build time.
- predakanga 10y ago
  One potential reason is to handle version promotion. For instance, you may want to run an image for a while on a staging server before re-tagging it as the current stable release.
  - amouat 10y ago
    But then you could just push from the staging server. And it's probably best to keep this independent of the image tag i.e. your system knows image v1.1.2 is the latest stable, but the image name never changes. If you retag your stable image "latest", you run into issues with rollback and figuring out what you're running.
    - ones_and_zeros 10y ago
      But now your staging server is also your docker image management server, breaking the JLP contract.
      - dserodio 10y ago
        What do you mean by JLP contract?
  - jabzd 10y ago
    We are currently moving to using Docker swarmkit (from non-Docker environment). We thought about using tags to communicate environment but it felt like an anti-pattern given how tags work. What we ended up with is the following: Per publishable branch (e.g. master to prod, dev to QA, shared integration that isn't ready for dev to secondary QA): - Publish image to private docker registry with tag in the format of <branch>-latest (e.g. master-latest, dev-latest). Also add labels with git revision hash and CI build number to reconcile what is in the image both in CI and in git repo. - Capture the digest of the published image and put in .txt file as an artifact from CI. - Add a tag to the git repo with the build number, treated as version number in the form of v<build-number>-<branch> (e.g. v1-master). - Maintain list of applications' docker image digests that should be running in each environment. We currently have this automatically determined by pulling the digest artifact from the passing-build digest. We do allow for manual overrides, however. - We have a CI project that updates the docker swarm with these digests and other settings (replicas, mounts, environment variables). We wrote a small tool that does this similar to where stacks/dab files are going but with more functionality. With the labels and published digests we can go from build to digest, digest to build, etc. This is working really well so far and we aren't fighting the tools. We may separate the non-master registry later in life, and that's really easy given that docker treats the private registry hostname as part of the tag.

by FuNe 10y ago

mobiuscog 10y ago

I'm confused by this. You obviously have a process to build new images... why doesn't the same process prune old ones ?

n0us 10y ago
No, there is no simple command to remove old images because the creators of docker want you to compose other existing commands to delete them and it would be unnecessary to add a command. IMO this is a terrible decision but that was apparently the rationalization.
- thaJeztah 10y ago
  A new "docker prune" command is added to docker 1.13 to address this, see https://github.com/docker/docker/pull/26108 https://github.com/docker/docker/pull/26108
FuNe 10y ago
That's what I do but given that the process is run in parallel being part of a complex CI system I needed to write quite a bit of logic to do the pruning safely myself. Main problem being that I couldn't rely on docker daemon not deleting images that might actually be still needed by other processes building in parallel.

mgliwka 10y ago

The team at spotify has built this (see above): https://github.com/spotify/docker-gc https://github.com/spotify/docker-gc

FuNe 10y ago
Thanks! That looks promising.
bostonvaulter2 10y ago
Why isn't that just built-in?

amouat 10y ago

Why do you want to retag images? I'd suggest simply not doing this and adding all the tags and labels you need at build time.

predakanga 10y ago
One potential reason is to handle version promotion. For instance, you may want to run an image for a while on a staging server before re-tagging it as the current stable release.
- amouat 10y ago
  But then you could just push from the staging server. And it's probably best to keep this independent of the image tag i.e. your system knows image v1.1.2 is the latest stable, but the image name never changes. If you retag your stable image "latest", you run into issues with rollback and figuring out what you're running.
  - ones_and_zeros 10y ago
    But now your staging server is also your docker image management server, breaking the JLP contract.
    - dserodio 10y ago
      What do you mean by JLP contract?
- jabzd 10y ago
  We are currently moving to using Docker swarmkit (from non-Docker environment). We thought about using tags to communicate environment but it felt like an anti-pattern given how tags work. What we ended up with is the following: Per publishable branch (e.g. master to prod, dev to QA, shared integration that isn't ready for dev to secondary QA): - Publish image to private docker registry with tag in the format of <branch>-latest (e.g. master-latest, dev-latest). Also add labels with git revision hash and CI build number to reconcile what is in the image both in CI and in git repo. - Capture the digest of the published image and put in .txt file as an artifact from CI. - Add a tag to the git repo with the build number, treated as version number in the form of v<build-number>-<branch> (e.g. v1-master). - Maintain list of applications' docker image digests that should be running in each environment. We currently have this automatically determined by pulling the digest artifact from the passing-build digest. We do allow for manual overrides, however. - We have a CI project that updates the docker swarm with these digests and other settings (replicas, mounts, environment variables). We wrote a small tool that does this similar to where stacks/dab files are going but with more functionality. With the labels and published digests we can go from build to digest, digest to build, etc. This is working really well so far and we aren't fighting the tools. We may separate the non-master registry later in life, and that's really easy given that docker treats the private registry hostname as part of the tag.