4 ms·

Overall great post. > If you rely on latest you might silently inherit updated packages that in the best worst case might impact your application reliability,

Overall great post. > If you rely on latest you might silently inherit updated packages that in the best worst case might impact your application reliability, in the worst worst case might introduce a vulnerability. On this one I disagree. Your CI should handle reliability (and if not, you have a bigger problem) and you're more likely to patch a vulnerability than to introduce a new one and it's unlikely that by the time a PR hits production that the version is compromised. I understand that updating cuts both ways when it comes to security, but I agree with Matt Tait's ultimate conclusion when he spoke on this issue a few years ago: For most medium size and smaller companies constantly updating is safer than delaying. He had real world data and graphs of compromise windows, etc. Short answer was that attackers are more time motivated than defenders.

dbrgn 6y ago
Operating system updates are the responsibility of the base image. Use a base image that is regularly updated. For your own images, ensure that they are also being rebuilt regularly. There are a lot of semi-official Docker images on Docker Hub that are published once and never updated until the next software release. That is a huge anti-pattern in my view, those images should not be relied on for production.
cooljacob204 6y ago
Would you have a link to the talk? I couldn't find it with a google search.
- 3pt14159 6y ago
  I don't have time to track down a link to a video at the moment, but it was at Infiltrate, an offensive-minded cybersecurity conference out of Miami.
  - kronin 6y ago
    Is it this one? https://vimeo.com/267445424 https://vimeo.com/267445424
tomphoolery 6y ago
Pinning the version tag for base images in Dockerfile is a good idea beyond the scope of security. It helps with onboarding as well. I've been in situations where depending on the `:latest` tag of a base image caused different versions of that image to be used on different machines, resulting in developers having weird issues that no one else was having ("I thought Docker was supposed to solve this!"). Now, I only use non-specific tags like `:latest` or `:12-alpine` before I distribute the Docker image, by either collaborating with others or pushing it to Docker Hub. It just gives me peace of mind to know that others are building on the exact same stack as I was building on.
- 3pt14159 6y ago
  This is a reasonable choice to make depending on the complexity of your project. For the projects I've been on, the latest version plus the test suite is enough to catch weirdness creeping in and, as a side benefit, it gets fixed faster than if it were pinned. Sometimes the issue really is caused by the base image and it is easier to get a fix merged if the issue was caused quite recently because the developers responsible see early reports of issues as more endemic than if they're reported days or weeks later.
  - piaste 6y ago
    For many popular images, you can use a major-version tag and fetch the latest patch but avoid breaking changes. Eg. postgres:12 instead of either postgres:latest or postgres:12.0.1
donmcronald 6y ago
I always thought `:latest` was a bit of a special case. Ex: Pushing `2.0.1` followed by `1.1.4` would leave `1.1.4` as the `latest` image which could be an issue by itself. Is that wrong? I've always tried to pick stable tags [1] if they're available. 1. https://docs.microsoft.com/en-us/archive/blogs/stevelasker/docker-tagging-best-practices-for-tagging-and-versioning-docker-images#stable-tags https://docs.microsoft.com/en-us/archive/blogs/stevelasker/d...
- sfoley 6y ago
  There is absolutely nothing special about `latest`, it’s just a tag like any other. It simply happens to be the default tag when omitted, just like how `origin` is the default remote name when performing a `git clone`.
  - GauntletWizard 6y ago
    To expand on this: Well maintained projects do not often push to `:latest`; They treat that tag like a `stable` version and push to it only public, stable releases. Roughly, it should always be max(semver), though practically it often trails when a new major release happens.
  - donmcronald 6y ago
    > There is absolutely nothing special about `latest` > It simply happens to be the default tag when omitted That makes it special though, right? It's different than any other tag because it can be inadvertently pushed via accidental omission.
michaelperel 6y ago
Shameless Plug: I wrote a cli-plugin for docker, docker-lock, that will track the image digests (SHA's) of your images in a separate file that you can check in to git, so you will always know exactly which image you are using (even if you are using the latest tag). It can even rewrite all your Dockerfiles and docker-compose files to use those digests. https://github.com/safe-waters/docker-lock https://github.com/safe-waters/docker-lock Would love for anyone dealing with this issue to check it out!
5d749d7da7d5 6y ago
This makes me wonder, why there has not been a bigger push towards microkernel/minimal OS with audited toolchains that were "done". Minimal features and minimal surface area. A plug and play distribution with security at the forefront which rarely needed updating because only the essential was available. I would be fine taking a healthy performance hit if I knew that the base OS was secure. (At this point I expect the BSD folks to chime in that they have had this for years)
- piaste 6y ago
  Isn't that (one of) the design goals of CoreOS, Alpine, Clear Linux, etc.?
  - theptip 6y ago
    Further in that direction, https://github.com/GoogleContainerTools/distroless https://github.com/GoogleContainerTools/distroless — stripping out as much of the OS code as possible.

dbrgn 6y ago

Operating system updates are the responsibility of the base image. Use a base image that is regularly updated. For your own images, ensure that they are also being rebuilt regularly. There are a lot of semi-official Docker images on Docker Hub that are published once and never updated until the next software release. That is a huge anti-pattern in my view, those images should not be relied on for production.

cooljacob204 6y ago

Would you have a link to the talk? I couldn't find it with a google search.

3pt14159 6y ago
I don't have time to track down a link to a video at the moment, but it was at Infiltrate, an offensive-minded cybersecurity conference out of Miami.
- kronin 6y ago
  Is it this one? https://vimeo.com/267445424 https://vimeo.com/267445424

tomphoolery 6y ago

Pinning the version tag for base images in Dockerfile is a good idea beyond the scope of security. It helps with onboarding as well. I've been in situations where depending on the `:latest` tag of a base image caused different versions of that image to be used on different machines, resulting in developers having weird issues that no one else was having ("I thought Docker was supposed to solve this!"). Now, I only use non-specific tags like `:latest` or `:12-alpine` before I distribute the Docker image, by either collaborating with others or pushing it to Docker Hub. It just gives me peace of mind to know that others are building on the exact same stack as I was building on.

3pt14159 6y ago
This is a reasonable choice to make depending on the complexity of your project. For the projects I've been on, the latest version plus the test suite is enough to catch weirdness creeping in and, as a side benefit, it gets fixed faster than if it were pinned. Sometimes the issue really is caused by the base image and it is easier to get a fix merged if the issue was caused quite recently because the developers responsible see early reports of issues as more endemic than if they're reported days or weeks later.
- piaste 6y ago
  For many popular images, you can use a major-version tag and fetch the latest patch but avoid breaking changes. Eg. postgres:12 instead of either postgres:latest or postgres:12.0.1

donmcronald 6y ago

I always thought `:latest` was a bit of a special case. Ex: Pushing `2.0.1` followed by `1.1.4` would leave `1.1.4` as the `latest` image which could be an issue by itself. Is that wrong? I've always tried to pick stable tags [1] if they're available. 1. https://docs.microsoft.com/en-us/archive/blogs/stevelasker/docker-tagging-best-practices-for-tagging-and-versioning-docker-images#stable-tags https://docs.microsoft.com/en-us/archive/blogs/stevelasker/d...

sfoley 6y ago
There is absolutely nothing special about `latest`, it’s just a tag like any other. It simply happens to be the default tag when omitted, just like how `origin` is the default remote name when performing a `git clone`.
- GauntletWizard 6y ago
  To expand on this: Well maintained projects do not often push to `:latest`; They treat that tag like a `stable` version and push to it only public, stable releases. Roughly, it should always be max(semver), though practically it often trails when a new major release happens.
- donmcronald 6y ago
  > There is absolutely nothing special about `latest` > It simply happens to be the default tag when omitted That makes it special though, right? It's different than any other tag because it can be inadvertently pushed via accidental omission.

michaelperel 6y ago

Shameless Plug: I wrote a cli-plugin for docker, docker-lock, that will track the image digests (SHA's) of your images in a separate file that you can check in to git, so you will always know exactly which image you are using (even if you are using the latest tag). It can even rewrite all your Dockerfiles and docker-compose files to use those digests. https://github.com/safe-waters/docker-lock https://github.com/safe-waters/docker-lock Would love for anyone dealing with this issue to check it out!

5d749d7da7d5 6y ago

This makes me wonder, why there has not been a bigger push towards microkernel/minimal OS with audited toolchains that were "done". Minimal features and minimal surface area. A plug and play distribution with security at the forefront which rarely needed updating because only the essential was available. I would be fine taking a healthy performance hit if I knew that the base OS was secure. (At this point I expect the BSD folks to chime in that they have had this for years)

piaste 6y ago
Isn't that (one of) the design goals of CoreOS, Alpine, Clear Linux, etc.?
- theptip 6y ago
  Further in that direction, https://github.com/GoogleContainerTools/distroless https://github.com/GoogleContainerTools/distroless — stripping out as much of the OS code as possible.