3 ms·

Minimizing Docker image size is my favorite premature optimization. In most cases it's not even an optimization! Docker's layered caching means the most releva

Minimizing Docker image size is my favorite premature optimization. In most cases it's not even an optimization! Docker's layered caching means the most relevant size is the diff between the base image and the final image. This is generally independent of which base image you use. It has a bunch of negative consequences like in this blog post. It's something that many people new to Docker feel they should be doing, so you see it a lot. Just an all-round great premature optimization.

sshine 3y ago
I deploy Docker images on a lot of small devices with sketchy internet connections. During development, it really does matter if my images are 4MB or 80MB. Just like when compilation times and tests start growing in the minutes, my feedback cycle is broken. I also compile Rust statically to musl, so I'll seriously consider building the final image on the device to avoid copying the base image during deployment. Or ditch Docker entirely.
- Xelynega 3y ago
  Isn't their point that docker brought incrementalness to this? So just like when you change one c file you don't have to recompile the whole project, updating the last layer in the image should only require a few mb of transfer, the base layers would only be transfered once.
- returningfory2 3y ago
  My initial suspension is that this falls into the "In most cases it's not even an optimization" case, but do correct me if I'm wrong. When you create a new Docker image with a recompiled Rust binary, only the last Docker layer will need to be downloaded to the device. Incremental caching means Docker won't re-download the base layer. The last layer is basically the new binary itself. So I don't see how it would be slower than copying the raw binary. But maybe I'm missing something.
maxmcd 3y ago
Maybe "picking alpine images because they are small without thinking deeply about it" is the mistake here? Debian images can end up using a lot of apt-get tricks to install dependencies cleanly. Way easier with alpine. I think there are plenty of reasons people are quick to make this decision without the necessary depth of investigation. Minimizing docker image size is great. Cold starts with large image downloads can cause all sorts of fun issues. Yes, if you're shipping lots of dependencies in your containers the size of the base image can quickly be irrelevant. And yes, I think it is usually reached for prematurely, but the benefits are there for the few that need them.
egnehots 3y ago
Yes, but on the other hand, people who don't care can create huge images (especially in Node.js and Java ecosystems) that slow down CI and have a large attack surface.
eikenberry 3y ago
You don't use Alpine because it is the smallest anymore, but because it has the best package repo. It beats the big 2 (and Nix), Fedora and Debian, in terms of package coverage and being kept up to date. At least in all my testing. I don't think it is a premature optimization anymore, it has become the best practice by default. The network effect at work.
- whoopdedo 3y ago
  It has to do that because upstream software won't have Alpine-specific build pipelines. (Though I expect a lot more do nowadays.) Meanwhile everyone tests on Fedora, Debian, and Ubuntu so if it isn't in the distribution repository, or the version isn't up-to-date, it's as simple as having your Dockerfile fetch and build from upstream.
- amarshall 3y ago
  Newness is at best only partly true. As far as total packages go, Alpine definitely loses to all three in your cohort. No need to guess, plenty of statistics at https://repology.org/repositories/statistics https://repology.org/repositories/statistics
  - eikenberry 3y ago
    2 points in relation to those numbers. First is that distro's don't package things the same way. Debian is most famous example for how they break up packages into many smaller ones compared to the other distros. Ie. comparing raw numbers doesn't work exactly. Secondly we are talking about packages that are used in Docker images, a definite subset of those for general purpose distros. I've always found Alpine to beat the other distros packages on these terms.
    - amarshall 3y ago
      Indeed it’s not perfect. But relying on one anecdote is not better, either. Further, it’s difficult to justify the claim that Alpine has more packages than nixpkgs when the raw numbers are >7x the other way.
zzyzxd 3y ago
I still like minimizing container image sizes for the security and performance benefits. It's just not a strong enough reason to choose a different linux distro for your production environment. In other words, switching to alpine is the wrong approach to achieve smaller image sizes. Unlike the early days of Docker, today you see popular distros provides multiple official image flavors depends on how minimal you are comfortable with. Or you can go even further with things like Google distroless, or even just scratch. I rarely see negative consequences when it's done appropriately. In fact, we see a lot of benefits by doing this. for instance, developers now understand their own app's dependencies much better ("oh your python app is having a cert error? does it come with its own cert bundle, use 3rd party package like certifi, ca-certificates from OS package manager, or a combination of some of them?" It's much easier to answer this question now than before) Package only the stuff your app needs.

sshine 3y ago

I deploy Docker images on a lot of small devices with sketchy internet connections. During development, it really does matter if my images are 4MB or 80MB. Just like when compilation times and tests start growing in the minutes, my feedback cycle is broken. I also compile Rust statically to musl, so I'll seriously consider building the final image on the device to avoid copying the base image during deployment. Or ditch Docker entirely.

Xelynega 3y ago
Isn't their point that docker brought incrementalness to this? So just like when you change one c file you don't have to recompile the whole project, updating the last layer in the image should only require a few mb of transfer, the base layers would only be transfered once.
returningfory2 3y ago
My initial suspension is that this falls into the "In most cases it's not even an optimization" case, but do correct me if I'm wrong. When you create a new Docker image with a recompiled Rust binary, only the last Docker layer will need to be downloaded to the device. Incremental caching means Docker won't re-download the base layer. The last layer is basically the new binary itself. So I don't see how it would be slower than copying the raw binary. But maybe I'm missing something.

maxmcd 3y ago

Maybe "picking alpine images because they are small without thinking deeply about it" is the mistake here? Debian images can end up using a lot of apt-get tricks to install dependencies cleanly. Way easier with alpine. I think there are plenty of reasons people are quick to make this decision without the necessary depth of investigation. Minimizing docker image size is great. Cold starts with large image downloads can cause all sorts of fun issues. Yes, if you're shipping lots of dependencies in your containers the size of the base image can quickly be irrelevant. And yes, I think it is usually reached for prematurely, but the benefits are there for the few that need them.

egnehots 3y ago

Yes, but on the other hand, people who don't care can create huge images (especially in Node.js and Java ecosystems) that slow down CI and have a large attack surface.

eikenberry 3y ago

You don't use Alpine because it is the smallest anymore, but because it has the best package repo. It beats the big 2 (and Nix), Fedora and Debian, in terms of package coverage and being kept up to date. At least in all my testing. I don't think it is a premature optimization anymore, it has become the best practice by default. The network effect at work.

whoopdedo 3y ago
It has to do that because upstream software won't have Alpine-specific build pipelines. (Though I expect a lot more do nowadays.) Meanwhile everyone tests on Fedora, Debian, and Ubuntu so if it isn't in the distribution repository, or the version isn't up-to-date, it's as simple as having your Dockerfile fetch and build from upstream.
amarshall 3y ago
Newness is at best only partly true. As far as total packages go, Alpine definitely loses to all three in your cohort. No need to guess, plenty of statistics at https://repology.org/repositories/statistics https://repology.org/repositories/statistics
- eikenberry 3y ago
  2 points in relation to those numbers. First is that distro's don't package things the same way. Debian is most famous example for how they break up packages into many smaller ones compared to the other distros. Ie. comparing raw numbers doesn't work exactly. Secondly we are talking about packages that are used in Docker images, a definite subset of those for general purpose distros. I've always found Alpine to beat the other distros packages on these terms.
  - amarshall 3y ago
    Indeed it’s not perfect. But relying on one anecdote is not better, either. Further, it’s difficult to justify the claim that Alpine has more packages than nixpkgs when the raw numbers are >7x the other way.

zzyzxd 3y ago

I still like minimizing container image sizes for the security and performance benefits. It's just not a strong enough reason to choose a different linux distro for your production environment. In other words, switching to alpine is the wrong approach to achieve smaller image sizes. Unlike the early days of Docker, today you see popular distros provides multiple official image flavors depends on how minimal you are comfortable with. Or you can go even further with things like Google distroless, or even just scratch. I rarely see negative consequences when it's done appropriately. In fact, we see a lot of benefits by doing this. for instance, developers now understand their own app's dependencies much better ("oh your python app is having a cert error? does it come with its own cert bundle, use 3rd party package like certifi, ca-certificates from OS package manager, or a combination of some of them?" It's much easier to answer this question now than before) Package only the stuff your app needs.