4 ms·

I really don't understand the obsession with small docker final images, unless you're doing something silly like export/import and throwing away the benefits of

I really don't understand the obsession with small docker final images, unless you're doing something silly like export/import and throwing away the benefits of layers. Removing temporary files from layers is great, but you get very little benefit from starting smaller. Especially so if that means your final layer is bigger. That's actually a lot worse storage-wise than having a fat base image and a bunch of tiny final layers. I do understand the desire to omit unnecessary attack surface, but if you have a good common base image, then you're not incurring any additional storage beyond once per docker host for all the layers your images have in common.

ben_jones 9y ago
Not a huge docker user but I've noticed various build systems and deployment mechanisms operating slower as a result of large docker images. Faster build times are really attractive to me so I keep my images small.
ty_a 9y ago
The major problem with very large containers is that it can cause failed jobs in container schedulers due to the holdoff timer expiring before the container is even downloaded. I've only seen this with Marathon so far but I've not yet tried to launch a 2GB container in k8s.
- alberteinstein 9y ago
  This doesn't happen on Kubernetes. The Pod stays in ContainerCreating state unless the entire image is pulled. We are running some Java containers which are >2GB and apart from the time it takes to start-up on a fresh node, everything is fine.
alberteinstein 9y ago
When a container is scheduled to a node in Kubernetes, if the image is already available, it takes only ~2s for it to be up and running. But, if it is not present on a node and the image is of 2GB, the download speed is the rate limiting factor, taking start up times well beyond 2 seconds. And in a multi node environment where you do scaling up and down or maybe autoscale, new nodes could come up a lot. So, Leaner images are always better.
- andrewstuart2 9y ago
  > So, Leaner images are always better. That's a very vague statement, though, given the layer concept. The same final filesystem image can use any number of layers to get there. The way you organize the layers in your images, and share data between multiple apps, is important on top of generally keeping things reasonably small. What I think you're trying to say is that fast startup is preferable. What you have not addressed is average startup time, which is where you still benefit dramatically by understanding layers. The smallest average amortized image size is best. That means if you can get all the services for a new node started with 2GB total download, it's better than 3GB total download. Using layers appropriately, and perhaps adding dependencies used by 80% services to your base image, may be best for overall efficiency. Know the tool and know your usage patterns.
  - alberteinstein 9y ago
    This adds an un-necessary dependency between services. Team working on service A would need to coordinate with team working on service B to keep same layers. The contract should be to keep the image size as small as possible, and teams can work independently. Also, python:3.6 doesn't always mean the same layers, since it could be rebuilt with entirely different layers with same tag, unless FROM is locked to a particular layer.
devonkim 9y ago
I’ve noticed some Docker image registry solutions out there billing / chargeback based upon total image size committed to the repository rather than realized disk usage. This pretty much destroys the space advantages of Docker images completely (I suspect it’s because the underlying asset storage system by such services is super naive or the folks in charge of said companies simply want to charge enterprises boatloads because that will absolutely be noticed by anyone cost-sensitive whatsoever).
robert-wallis 9y ago
We have had our entire kube go down because the image size was huge and the master node only had 20gb disk and docker wasn’t cleaning up old named images. Smaller just extends the problem to the future, but it gives us time to re-build our cluster with a new kube version.

ben_jones 9y ago

Not a huge docker user but I've noticed various build systems and deployment mechanisms operating slower as a result of large docker images. Faster build times are really attractive to me so I keep my images small.

ty_a 9y ago

The major problem with very large containers is that it can cause failed jobs in container schedulers due to the holdoff timer expiring before the container is even downloaded. I've only seen this with Marathon so far but I've not yet tried to launch a 2GB container in k8s.

alberteinstein 9y ago
This doesn't happen on Kubernetes. The Pod stays in ContainerCreating state unless the entire image is pulled. We are running some Java containers which are >2GB and apart from the time it takes to start-up on a fresh node, everything is fine.

alberteinstein 9y ago

When a container is scheduled to a node in Kubernetes, if the image is already available, it takes only ~2s for it to be up and running. But, if it is not present on a node and the image is of 2GB, the download speed is the rate limiting factor, taking start up times well beyond 2 seconds. And in a multi node environment where you do scaling up and down or maybe autoscale, new nodes could come up a lot. So, Leaner images are always better.

andrewstuart2 9y ago
> So, Leaner images are always better. That's a very vague statement, though, given the layer concept. The same final filesystem image can use any number of layers to get there. The way you organize the layers in your images, and share data between multiple apps, is important on top of generally keeping things reasonably small. What I think you're trying to say is that fast startup is preferable. What you have not addressed is average startup time, which is where you still benefit dramatically by understanding layers. The smallest average amortized image size is best. That means if you can get all the services for a new node started with 2GB total download, it's better than 3GB total download. Using layers appropriately, and perhaps adding dependencies used by 80% services to your base image, may be best for overall efficiency. Know the tool and know your usage patterns.
- alberteinstein 9y ago
  This adds an un-necessary dependency between services. Team working on service A would need to coordinate with team working on service B to keep same layers. The contract should be to keep the image size as small as possible, and teams can work independently. Also, python:3.6 doesn't always mean the same layers, since it could be rebuilt with entirely different layers with same tag, unless FROM is locked to a particular layer.

devonkim 9y ago

I’ve noticed some Docker image registry solutions out there billing / chargeback based upon total image size committed to the repository rather than realized disk usage. This pretty much destroys the space advantages of Docker images completely (I suspect it’s because the underlying asset storage system by such services is super naive or the folks in charge of said companies simply want to charge enterprises boatloads because that will absolutely be noticed by anyone cost-sensitive whatsoever).

robert-wallis 9y ago

We have had our entire kube go down because the image size was huge and the master node only had 20gb disk and docker wasn’t cleaning up old named images. Smaller just extends the problem to the future, but it gives us time to re-build our cluster with a new kube version.