4 ms·

You can use "--squash" to remove all intermediate layers https://docs.docker.com/engine/reference/commandline/build/#squash-an-images-layers---squash-experimen

You can use "--squash" to remove all intermediate layers https://docs.docker.com/engine/reference/commandline/build/#squash-an-images-layers---squash-experimental https://docs.docker.com/engine/reference/commandline/build/#... The downside of trying to jam all of your commands into a gigantic single RUN invocation is that if it isn't correct/you need to troubleshoot it, you can wind up waiting 10-20 minutes between each single line change just waiting for your build to finish. You lose all the layer caching benefits and it has to re-do the entire build. Just a heads up for anyone that's not suffered through this before.

imglorp 5y ago
This is huge, thanks for the lead. Others should note it's still experimental and your build command may fail with > "--squash" is only supported on a Docker daemon with experimental features enabled Up til now, our biggest improvement was with "FROM SCRATCH".
- selfup 5y ago
  Good to know. `FROM scratch` is such a breath of fresh air for compiled apps. No need for Alpine if I just need to run a binary!
  - jrockway 5y ago
    Do keep in mind that you might want a set of trusted TLS certificates and the timezone database. Both will be annoying runtime errors when you don't trust https://api.example.com https://api.example.com or try to return a time to a user in their preferred time zone. Distroless includes these.
    - selfup 5y ago
      Yea CA certs are the first pain point I hit. Worth the hurdle. Noted on the timezone. Never really thought about that one. Thanks!
- gavinray 5y ago
  No problem. > Others should note it's still experimental and your build command may fail with You might try "docker buildx build", to use the BuildKit client -- squash isn't experimental in that one I believe =) https://docs.docker.com/engine/reference/commandline/buildx_build/ https://docs.docker.com/engine/reference/commandline/buildx_...
selfup 5y ago
Had no idea about squash. Using cached layers can really save time, especially when you already have OS deps/project deps installed. Thanks!
deleted 5y ago
[deleted]
yjftsjthsd-h 5y ago
But then you end up with just one layer, so you lose out on any caching and sharing you might have gotten. Whether this matters is of course very context dependent, but there are times when it'll cost you space.
a_t48 5y ago
Downside - squash makes for longer pulls from the image repository, which can matter for large images or slow connections (you keep build layers but now have no caching for consumers of the image). There's various tricks to be pulled that don't use squash - I've had the most luck putting multiple commands into a buildkit stage, then mounting the results of that stage and copying the output in (either by manually getting the list of files, or using rysnc to figure it out for me).
JimDabell 5y ago
That’s useful, thanks. I’m confused why they haven’t implemented a COMMIT instruction. It’s so common to have people chain “command && command && command && command” to group things into a single layer. Surely it would be better to put something like “AUTOCOMMIT off” at the start of the Dockerfile and then “COMMIT” whenever you want to explicitly close the current layer. It seems much simpler than everybody hacking around it with shell side-effects.
- franga2000 5y ago
  There's an issue on Github about that and it's been open for about as long as Docker has existed. Looks like they just don't care.
- bborud 5y ago
  ...or at least some syntax that represents layers like (block) scopes do in a programming language so it is visually easier to see what is going on.
fragmede 5y ago
Or build using the extra layers, and remove them by squishing the commands together once you've got the Dockerfile right.
franga2000 5y ago
Doesn't that squash all your layers though? That defeats the whole purpose of there being layers. Now instead of a huge total size, but pushing only a fraction of it, your total is lower but you're pushing all of it every time. Same goes for disk space if you're running multiple instances or many images with shared lineage.

imglorp 5y ago

This is huge, thanks for the lead. Others should note it's still experimental and your build command may fail with > "--squash" is only supported on a Docker daemon with experimental features enabled Up til now, our biggest improvement was with "FROM SCRATCH".

selfup 5y ago
Good to know. `FROM scratch` is such a breath of fresh air for compiled apps. No need for Alpine if I just need to run a binary!
- jrockway 5y ago
  Do keep in mind that you might want a set of trusted TLS certificates and the timezone database. Both will be annoying runtime errors when you don't trust https://api.example.com https://api.example.com or try to return a time to a user in their preferred time zone. Distroless includes these.
  - selfup 5y ago
    Yea CA certs are the first pain point I hit. Worth the hurdle. Noted on the timezone. Never really thought about that one. Thanks!
gavinray 5y ago
No problem. > Others should note it's still experimental and your build command may fail with You might try "docker buildx build", to use the BuildKit client -- squash isn't experimental in that one I believe =) https://docs.docker.com/engine/reference/commandline/buildx_build/ https://docs.docker.com/engine/reference/commandline/buildx_...

selfup 5y ago

Had no idea about squash. Using cached layers can really save time, especially when you already have OS deps/project deps installed. Thanks!

deleted 5y ago

[deleted]

yjftsjthsd-h 5y ago

But then you end up with just one layer, so you lose out on any caching and sharing you might have gotten. Whether this matters is of course very context dependent, but there are times when it'll cost you space.

a_t48 5y ago

Downside - squash makes for longer pulls from the image repository, which can matter for large images or slow connections (you keep build layers but now have no caching for consumers of the image). There's various tricks to be pulled that don't use squash - I've had the most luck putting multiple commands into a buildkit stage, then mounting the results of that stage and copying the output in (either by manually getting the list of files, or using rysnc to figure it out for me).

JimDabell 5y ago

That’s useful, thanks. I’m confused why they haven’t implemented a COMMIT instruction. It’s so common to have people chain “command && command && command && command” to group things into a single layer. Surely it would be better to put something like “AUTOCOMMIT off” at the start of the Dockerfile and then “COMMIT” whenever you want to explicitly close the current layer. It seems much simpler than everybody hacking around it with shell side-effects.

franga2000 5y ago
There's an issue on Github about that and it's been open for about as long as Docker has existed. Looks like they just don't care.
bborud 5y ago
...or at least some syntax that represents layers like (block) scopes do in a programming language so it is visually easier to see what is going on.

fragmede 5y ago

Or build using the extra layers, and remove them by squishing the commands together once you've got the Dockerfile right.

franga2000 5y ago

Doesn't that squash all your layers though? That defeats the whole purpose of there being layers. Now instead of a huge total size, but pushing only a fraction of it, your total is lower but you're pushing all of it every time. Same goes for disk space if you're running multiple instances or many images with shared lineage.