4 ms·

This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. With Docker it is easy to make this work usin

This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. With Docker it is easy to make this work using multi-stage builds, I'm not sure if buildpacks account for this at all.

jacques_chester 8y ago
> This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. This isn't strictly true of v2 designs (both Heroku and Cloud Foundry have schemes for multi-buildpack support) and in the v3 design explicit consideration is given to making mix-and-match a triviality. Buildpacks can cooperate quite easily and in multiple groups. Where this shines is in updates. If I have an OS base layer, a JDK buildpack and a Maven buildpack, the layers they generate can be independently updated without needing a complete rebuild. So far as I am aware, this is not currently possible with a Dockerfile, multibuild or not. If you invalidate the OS layer, everything else gets rebuilt whether it needed to be or not.
- justinsaccount 8y ago
  That's cool, I'll need to read up on the v3 design. I think you are right about the rebuilds, but maybe not in all cases.. If you had. FROM nodejs:whatever as js RUN npm build FROM golang:whatever as go RUN go build FROM base COPY --from js app.js COPY --from go app If only 'base' was updated rebuilding the image would just need to re-run the COPY commands. The only way everything would get rebuilt is if you also updated the go and js images. I think for more complicated applications https://grahamc.com/blog/nix-and-layered-docker-images https://grahamc.com/blog/nix-and-layered-docker-images is ultimately the best way to do things... ensuring that every step is isolated with explicit dependency information.
  - jacques_chester 8y ago
    I agree, the Nix approach makes lots of problems go away, but you have to be bought in first. I wrote something distinct a few months back which overlaps on the "we can be smarter about layers" thing[0]. Google folks worked on "FTL", which is a technique for determining layering by reasoning about packaging system information. Jib[1] is one such system. There is a view that it will be possible to use FTL implementations or derivatives as buildpack components in the future. [0] https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit# https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU... [1] https://github.com/GoogleContainerTools/jib https://github.com/GoogleContainerTools/jib
- coredog64 8y ago
  Isn’t that the problem that Google’s image-rebase tool [0] is supposed to solve? Given a well constructed image with an OS, JDK, and app layers, it could rewrite any (or all) layers. [0] https://github.com/google/image-rebase https://github.com/google/image-rebase
  - jacques_chester 8y ago
    That's one the technologies being used in buildpacks v3. It's the key to very fast updates. But it's not the whole picture. Having a standard way to assemble the images means that the rebasing operation can be done safely. We've also met to talk about buildpacks and have pre-existing working relationships with all the relevant folks in Google Cloud Builder and Google Container Tools teams through our work on Knative.

jacques_chester 8y ago

> This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. This isn't strictly true of v2 designs (both Heroku and Cloud Foundry have schemes for multi-buildpack support) and in the v3 design explicit consideration is given to making mix-and-match a triviality. Buildpacks can cooperate quite easily and in multiple groups. Where this shines is in updates. If I have an OS base layer, a JDK buildpack and a Maven buildpack, the layers they generate can be independently updated without needing a complete rebuild. So far as I am aware, this is not currently possible with a Dockerfile, multibuild or not. If you invalidate the OS layer, everything else gets rebuilt whether it needed to be or not.

justinsaccount 8y ago
That's cool, I'll need to read up on the v3 design. I think you are right about the rebuilds, but maybe not in all cases.. If you had. FROM nodejs:whatever as js RUN npm build FROM golang:whatever as go RUN go build FROM base COPY --from js app.js COPY --from go app If only 'base' was updated rebuilding the image would just need to re-run the COPY commands. The only way everything would get rebuilt is if you also updated the go and js images. I think for more complicated applications https://grahamc.com/blog/nix-and-layered-docker-images https://grahamc.com/blog/nix-and-layered-docker-images is ultimately the best way to do things... ensuring that every step is isolated with explicit dependency information.
- jacques_chester 8y ago
  I agree, the Nix approach makes lots of problems go away, but you have to be bought in first. I wrote something distinct a few months back which overlaps on the "we can be smarter about layers" thing[0]. Google folks worked on "FTL", which is a technique for determining layering by reasoning about packaging system information. Jib[1] is one such system. There is a view that it will be possible to use FTL implementations or derivatives as buildpack components in the future. [0] https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POUaadcvK3ZNT9QEGDZhPk/edit# https://docs.google.com/document/d/1M2PJ_h6GzviUNHMPt7x-5POU... [1] https://github.com/GoogleContainerTools/jib https://github.com/GoogleContainerTools/jib
coredog64 8y ago
Isn’t that the problem that Google’s image-rebase tool [0] is supposed to solve? Given a well constructed image with an OS, JDK, and app layers, it could rewrite any (or all) layers. [0] https://github.com/google/image-rebase https://github.com/google/image-rebase
- jacques_chester 8y ago
  That's one the technologies being used in buildpacks v3. It's the key to very fast updates. But it's not the whole picture. Having a standard way to assemble the images means that the rebasing operation can be done safely. We've also met to talk about buildpacks and have pre-existing working relationships with all the relevant folks in Google Cloud Builder and Google Container Tools teams through our work on Knative.