7 ms·

CNCF to Host Cloud Native Buildpacks in the Sandbox

81 points by rettori 8y ago

CameronNemo 8y ago
I wonder how this relates to LinuxKit.
- codefinger 8y ago
  LinuxKit (as I understand it) is for building the OS itself. Buildpacks are a higher level abstract for building a complete application image (with the emphasis on application). They take app source code as input, and output a docker image that's ready for prod.
- tjfontaine 8y ago
  LinuxKit is a way to build immutable machine images by stitching multiple containers together. You could use the resulting container from a buildpack in an image built by LinuxKit.
elyonkiNIR 8y ago
Can’t believe containers made it this far. It’s neckbeard tech with no benefit.
jacques_chester 8y ago
Hi, in my role as professional gadfly, I have been involved with this tangentially for a few months and directly for a few weeks (on behalf of Pivotal). I will make mistakes, but ... AMA.
- spenczar5 8y ago
  I know almost nothing about this topic. I have only really built container images with Dockerfiles. How would you introduce buildpacks to me?
  - jacques_chester 8y ago
    I used the Onsi Haiku over on the equivalent HN thread on Heroku's blogpost: Here is my source code. Run it on the cloud for me. I do not care how. The gist is that Docker containers are awesome for the Day 1 experience. I write a Dockerfile and I'm off to the races. But then Day 2 rolls around and I have a production system with 12,000 containers[0]. 1. What the hell is in those containers, anyhow? 2. A new CVE landed and I want to upgrade all of them in a few minutes without anyone being interrupted (or even having to know). How? 3. I have a distributed system with many moving parts. I build a giant fragile hierarchy of Dockerfiles to efficiently contain the right dependencies, making development slower. Then I snap and turn it into a giant kitchen-sink Dockerfile with the union of all the dependencies in it. Now production is slow as hell. 4. Operations become upset about points 1-3. Now I can only use curated Dockerfiles, can only come through our elaborate Jenkins farm, rules rules rules. Wasn't the purpose of Dockerfiles to make this all just ... go away? Buildpacks solve all of these. I know what's in the container because buildpacks control the build. I can update CVE flaws in potentially seconds. Each container can have what it needs - no more, no less. And most important: the buildpack runs locally, or in the cluster, exactly the same. It's all the developer benefits of Dockerfiles/docker build, minus most of the suck. [0] Yahoo! Japan has 12,000 prod and 8,000 dev containers using buildpacks: https://www.slideshare.net/iranainanimosuteteshimaou/yahoo-japan20000-container-journey https://www.slideshare.net/iranainanimosuteteshimaou/yahoo-j...
    - kgilpin 8y ago
      It sounds like a buildpack identifies a common case of what someone might be doing in a container / Dockerfile, and standardizes it to remove the arbitrary variation that inevitably occurs when lots of people are independently solving exactly the same problem. For example, the patch level of the OS is directly or indirectly specified in the Dockerfile, but generally doesn’t actually matter to the application. If your use case fits into a scenario which is handled by an existing buildpack, then the claim is that you’ll be better off using the buildpack because the infrastructure can make optimizations that can’t be made with arbitrary containers. If your use case isn’t covered by a buildpack, then you can either (1) make a buildpack or (2) revert to raw containers. (Although your platform may not allow (2)).
sclevine 8y ago
For those who would like to better understand the technology: Presentation to CNCF TOC: https://docs.google.com/presentation/d/1RkygwZw7ILVgGhBpKnFNgJ4BCc_9qMG8cIf0MRbuzB4/edit#slide=id.g4054009e6b_5_0 https://docs.google.com/presentation/d/1RkygwZw7ILVgGhBpKnFN... Formal specification: https://github.com/buildpack/spec https://github.com/buildpack/spec Sample buildpacks: https://github.com/buildpack/samples https://github.com/buildpack/samples
- nailer 8y ago
  In other words: they're (typically) shell scripts that set up an OS to run a language. Here's the node one: https://github.com/buildpack/samples/blob/master/nodejs-buildpack/bin/build https://github.com/buildpack/samples/blob/master/nodejs-buil...
a012 8y ago
It's first time I heard of buildpacks, can I just throw in other than nodejs, ie. Laravel source code and build an image?
- jacques_chester 8y ago
  If there is a conforming buildpack, then yes. What you're looking at here is informally the "v3" effort, which extends and consolidates the "v2a" and "v2b" designs evolved by Heroku and Cloud Foundry respectively from the original Heroku design. In v2 both Heroku and Cloud Foundry provide supported PHP buildpacks, as well as Java, Ruby, Python, .NET Core and I forget the rest right now. There are hundreds of community buildpacks.
- dreyfiz 8y ago
  Yeah, do you have a composer.json in the root of your repo? Buildpack will detect a PHP app...
  - dullgiulio 8y ago
    I have the feeling that the detect phase is absolutely orthogonal to delivering what should basically be an executable package. What is the rationale behind having the build step tied to the packaging format?
    - jacques_chester 8y ago
      It's not tied to the packaging format. Detect is the step that decides which buildpack or buildpacks will be responsible for constructing the image from the sourcecode. Typically this means that buildpacks look for files that correspond to the relevant ecosystem. Maven buildpacks look for pom.xml. PHP buildpacks look for composer.json. Etc. Nothing in this creates a hard binding. Detect steps may use whatever logic they need to decide on whether to signal they can work on a codebase. Edit: in the v3 design the detect script can also provide dependency information that later steps can pick up. So for example, a JDK buildpack can say "yes, I can interpret this codebase, and I can contribute a JDK". A later buildpack can then look for this contribution as a condition, eg. the Maven buildpack can say "I will proceed if I see a pom.xml and if there is a JDK available".
gigatexal 8y ago
What am I missing? Looking at the example for java it looks a lot more complicated and a step back from docker and its layers concept.
- coredog64 8y ago
  I don't think you're missing anything. It's vendors that don't do Docker trying to stay relevant in a Docker/Kubernetes world. By making Docker more complex, said vendors can continue to charge $500/pod for OSS k8s.
  - jacques_chester 8y ago
    It's opensource and you can run it yourself, which has incidentally been true since before either Docker or Kubernetes existed. We've got an open PR to add Cloud Native Buildpack support to Knative Build. You can take these containers and run them on whatever Kubernetes you like. Disclosure: I work for a Said Vendor.
- jacques_chester 8y ago
  There are several audiences. 1. Developers. What you need to know is: it just works. You don't even need to write a Dockerfile any more. The buildpack turns your code into a well-structured, efficient runnable image with no additional effort. 2. Operators. What you need to know is: oh thank god no more mystery meat in production. Patching the OS is a ho-hum affair instead of a stone grinding nightmare that turns your developers into a white hot bucket of rage because you have to nag them or block them from deploying or both. 3. Platform vendors, buildpack authors and curious passers-by What you need to know is: All the other stuff about detect, analyse, build or export. Unless you are in group 3, the basic thing is that Buildpacks require less effort than Dockerfiles with more safety and faster builds.
- sclevine 8y ago
  Dockerfiles require you to rebuild lower layers when any upper layers change, even though the OCI image format doesn't care about this. Cloud Native Buildpacks can intelligently choose which layers to rebuild and replace. Additionally, certain layers can be updated en mass for many images on a registry (using cross-repo blob mounting, with no real data transfer!), which is useful for patching CVEs quickly at scale. The samples take advantage of this (as well as a separate transparent cache) in order to demonstrate the different aspects of the formal spec. A simple buildpack is not necessarily much more complicated than a simple Dockerfile.
deleted 8y ago
[deleted]
EdSchouten 8y ago
I currently use Bazel in combination with rules_docker to construct container images of applications, often written in Go. As I don't depend on cgo, my container images may be bare (i.e., only containing a Go executable and some static artifacts). Though I understand that there are likely only dozens of us, my question is: what would the use of Buildpacks buy us for this specific use case?
- justinsaccount 8y ago
  one case is that you could have a bazel buildpack that contains bazel and java and all that stuff requires in order to run the build. Unfortunately the resulting image would be larger than just a single binary, but building it would be a lot easier and repeatable for other people working on the project. The base layers would be cached though so in practice it might not be that much larger on disk. I looked into using bazel for doing similar things, and the biggest stumbling block is that bazel itself is a PITA to install on all platforms. May end up trying https://please.build/ https://please.build/ at some point.
  - jbott 8y ago
    The bazel installation process has gotten a lot better lately - they now bundle OpenJDK inside the binary and it is only a single download.
    - justinsaccount 8y ago
      Unfortunately that's a non starter for things like debian.. but that's good to know, I'll give it another look. One thing I really like about bazel is the pkg rules.. I currently use fpm/goreleaser(nfpm) to build rpms for things, and it's nice having a single build tool that can build the app and spit out an rpm.
- jacques_chester 8y ago
  Mostly the advantage would be that 1) you wouldn't have to maintain the buildpack yourself, 2) operators can upgrade your software without needing to get you to do it, 3) upgrades might actually be faster due to layer rebasing and 4) if you add other stacks to your architecture, you still have the same development, deployment and update experience.
mosselman 8y ago
I just tried this on my rails project and it is detecting a nodejs project. Is there a way to have it use ruby instead while still taking care of my yarn dependencies?
- justinsaccount 8y ago
  This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. With Docker it is easy to make this work using multi-stage builds, I'm not sure if buildpacks account for this at all.
  - jacques_chester 8y ago
    > This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. This isn't strictly true of v2 designs (both Heroku and Cloud Foundry have schemes for multi-buildpack support) and in the v3 design explicit consideration is given to making mix-and-match a triviality. Buildpacks can cooperate quite easily and in multiple groups. Where this shines is in updates. If I have an OS base layer, a JDK buildpack and a Maven buildpack, the layers they generate can be independently updated without needing a complete rebuild. So far as I am aware, this is not currently possible with a Dockerfile, multibuild or not. If you invalidate the OS layer, everything else gets rebuilt whether it needed to be or not.
    - justinsaccount 8y ago
      That's cool, I'll need to read up on the v3 design. I think you are right about the rebuilds, but maybe not in all cases.. If you had. FROM nodejs:whatever as js RUN npm build FROM golang:whatever as go RUN go build FROM base COPY --from js app.js COPY --from go app If only 'base' was updated rebuilding the image would just need to re-run the COPY commands. The only way everything would get rebuilt is if you also updated the go and js images. I think for more complicated applications https://grahamc.com/blog/nix-and-layered-docker-images https://grahamc.com/blog/nix-and-layered-docker-images is ultimately the best way to do things... ensuring that every step is isolated with explicit dependency information.
josegonzalez 8y ago
> As one of the maintainers of Herokuish - a tool that manages the application of buildpacks to repositories and is used in quite a few Buildpack implementations - I am super happy that CNCF reached out to us and included us in the process. Oh wait, that didn't happen... All jokes aside, this looks great. Super-early of course - seems like there are quite a few issues in the `pack` repository to be implemented - but I'm excited to see where this lands. Buildpack detection and application is not a straightforward problem.
- jacques_chester 8y ago
  Please join us at https://slack.buildpacks.io/ https://slack.buildpacks.io/ ! (ps. I know what these surprises can feel like -- for what it's worth, I am sorry)
m-arnold 8y ago
What is the difference between buildpacks and packer?

81 points by rettori 8y ago

CameronNemo 8y ago

I wonder how this relates to LinuxKit.

codefinger 8y ago
LinuxKit (as I understand it) is for building the OS itself. Buildpacks are a higher level abstract for building a complete application image (with the emphasis on application). They take app source code as input, and output a docker image that's ready for prod.
tjfontaine 8y ago
LinuxKit is a way to build immutable machine images by stitching multiple containers together. You could use the resulting container from a buildpack in an image built by LinuxKit.

elyonkiNIR 8y ago

Can’t believe containers made it this far. It’s neckbeard tech with no benefit.

jacques_chester 8y ago

Hi, in my role as professional gadfly, I have been involved with this tangentially for a few months and directly for a few weeks (on behalf of Pivotal). I will make mistakes, but ... AMA.

spenczar5 8y ago
I know almost nothing about this topic. I have only really built container images with Dockerfiles. How would you introduce buildpacks to me?
- jacques_chester 8y ago
  I used the Onsi Haiku over on the equivalent HN thread on Heroku's blogpost: Here is my source code. Run it on the cloud for me. I do not care how. The gist is that Docker containers are awesome for the Day 1 experience. I write a Dockerfile and I'm off to the races. But then Day 2 rolls around and I have a production system with 12,000 containers[0]. 1. What the hell is in those containers, anyhow? 2. A new CVE landed and I want to upgrade all of them in a few minutes without anyone being interrupted (or even having to know). How? 3. I have a distributed system with many moving parts. I build a giant fragile hierarchy of Dockerfiles to efficiently contain the right dependencies, making development slower. Then I snap and turn it into a giant kitchen-sink Dockerfile with the union of all the dependencies in it. Now production is slow as hell. 4. Operations become upset about points 1-3. Now I can only use curated Dockerfiles, can only come through our elaborate Jenkins farm, rules rules rules. Wasn't the purpose of Dockerfiles to make this all just ... go away? Buildpacks solve all of these. I know what's in the container because buildpacks control the build. I can update CVE flaws in potentially seconds. Each container can have what it needs - no more, no less. And most important: the buildpack runs locally, or in the cluster, exactly the same. It's all the developer benefits of Dockerfiles/docker build, minus most of the suck. [0] Yahoo! Japan has 12,000 prod and 8,000 dev containers using buildpacks: https://www.slideshare.net/iranainanimosuteteshimaou/yahoo-japan20000-container-journey https://www.slideshare.net/iranainanimosuteteshimaou/yahoo-j...
  - kgilpin 8y ago
    It sounds like a buildpack identifies a common case of what someone might be doing in a container / Dockerfile, and standardizes it to remove the arbitrary variation that inevitably occurs when lots of people are independently solving exactly the same problem. For example, the patch level of the OS is directly or indirectly specified in the Dockerfile, but generally doesn’t actually matter to the application. If your use case fits into a scenario which is handled by an existing buildpack, then the claim is that you’ll be better off using the buildpack because the infrastructure can make optimizations that can’t be made with arbitrary containers. If your use case isn’t covered by a buildpack, then you can either (1) make a buildpack or (2) revert to raw containers. (Although your platform may not allow (2)).

sclevine 8y ago

For those who would like to better understand the technology: Presentation to CNCF TOC: https://docs.google.com/presentation/d/1RkygwZw7ILVgGhBpKnFNgJ4BCc_9qMG8cIf0MRbuzB4/edit#slide=id.g4054009e6b_5_0 https://docs.google.com/presentation/d/1RkygwZw7ILVgGhBpKnFN... Formal specification: https://github.com/buildpack/spec https://github.com/buildpack/spec Sample buildpacks: https://github.com/buildpack/samples https://github.com/buildpack/samples

nailer 8y ago
In other words: they're (typically) shell scripts that set up an OS to run a language. Here's the node one: https://github.com/buildpack/samples/blob/master/nodejs-buildpack/bin/build https://github.com/buildpack/samples/blob/master/nodejs-buil...

a012 8y ago

It's first time I heard of buildpacks, can I just throw in other than nodejs, ie. Laravel source code and build an image?

jacques_chester 8y ago
If there is a conforming buildpack, then yes. What you're looking at here is informally the "v3" effort, which extends and consolidates the "v2a" and "v2b" designs evolved by Heroku and Cloud Foundry respectively from the original Heroku design. In v2 both Heroku and Cloud Foundry provide supported PHP buildpacks, as well as Java, Ruby, Python, .NET Core and I forget the rest right now. There are hundreds of community buildpacks.
dreyfiz 8y ago
Yeah, do you have a composer.json in the root of your repo? Buildpack will detect a PHP app...
- dullgiulio 8y ago
  I have the feeling that the detect phase is absolutely orthogonal to delivering what should basically be an executable package. What is the rationale behind having the build step tied to the packaging format?
  - jacques_chester 8y ago
    It's not tied to the packaging format. Detect is the step that decides which buildpack or buildpacks will be responsible for constructing the image from the sourcecode. Typically this means that buildpacks look for files that correspond to the relevant ecosystem. Maven buildpacks look for pom.xml. PHP buildpacks look for composer.json. Etc. Nothing in this creates a hard binding. Detect steps may use whatever logic they need to decide on whether to signal they can work on a codebase. Edit: in the v3 design the detect script can also provide dependency information that later steps can pick up. So for example, a JDK buildpack can say "yes, I can interpret this codebase, and I can contribute a JDK". A later buildpack can then look for this contribution as a condition, eg. the Maven buildpack can say "I will proceed if I see a pom.xml and if there is a JDK available".

gigatexal 8y ago

What am I missing? Looking at the example for java it looks a lot more complicated and a step back from docker and its layers concept.

coredog64 8y ago
I don't think you're missing anything. It's vendors that don't do Docker trying to stay relevant in a Docker/Kubernetes world. By making Docker more complex, said vendors can continue to charge $500/pod for OSS k8s.
- jacques_chester 8y ago
  It's opensource and you can run it yourself, which has incidentally been true since before either Docker or Kubernetes existed. We've got an open PR to add Cloud Native Buildpack support to Knative Build. You can take these containers and run them on whatever Kubernetes you like. Disclosure: I work for a Said Vendor.
jacques_chester 8y ago
There are several audiences. 1. Developers. What you need to know is: it just works. You don't even need to write a Dockerfile any more. The buildpack turns your code into a well-structured, efficient runnable image with no additional effort. 2. Operators. What you need to know is: oh thank god no more mystery meat in production. Patching the OS is a ho-hum affair instead of a stone grinding nightmare that turns your developers into a white hot bucket of rage because you have to nag them or block them from deploying or both. 3. Platform vendors, buildpack authors and curious passers-by What you need to know is: All the other stuff about detect, analyse, build or export. Unless you are in group 3, the basic thing is that Buildpacks require less effort than Dockerfiles with more safety and faster builds.
sclevine 8y ago
Dockerfiles require you to rebuild lower layers when any upper layers change, even though the OCI image format doesn't care about this. Cloud Native Buildpacks can intelligently choose which layers to rebuild and replace. Additionally, certain layers can be updated en mass for many images on a registry (using cross-repo blob mounting, with no real data transfer!), which is useful for patching CVEs quickly at scale. The samples take advantage of this (as well as a separate transparent cache) in order to demonstrate the different aspects of the formal spec. A simple buildpack is not necessarily much more complicated than a simple Dockerfile.

deleted 8y ago

[deleted]

EdSchouten 8y ago

I currently use Bazel in combination with rules_docker to construct container images of applications, often written in Go. As I don't depend on cgo, my container images may be bare (i.e., only containing a Go executable and some static artifacts). Though I understand that there are likely only dozens of us, my question is: what would the use of Buildpacks buy us for this specific use case?

justinsaccount 8y ago
one case is that you could have a bazel buildpack that contains bazel and java and all that stuff requires in order to run the build. Unfortunately the resulting image would be larger than just a single binary, but building it would be a lot easier and repeatable for other people working on the project. The base layers would be cached though so in practice it might not be that much larger on disk. I looked into using bazel for doing similar things, and the biggest stumbling block is that bazel itself is a PITA to install on all platforms. May end up trying https://please.build/ https://please.build/ at some point.
- jbott 8y ago
  The bazel installation process has gotten a lot better lately - they now bundle OpenJDK inside the binary and it is only a single download.
  - justinsaccount 8y ago
    Unfortunately that's a non starter for things like debian.. but that's good to know, I'll give it another look. One thing I really like about bazel is the pkg rules.. I currently use fpm/goreleaser(nfpm) to build rpms for things, and it's nice having a single build tool that can build the app and spit out an rpm.
jacques_chester 8y ago
Mostly the advantage would be that 1) you wouldn't have to maintain the buildpack yourself, 2) operators can upgrade your software without needing to get you to do it, 3) upgrades might actually be faster due to layer rebasing and 4) if you add other stacks to your architecture, you still have the same development, deployment and update experience.

mosselman 8y ago

I just tried this on my rails project and it is detecting a nodejs project. Is there a way to have it use ruby instead while still taking care of my yarn dependencies?

justinsaccount 8y ago
This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. With Docker it is easy to make this work using multi-stage builds, I'm not sure if buildpacks account for this at all.
- jacques_chester 8y ago
  > This can be a problem with buildpacks, since you'd need one buildpack that has both the ruby and nodejs runtimes. This isn't strictly true of v2 designs (both Heroku and Cloud Foundry have schemes for multi-buildpack support) and in the v3 design explicit consideration is given to making mix-and-match a triviality. Buildpacks can cooperate quite easily and in multiple groups. Where this shines is in updates. If I have an OS base layer, a JDK buildpack and a Maven buildpack, the layers they generate can be independently updated without needing a complete rebuild. So far as I am aware, this is not currently possible with a Dockerfile, multibuild or not. If you invalidate the OS layer, everything else gets rebuilt whether it needed to be or not.
  - justinsaccount 8y ago
    That's cool, I'll need to read up on the v3 design. I think you are right about the rebuilds, but maybe not in all cases.. If you had. FROM nodejs:whatever as js RUN npm build FROM golang:whatever as go RUN go build FROM base COPY --from js app.js COPY --from go app If only 'base' was updated rebuilding the image would just need to re-run the COPY commands. The only way everything would get rebuilt is if you also updated the go and js images. I think for more complicated applications https://grahamc.com/blog/nix-and-layered-docker-images https://grahamc.com/blog/nix-and-layered-docker-images is ultimately the best way to do things... ensuring that every step is isolated with explicit dependency information.

josegonzalez 8y ago

> As one of the maintainers of Herokuish - a tool that manages the application of buildpacks to repositories and is used in quite a few Buildpack implementations - I am super happy that CNCF reached out to us and included us in the process. Oh wait, that didn't happen... All jokes aside, this looks great. Super-early of course - seems like there are quite a few issues in the `pack` repository to be implemented - but I'm excited to see where this lands. Buildpack detection and application is not a straightforward problem.

jacques_chester 8y ago
Please join us at https://slack.buildpacks.io/ https://slack.buildpacks.io/ ! (ps. I know what these surprises can feel like -- for what it's worth, I am sorry)

m-arnold 8y ago

What is the difference between buildpacks and packer?