6 ms·

Breaking up the Container Monolith

jmspring 9y ago
I've not really been a fan of the Redhat ecosystem for years. Fedora, RHEL, etc. while "stable" always feel like if I want to build something a bit more modern requires contortions that even Debian Stable did away with years ago. As for "Buildah", what about Deis and build packs deployed to Kubernetes? RH has gotten better in recent years with Openshift syncing closer to Kubernetes releases, but are still a far cry from how CoreOS handles things with Tectonic.
- noja 9y ago
  You are choosing the wrong distro if you want modern. Red Hat Linux is about stability over a long period of time. Don't have a clue what you're talking about with Fedora, it does exactly what you want.
- jacques_chester 9y ago
  I've worked on Cloud Foundry buildpacks. Something that does a better job at assembling OCI images than Dockerfiles is still welcome. Buildpacks have the advantage that the developer doesn't need to care about anything below their own direct dependencies. The runtime and OS are kept up to date by the platform, so you're relieved of the broadest angles of attack. It's not a panacea. Just as with Dockerfiles, you have other upstream dependencies (Ruby gems, NPM modules, JARs etc) that need to be tracked and updated. Some of that can be delegated to the buildpack (ie, setting version range specifiers like '^1.1.2' or '1.1.*'), but that introduces some uncertainty about what is actually running in production. Deis I believe did what we originally did in Cloud Foundry, which is to use Heroku's buildpack code. We ran soft forks for several years, adding only enough to allow execution in disconnected environments. The Buildpacks team have since adopted a hard fork policy and have been replacing Heroku code with a uniform codebase called libbuildpack[0]. The new ones pass the same test suite we run and should work for Deis. [0] https://github.com/cloudfoundry/libbuildpack https://github.com/cloudfoundry/libbuildpack
- veeti 9y ago
  Huh? Fedora is one of the most bleeding edge, "modern" distributions you'll find.
  - Spivak 9y ago
    And then you can move to Rawhide if that still isn't new enough.
numbsafari 9y ago
Embrace and extend?
pmahoney 9y ago
Broken link to Buildah https://github.com/projectatomic/buildah https://github.com/projectatomic/buildah
e_d_e_v 9y ago
They complain about Dockerfiles and have a broken link to some project with 288 commits, without even acknowledging that a Dockerfile isn't needed to build a docker image. It is just as easy to build from a regular old disk image, a chroot, etc. I don't blame RedHat for trying to capitalize on the container frenzy, but this doesn't reflect well on them. Also, kind of expected more from the SELinux Coloring Book.
- equalunique 9y ago
  I don't put expectations on coloring books.
- imtringued 9y ago
  I also feel like Docker's multistage feature already solves the container size flaw. >This tool also allows us to build containers without all the container development tools inside it (yum, GCC, secrets, etc). Basically, Buildah produces clean containers, which means the container can be much, much smaller. With docker you basically have one stage where you load all the dev tools to build the application or library. Then you have a second stage where you merely copy your application and run it. I often have a Dockerfile that builds a java library which then merely copies the .jar file into an empty docker image. The end result is usually a docker image smaller than 10MB. Then I add this image to the dockerfile that actually needs the library by adding it as an empty multistage and as part of the last stage I just copy the files of the library image into the application image. This way you can "inherit" multiple docker library images into one application image.
  - user5994461 9y ago
    >>> The end result is usually a docker image smaller than 10MB. You forgot to install the JVM to run the .jar, didn't you?
    - devonkim 9y ago
      A possible option for containers that all run JVM applications is to have a volume mount for all of them that points to an external filesystem containing the JAVA_HOME contents. This somewhat violates certain containerization principles but can help reduce some disk usage across dozens of containers on a host.
frigen 9y ago
Unikernels are a better simpler tech than containers but somehow we're stuck with containers.
- robert_foss 9y ago
  How does a unikernel solve all of the same problems as container?
- jacques_chester 9y ago
  Containers don't require a per-technology implementation, so it's easier to adapt existing systems to use them.
Siilwyn 9y ago
Also no mention of https://github.com/rkt/rkt https://github.com/rkt/rkt ?
- whipoodle 9y ago
  Oh yeah, what's happening with that? Haven't heard anything about it lately
atemerev 9y ago
Dockerfile is the best thing Docker has to offer. Simple, text-based, easy to learn, consistent enough for practical use. What's the problem?
- jacques_chester 9y ago
  * Single inheritance. Dockerfiles cannot be easily composed, you need to arrange them in particular order. This leads to either kitchen-sink images or deep, brittle hierarchies. * Leading to: inheritance for construction. Lots of images include a bunch of irrelevant material because it was easier to FROM an existing image than to tease apart the kitchen-sink into the brittle hierarchy. * Meaning: using composition isn't really "a thing" in the sense that it's commonly used. I'm given to understand that OCI layers can, if you recalculate the hashes, be layered in arbitrary order. In an ideal world the build tool can check that there won't be layer conflicts, meaning you can treat layers as immutable mixins, rather than immutable parents. Instead of FROM, you could have UNION or PLUS. Then of course the less abstract stuff: * Mystery meat. Dockerfiles make development solely responsible for managing all dependencies from the OS up. An awesome tool and a problematic responsibility. You've graduated from worrying about XSS holes in your web framework to fretting about kernel bugs and glibc glitches. Where I work we have an entire team devoted to tracking CVEs that affect code we ship. Do you? * Mystery meat, at scale. Now you have 5,000 containers running ... what, exactly? What versions of what OSes? Are the runtimes up to date? You're going to need to either rigidly control Dockerfiles -- in which case, what did it buy in terms of dev flexibility? -- or you're going to need to buy tools to scan all your images. Docker sells such a service, as it happens. Dockerfiles are a simple, clean exposure of the original concept of layered FSes, processes and ports. But they are not without limitations. Disclosure: I work for Pivotal, we compete with Red Hat and Docker in the PaaS space.
  - atemerev 9y ago
    I don't know if the problems you describe can be solved by anything less than fully declarative approach; and even NixOS isn't there yet.
    - 9y ago

I've not really been a fan of the Redhat ecosystem for years. Fedora, RHEL, etc. while "stable" always feel like if I want to build something a bit more modern requires contortions that even Debian Stable did away with years ago. As for "Buildah", what about Deis and build packs deployed to Kubernetes? RH has gotten better in recent years with Openshift syncing closer to Kubernetes releases, but are still a far cry from how CoreOS handles things with Tectonic.

noja 9y ago
You are choosing the wrong distro if you want modern. Red Hat Linux is about stability over a long period of time. Don't have a clue what you're talking about with Fedora, it does exactly what you want.
jacques_chester 9y ago
I've worked on Cloud Foundry buildpacks. Something that does a better job at assembling OCI images than Dockerfiles is still welcome. Buildpacks have the advantage that the developer doesn't need to care about anything below their own direct dependencies. The runtime and OS are kept up to date by the platform, so you're relieved of the broadest angles of attack. It's not a panacea. Just as with Dockerfiles, you have other upstream dependencies (Ruby gems, NPM modules, JARs etc) that need to be tracked and updated. Some of that can be delegated to the buildpack (ie, setting version range specifiers like '^1.1.2' or '1.1.*'), but that introduces some uncertainty about what is actually running in production. Deis I believe did what we originally did in Cloud Foundry, which is to use Heroku's buildpack code. We ran soft forks for several years, adding only enough to allow execution in disconnected environments. The Buildpacks team have since adopted a hard fork policy and have been replacing Heroku code with a uniform codebase called libbuildpack[0]. The new ones pass the same test suite we run and should work for Deis. [0] https://github.com/cloudfoundry/libbuildpack https://github.com/cloudfoundry/libbuildpack
veeti 9y ago
Huh? Fedora is one of the most bleeding edge, "modern" distributions you'll find.
- Spivak 9y ago
  And then you can move to Rawhide if that still isn't new enough.

numbsafari 9y ago

Embrace and extend?

pmahoney 9y ago

Broken link to Buildah https://github.com/projectatomic/buildah https://github.com/projectatomic/buildah

e_d_e_v 9y ago

They complain about Dockerfiles and have a broken link to some project with 288 commits, without even acknowledging that a Dockerfile isn't needed to build a docker image. It is just as easy to build from a regular old disk image, a chroot, etc. I don't blame RedHat for trying to capitalize on the container frenzy, but this doesn't reflect well on them. Also, kind of expected more from the SELinux Coloring Book.

equalunique 9y ago
I don't put expectations on coloring books.
imtringued 9y ago
I also feel like Docker's multistage feature already solves the container size flaw. >This tool also allows us to build containers without all the container development tools inside it (yum, GCC, secrets, etc). Basically, Buildah produces clean containers, which means the container can be much, much smaller. With docker you basically have one stage where you load all the dev tools to build the application or library. Then you have a second stage where you merely copy your application and run it. I often have a Dockerfile that builds a java library which then merely copies the .jar file into an empty docker image. The end result is usually a docker image smaller than 10MB. Then I add this image to the dockerfile that actually needs the library by adding it as an empty multistage and as part of the last stage I just copy the files of the library image into the application image. This way you can "inherit" multiple docker library images into one application image.
- user5994461 9y ago
  >>> The end result is usually a docker image smaller than 10MB. You forgot to install the JVM to run the .jar, didn't you?
  - devonkim 9y ago
    A possible option for containers that all run JVM applications is to have a volume mount for all of them that points to an external filesystem containing the JAVA_HOME contents. This somewhat violates certain containerization principles but can help reduce some disk usage across dozens of containers on a host.

frigen 9y ago

Unikernels are a better simpler tech than containers but somehow we're stuck with containers.

robert_foss 9y ago
How does a unikernel solve all of the same problems as container?
jacques_chester 9y ago
Containers don't require a per-technology implementation, so it's easier to adapt existing systems to use them.

Siilwyn 9y ago

Also no mention of https://github.com/rkt/rkt https://github.com/rkt/rkt ?

whipoodle 9y ago
Oh yeah, what's happening with that? Haven't heard anything about it lately

atemerev 9y ago

Dockerfile is the best thing Docker has to offer. Simple, text-based, easy to learn, consistent enough for practical use. What's the problem?

jacques_chester 9y ago
* Single inheritance. Dockerfiles cannot be easily composed, you need to arrange them in particular order. This leads to either kitchen-sink images or deep, brittle hierarchies. * Leading to: inheritance for construction. Lots of images include a bunch of irrelevant material because it was easier to FROM an existing image than to tease apart the kitchen-sink into the brittle hierarchy. * Meaning: using composition isn't really "a thing" in the sense that it's commonly used. I'm given to understand that OCI layers can, if you recalculate the hashes, be layered in arbitrary order. In an ideal world the build tool can check that there won't be layer conflicts, meaning you can treat layers as immutable mixins, rather than immutable parents. Instead of FROM, you could have UNION or PLUS. Then of course the less abstract stuff: * Mystery meat. Dockerfiles make development solely responsible for managing all dependencies from the OS up. An awesome tool and a problematic responsibility. You've graduated from worrying about XSS holes in your web framework to fretting about kernel bugs and glibc glitches. Where I work we have an entire team devoted to tracking CVEs that affect code we ship. Do you? * Mystery meat, at scale. Now you have 5,000 containers running ... what, exactly? What versions of what OSes? Are the runtimes up to date? You're going to need to either rigidly control Dockerfiles -- in which case, what did it buy in terms of dev flexibility? -- or you're going to need to buy tools to scan all your images. Docker sells such a service, as it happens. Dockerfiles are a simple, clean exposure of the original concept of layered FSes, processes and ports. But they are not without limitations. Disclosure: I work for Pivotal, we compete with Red Hat and Docker in the PaaS space.
- atemerev 9y ago
  I don't know if the problems you describe can be solved by anything less than fully declarative approach; and even NixOS isn't there yet.
  - 9y ago