8 ms·
Microcontainers – Tiny, Portable Docker Containers
- _Marak_ 11y agoThis seems to solve one of my biggest issues with trying to use individual Dockers to run microservices. Past experiments proved that it wasn't feasible to match one microservice request to one Docker instance. The size of each Docker required vertical scaling of the microservices per container, which was not ideal. Good job Iron.io! Might trying playing with these at some point.
- TheDong 11y ago... Why? The scalability difference (in terms of disk space) between having multiple services per container and many containers is, in practice, zero. That's because of layered filesystems and most of the language runtimes and other images pulling from a small set of base images. For example, if you have two node applications, the base "nodejs" image is 633MB in size. If both of your applications have "FROM node" at the top and proceed to add different applications (say 2MB each), you'll find the "node" portion is totally shared and only the diff, you copying in your application's files, is different. You'll only store a single copy of the 633MB node image + your two application image's diffs, for a total of 637MB. That number is the same if you bundle the applications (vertically scale as you say it). This is also sorta true across language runtimes too! With some creative diffing, I experimentally tested the difference between the ruby and nodejs images. It turns out there's just under 600MB in common. The ruby image has 116MB of data the node one doesn't, and the node one has ~40MB of data the ruby one doesn't. Again, you'll end up using a very small amount of disk space. These savings also apply to downloading images after the initial base layers are downloaded (e.g. docker pull ruby; docker pull node will only download roughly 750MB of data, even though that's two >600MB images). In any case, it's strange that disk space on the order of a couple gigs is even a scaling concern at all; I'd imagine memory or your application's disk needs would far outweigh such concerns. My main point, however, is that your discussion of one service per container vs multiple services per container being any different in terms of disk space is rubbish and utterly false.
- _Marak_ 11y agoI would assume the smaller images would also result in a smaller memory footprint for running the images and a general reduction in time of starting images. You seem to know a lot of about Docker, is that a wrong assumption? The scale which I'm discussing is in the order of at least several hundred docker images per second. Previous attempts at making this work involved keep a warm elastic pool of Dockers. I'm working with at least 11 environments, ( which all have separate dependency requirements ). Instead of trying to manage a very large pool of Dockers, I opted for a smaller pool with several larger servers to scale the microservices vertically ( using tools like chroot to help try to isolate each service per silo ). My main issue with using Docker for this was the bulk of the containers. Startup time, RAM consumption, and the size of the images were all causing me issues.
- lucaspiller 11y agoDocker isn't a VM, so the memory usage should be pretty much on par with chroot. The only difference is shared libraries will need to be duplicated in each container (as nothing is shared) and loaded into memory multiple times, but that should be on the order of a few megabytes.
- davexunit 11y agoThe duplication is worse than that. It's a data structure problem. Docker deals in opaque disk images, a linear, order-dependent sequence of them. The data structure is built this way because Docker has no knowledge of what the dependency graph of an application really is. This greatly limits the space/bandwidth efficiency Docker can ever hope to have. Cache hits are just too infrequent. So how do we improve? Functional package and configuration management, such as with GNU Guix. In Guix, a package describes its full dependency graph precisely, as does a full-system configuration. Because this is a graph, and because order doesn't matter (thanks to being functional and declarative), packages or systems that conceptually share branches really do share those branches on disk. The consequence of this design, in the context of containers, is that shared dependencies amongst containers running on the same host are deduplicated system-wide. This graph has the nice feature of being inspectable, unlike Docker where it is opaque, and allows for maximum cache hits.
- lox 11y agoDo we really need to brand using smaller containers as microcontainers? Why not submit these as pull requests to the official docker images rather than introducing more fragmentation? Also worth noting that the official docker images are moving towards alpine. Of course if you have ruby or node apps your biggest space hog is still going to be your packages.
- TheDong 11y agoThere's also a very good reason to not move some things to alpine, including ruby and node applications. A significant number of ruby and nodejs apps depend on some c/c++ code hidden away in this module or that gem. Any time you're compiling c/c++ code, you run into a chance of musl vs libc causing differences. These differences can range from building requiring different options to the application having strange performance characteristics to random crashes (rare!). In addition, people who write these modules typically test against an ubuntu or debian system, so those are much safer bets. Alpine makes no serious attempt to match packages with those, either in breadth nor version. It really isn't feasible to unilaterally move people, and for compatibility reasons Docker does not often change tags in such a drastic way. The official docker images are often offering alpine alternatives in different tags, which is not the same as "moving towards" as you remarked.
- shykes 11y agoWe are in fact moving towards alpine, but can't just switch the default tags overnight in case some users depend on the properties of certain distros.
- ishtu 11y agoexcited to see recently added support[1] of dns search domains in musl (and therefore Alpine). [1] http://git.musl-libc.org/cgit/musl/commit/?id=3d6e2e477ced37fd328870f018950b283cb7293c http://git.musl-libc.org/cgit/musl/commit/?id=3d6e2e477ced37...
- kylequest 11y agoAlpine is an option in some cases, but, like you said, it's not always straight forward. Native application modules, 3rd party OS packages and the overall package echosystem differences are important factors that need to be considered. This is why DockerSlim [1] was created. You use a regular distro like Ubuntu and you still get micro containers! Note that the sample node microcontainer with DockerSlim is 14MB and in this post it's 29MB :-) It uses hapi.js instead of express.js though, but it's probably not the main reason for the size difference. [1] http://dockersl.im http://dockersl.im
- schappim 11y agoI have to say I love the work iron.io does. Their "iron worker"[1] service is awesome (think Amazon Lambda for pretty much any common language). [1] http://www.iron.io/worker/ http://www.iron.io/worker/
- markherhold 11y agoInteresting. How long does it take an "iron worker" to start code execution after a webhook (latency)?
- treeder 11y agoTypically a couple seconds.
- antouank 11y agoIsn't it the same as using "alpine" as base ? https://news.ycombinator.com/item?id=10782897 https://news.ycombinator.com/item?id=10782897
- woadwarrior01 11y agoIt looks like their base image is based off of the alpine image[1]. [1]: https://github.com/iron-io/dockers/blob/master/base/Dockerfile https://github.com/iron-io/dockers/blob/master/base/Dockerfi...
- _query 11y ago> Docker enables you to package up your application along with all of the application’s dependencies into a nice self-contained image. You can then use use that image to run your application in containers. Actually docker was created to enable easy scaling of web applications. If you want to package your application up take a look at some modern package managers. By using a modern package manager like nix you can work around the issues of containers while still keeping the "it just works" advantage of containers. This does not mean that containers are bad, it's just that using a good package manager is usually a simpler solution. Take a look at my blog post if you want to read more on this: http://www.mpscholten.de/docker/2016/01/27/you-are-most-likely-misusing-docker.html http://www.mpscholten.de/docker/2016/01/27/you-are-most-like...
- pea 11y agoYes, layering is somewhat of a hack to get around the fact that most OS package managers are mutable. IMO nix / guix have a much better approach here.
- foldr 11y agoI don't know if things have changed in the past few months, but the last time I tried nix it's used too much memory to make it practical to run it within a VM on my laptop.
- bboreham 11y ago> docker was created to enable easy scaling of web applications Citation? From my perspective, most of the features you'd need to do that appeared around 2 years into Docker's life.
- _query 11y agoInitial docker was part of dotCloud, a platform-as-a-service company[0]. From the perspective of a paas-company container technology allows for easy scaling. From the perspective of a docker user you're right that many of these tools were not available until recently. [0]: https://en.wikipedia.org/wiki/Docker_(software)#History https://en.wikipedia.org/wiki/Docker_(software)#History
- jorgecurio 11y agohow are you guys using docker btw? I keep hearing about it, I like it but have almost to no reason to use it.
- tombh 11y agoAmongst other things I use it to run Skype on my Linux laptop, cos I don't trust Microsoft. https://github.com/sameersbn/docker-skype https://github.com/sameersbn/docker-skype
- kasey_junk 11y agoWhat trust does docker give you? It seems less secure in this instance than properly permissioning skype or creating a true vm for it?
- dc_new 11y agoI use it for dev to more closely mock prod, as well as ensure repeatable builds. It allows me to run all the services we use locally without a separate vm for each.
- mikewhy 11y agoWe use it in development to match production environments We ditched Heroku + S3 for dev/test/staging branches (if your app can be expressed in a docker-compose.yml file you can get a server up with the command `b3cmd server-scaffold`) Many small tools for random workflows
- cyphar 11y agoYou can use it to run desktop apps if you're as anal as me about having a clean host machine. I also use it to run all of my small services (my website, IRC bouncer, etc).
- tmd 11y agoDocker storage engines remember all the layers by storing incremental differences so it makes little sense to have a layer that only removes things (like the last RUN in https://github.com/iron-io/dockers/blob/master/ruby/Dockerfile https://github.com/iron-io/dockers/blob/master/ruby/Dockerfi...). You can see with `docker history <image>` that this layer has 0 bytes and you can access anything that it removes by using the parent image (`docker run -it 88ae7e32865f ls /var/cache/apk`). In this case it doesn't make much difference (a couple of MB perhaps) but often you'll see people chaining all the commands and ending it with a cleanup in a single RUN like here: https://github.com/docker-library/ruby/blob/ccbf9e54f60ba2455f4f18e223c5745a14657bdb/2.0/Dockerfile https://github.com/docker-library/ruby/blob/ccbf9e54f60ba245...
- grkvlt 11y agoHeh, looks like the author fixed this [1] and it now uses the technique you suggest, based on issue #22 [2] which restates the same thing as your comment, basically chaining everything together with `&&` in a single `RUN` entry. [1] https://github.com/iron-io/dockers/commit/71687edb849dcb9079572f1de251106f6e666ed9 https://github.com/iron-io/dockers/commit/71687edb849dcb9079... [2] https://github.com/iron-io/dockers/issues/22 https://github.com/iron-io/dockers/issues/22
- bmajz 11y agoWhile it's nice to have small on-disk containers for some applications (e.g. deployment pipelines/CI), for production, I've found that Alpine doesn't save you much in RAM. I'd love to see this become something people look at as well when evaluating base images. To me at least, this was a far more important constraint when running Docker in production.
- bmajz 11y agoTo be fair, the case I was testing was node, so the packages could be killing me. Could be much better for a standalone bundled application.
- BenjaminCoe 11y agoI believe Alpine is GPL licensed. Curious what companies are using Alpine? What ramifications does this have on the licensing of a micro service running in Alpine.
- danieldk 11y agoThe GPL only comes into effect when you redistribute software. Since most companies will probably use it to deploy their own software, GPL does not matter much in practice. Besides that, for an application only the licenses of libraries that you link against are relevant.
- cyphar 11y agoYou can't use the GPL for an entire distribution. You could argue that it's possible to only package GPL-compatible software and sublicense all of it under the GPL. This is impractical because of MPL and LGPL libraries. But even then the distribution isn't under the GPL, so you could inject an "MIT version" of musl and other non-GPL versions of the rest of the software. tl;dr: That's not how licenses work. There have been some interesting questions about packaging Ubuntu and their terms of use. I always found their terms of use odd, because they are basically reinstating trademark law inside a software license.
- mullsork 11y agoI thought everyone was doing this already. The first thing I noticed when pushing my first test version of our Node server was that the images was way above what I thought it should be. It seems really odd that the "official" Docker images are this big. Perhaps to enable easy development within the VMs, and not meaning for these to be distributed?
- ecesena 11y agoRelated repo with micro containers for each language: https://github.com/iron-io/dockers https://github.com/iron-io/dockers
- philipwood 11y agoI've always felt that VMs are a sign that show that OSes have imperfectly abstracted the virtualization of their resources. A "design smell" if you will. When you get right down to it a window is a virtualization of a screen, a packet switching network is a virtualization of a fixed line, a process is a virtualization of a CPU, etc. (I can't remember who I'm quoting). The direction this is all heading (VMs, containers, unikernels, etc.) feels like confirmation of this to me. A series of corrections aligning our OSes with more ideal/virtual abstractions.
- sirtaj 11y agoPlan 9 was a realization of this line of abstraction, in a lot of ways. Even though I never used the platform for anything serious, I still draw inspiration from it all these years later.
- ohdrat 11y agoI predict that the term "micro" will be replaced by the term "modular", and that docker will create a replacement for the microcontainer named the modular loader/container.