23 ms·

WTF is a container?

366 points by kawera 10y ago

chewxy 10y ago
This is not quite related to Docker/Kubernetes, but a phrase in the article triggered memories about a very good book that I think everyone should read - The Box by Marc Levinson. It really is a fascinating book.
- haalcion3 10y ago
  Definitely "not quite", as it is about shipping containers, but an interesting read.
bbcbasic 10y ago
I've read before on HN about 'don't use docker in production' can anyone elaborate on that for a newbie?
- knowaveragejoe 10y ago
  My understanding is that plenty of people do actually use Docker in production, but they use more than just Docker. A docker container isn't quite the same as a VM.
- hvidgaard 10y ago
  There is a lot of infrastructure around security and availability that isn't yet achiveable with off the shelf software. So ideally you use containers to define dependencies and deployment, and deploy into a proper environment.
simula67 10y ago
Putting a little bit more thought into naming things can help newcomers understand these concepts more easily. The word container is too generic, why not call it an 'application jail' or a 'virtual operating system' ? Another word that is over used in our field is 'context'.
- lgas 10y ago
  It's based on the idea of container ships where the contents of every container can vary widely but the external interface for dealing with the containers is always identical. It's a pretty apt metaphor and not too generic, IMHO.
- stass 10y ago
  They were called "jails" long before "containers" became a thing on HN. Somehow "jails" didn't stick, so I'd assume it was even more confusing for newcomers.
  - santaclaus 10y ago
    > Somehow "jails" didn't stick I hear chroot jail tossed around a fair amount.
  - chii 10y ago
    Jail had no marketing hook. It's a very negative word. I'd guess that's Probably the major reason.
    - hueving 10y ago
      Well that and it wasn't done by a VC backed firm with strong press connections.
  - jsjohnst 10y ago
    Jails is typically the FreeBSD name for a very similar concept. Linux "copied" (using the term very loosely) the idea, but not the name.
  - jbverschoor 10y ago
    I was running a couple of jails on my FreeBSD server 15 years ago.. Worked like a charm.. But eventually I switched to linux.
dancek 10y ago
I agree that containers (both for shipping and servers) are a great idea. And because I'm tired of always configuring servers, I decided to give it a try some time ago. I wrapped my IRC client (weechat + glowing-bear) in a Docker container. Oh, not a container though, because I also needed https, which meant I needed either a mechanism to build and update letsencrypt certs in the weird format that weechat expects, or to run an nginx instance in the front (and also somehow get the certs, but that's easier with nginx). So two containers. And even though there was a ready-made nginx+letsencrypt https reverse proxy container (actually several), I had a huge amount of headaches to get it actually working. Even with the system set up, I occasionally have the container crash with exit status 137 (IIRC), which I've assumed might be because weechat leaks/consumes memory and eventually the host server kernel kills the process. Maybe. So in my limited experience, comparing Docker containers to shipping containers is a gross simplification. Shipping containers are simple constructions requiring well-defined simple maintenance, while Docker containers seem to be complex thingamabobs that have multiple points of failure.
- hvidgaard 10y ago
  I wanted to write about my experience with containers. It mimics yours. The idea of a container is beautiful and elegant. The execution, not so much. I'm more or less convinced that to reduce this complexity, we need a limited scope ecosystem. Microsofts .Net would probably be a good place to start, but MS, being enterprise, have a habit of turning things into a mess of configuration.
- chewchew 10y ago
  Docker gives you the building blocks, but that means you have more pieces to arrange and manage. Take a look at Docker Compose if you haven't already, since the Docker CLI only gets you so far when you're creating apps that consist of multiple containers. I think the best approach for your cert issue is to abstract that into a separate service (nginx is an option, but I'd recommend the Rancher approach below). Yes, that means you have to add another container, but that's just another block in your docker-compose.yml file. Embrace the approach of separating your components into containers and organizing them as a stack. You can easily link containers together, share data volumes, and start/stop individual containers or the stack as a whole. The problems that you're having are pretty easy to fix with some tooling. Rancher (http://rancher.com/ http://rancher.com/) greatly simplifies the cert issue by allowing you to import certs and provide them to the Rancher loadbalancer service (which you can add to any stack). There's also a LetsEncrypt community catalog template that automatically retrieves and imports certificates to Rancher. There are other open source orchestrators like DCOS, but Rancher is probably the simplest to use, and it's the only one I'm very familiar with. There are SaaS options that you can look into, but I don't have experience with them. As for the container crashes, it's trivial to automatically restart them. Just pass the --restart=always flag to the Docker run command. You can also add the flag to a docker-compose.yml file.
dkarapetyan 10y ago
> The promise behind software containers is essentially the same. Instead of shipping around a full operating system and your software (and maybe the software that your software depends on), you simply pack your code and its dependencies into a container that can then run anywhere — and because they are usually pretty small, you can pack lots of containers onto a single computer. Already got it wrong. Current containers are exactly the OS and the kitchen sink for running 'printf("hello world")'.
- chewchew 10y ago
  It's up to you to make your containers bloated or keep them slim. You can use the alpine versions of the official Dockerhub images. Python on Alpine is 30 MB (vs 267 MB for the debian one). https://hub.docker.com/r/library/python/tags/ https://hub.docker.com/r/library/python/tags/ You can create containers that are just a few MB with compiled languages like Go (5 MB). https://www.iron.io/microcontainers-tiny-portable-containers/ https://www.iron.io/microcontainers-tiny-portable-containers... From the article: "Rather than starting with everything but the kitchen sink, start with the bare minimum and add dependencies on an as needed basis."
  - stephenr 10y ago
    If your app is a compiled go binary (so, it runs anywhere) why do you need a container? The whole point of containerisation is to group installed dependencies (as opposed to installable dependencies like with a regular deb or rpm package) and configuration into a 'black box'. If your binary is already a single-file distribution, why lump it in with the crapfest that is docker?
    - chewchew 10y ago
      I don't think that's the whole point. If all you care about is packaging your code in a container, then you don't need Docker. That was solved long ago. Docker simply adds a nice API on top which allows you to package, ship, and manage your apps in the same exact way. By using something like Docker Compose, you can explicitly define each container and the relationships between the containers. Then, you can start/stop groups of containers (an application stack) while still retaining some component isolation and the ability to upgrade and scale containers independently. All of this is defined in a relatively simple YAML file, which can be committed to VCS and tweaked. I can't tell you how awesome it is to find repos on Github that have use docker-compose. Even complex apps with many pieces can often be launched with a single command. It's easy to take these stacks and tweak them to your needs, adding/removing/changing components as necessary. Since Docker provides a standard way for managing any container, orchestrators like Swarm, Rancher (my preference for small-medium clusters), and DC/OS can take that functionality and scale it across many hosts. You can get a birds-eye view of your Docker cluster (all apps) or drill down into individual apps and their components. Each container is a uniform piece and can be controlled, scaled, updated, audited in the same way. Throw a UI in front of it and now you can manage applications that you know nothing about. That's great for developers that manage just a few apps, and it's incredible for enterprises with thousands of them. If you don't want to think about infrastructure at all, then you can use a SaaS Docker provider. Obviously there are pros and cons to each approach, and there are some remaining challenges. Docker isn't perfect, especially in regards to storage and networking. Distributed storage isn't simple, but a lot of progress has been made with volumes and volume drivers. It's not as easy as it needs to be, but the general direction seems to be good and with the proper tooling I think this will be less of an issue.
fenomas 10y ago
I'm only a beginner, but the analogy that makes sense to me is that containers do for app deployment what npm does for Javascript development. That is, the magical part isn't that Docker simulates an operating system and so on - the magic is that it allows a chunk of logic to precisely declare its dependencies - including on other pieces of logic which declare their own dependencies - and then Docker knows how to (in theory anyway) run the logic in such a way that its dependencies are all satisfied. And of course the meta-magic is then that there's a public registry of (in theory) solved problems, which one can build on top of by declaring dependencies against them. I have a pet theory that this "declared dependencies + dependency wrangler + public registry" is a general formula which will keep cropping up as we find new places to apply it.
- kpil 10y ago
  Not really - Linux package management systems do exactly that. The magic - if there is any - is to combine it all together; separation, discovery, relatively easy packaging and dependencies.
  - fenomas 10y ago
    Sure, at a different level. Package managers at the app level, docker at the deployment level, npm at the development level, or something vaguely along those lines. I wasn't suggesting this was unique to Docker - precisely the opposite, that it's a generally useful pattern, being applied here to deployment.
- bpizzi 10y ago
  To me it's more like a lightweight virtualization: processes inside containers have no clues that there are others processes running in others containers alongside its own, all of that without eating too much memory (at least well under the amount that virtual machines would use). Added benefits: - the 'host' os can be very light and tailored to run Docker and nothing more (cf CoreOS), - we can design orchestration software that handle containers operations across a herd of lightweight hosts (cf Kubernetes). The dependencies/recipes thing is more like a tool that enable those higher goals, but again that's just my point of view.
disposablezero 10y ago
Weak virtualization often lacking security, resource metering/prioritizing/quotas. Hurray, zombie Docker instances needs a whole VM reboot yet again, still not fixed in several years. But look how fast I can deploy millions of containers without SELinux, monitoring, HIDS, SDN, billing, live migration, backup/restore/DR/data lifecycling and all the other things we just pretend to ignore when throwing away sensible production VMs on Type 1 hypervisors devopsec.
- hvidgaard 10y ago
  The smart use of containers is using it to specify the deployment. You should use proper virtualization as the environment to deploy into. Because you use containers doesn't mean you throw all that away.
sytelus 10y ago
These hand-wavy explanations that constantly avoids explaining how things work at low level are not adequate. Here's short explanation for developers even with moderate understanding of how OSes work: http://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine/34757096#34757096 http://stackoverflow.com/questions/16047306/how-is-docker-di...
deleted 10y ago
[deleted]
parito 10y ago
for the love of god - forget docker, use lxc containers - its simple, secure, goes with its own init, cron, and you dont need to do somersaults to achieve simple tasks. Included with linux kernel. Your own isolated linux system. We use lxc in production for over three years, and we have over 3000 of them. No issues whatsoever.
- austinjp 10y ago
  Encouraging to hear. Who do you work for, who has these 3,000 LXC containers in production use? And I'm curious, what orchestration system do you use to manage them? Can you outline your toolset?
  - parito 10y ago
    We use ansible and bash scripts for orchestration.
    - austinjp 10y ago
      If you'd care to go into more detail, or could point me to your technical docs, I'd be very interested. How, for example, do you handle roll-out, destruction, monitoring, backups, network configs, secrets, etc etc... is there any degree of automation? And why not take advantage of existing orchestration solutions? Was nothing mature enough for your needs? How big is the team managing these 3,000 containers, what sort of traffic are you handling? Asking out of genuine curiosity. I'm keen on understanding how plain LXC can be used robustly in a production environment.
- choxi 10y ago
  You're right that LXC containers have a similar API compared to Docker, but I think developers often underestimate the benefit of the community around a certain technology. Docker has significantly better documentation, extensions, package management tools, and third-party integrations. Overall, Docker has an incredibly more robust community than LXC or closer competitors like Kubernetes, and those features are just as important as the API for developers.
sztwiorok 10y ago
Docker is the best thing since sliced bread! VM is much more resource and time consuming. # We switched our dev/stage env to containers 2 year ago. # We have made our own standalone app in Docker style. Once again - Easy as pie. if you are a developer you should add Docker + Docker Compose to you working tools.
- cZuLi 10y ago
  We moved over to Docker at the development process some time ago and it really speed up our work.
jdoliner 10y ago
This is a pretty good analogy for containers but there's an unfortunate conflation of terms. There's actually a distinction with computing containers between the thing that you ship code around as and the thing that you run code in. The latter is the real container while the prior is called a "container image" or often simply an "image." This gets confusing quick if you apply this analogy since you assume that the thing you ship the code as would naturally be called the container.
lottin 10y ago
I guess I'll never get it. Don't most OSs already run processes isolated from each other, have advanced process scheduling mechanisms and manage access to hardware resources? Also with static linking nothing stops you from creating huge binaries that "will run anywhere".
- justincormack 10y ago
  Not with a private root filesystem, a private loopback network, additional security isolation, no requirement for static linking, no.
- greggman 10y ago
  In containers a few more things are virtualized. The file system is semi-virtualized. Network ports are too. So from the pov of stuff inside the container nothing else is running. That's not true of processes in general. From outside the containers you can then choose how to map parts of the virtual file systems to parts of the real file system and what real network ports the virtual ports connect to etc... There's more to it. Containers aren't one process theyre as many processes as are launched in that container
  - sundarurfriend 10y ago
    So let me try to understand this from a different angle: what's something that a VM can and does do, that container software like Docker can't? TFA makes it sound like legacy systems is the only place for VMs anymore, but I'm guessing that's probably approximation+exaggeration.
    - austinjp 10y ago
      In my (limited) understanding: host OS and guest OSes can be completely unrelated. Whereas in containers, the host and guests share the kernel.
- austinjp 10y ago
  From my perspective, where I'm planning a hosting service with multiple customers, containers promise to allow me to slice machines into dedicated chunks. VMs could do this too, but they'd consume far more disk-space, which would mean my hosting costs would be comparatively higher. Containers promise to allow me to limit the resources each of my customers can consume without killing the entire box. For example, I can limit RAM, CPU and disk-space per customer. If one customer goes rogue, the shared box remains performant for my other customers. Also there's the data protection: in theory customers would not be able to access each others' data, even if they tried to. There are other considerations as others here have pointed out. This is just my primary concern at the moment.
ChoHag 10y ago
Take an operating system. Remove all the advantages of a shared environment like dynamic libraries, package management, clarity. Stick a chroot before every fork. Boom! Containers.
Hondor 10y ago
I never quite understood containers and this article makes them seem kind of similar to what OSs already do. How is a container different from just installing all the dependencies along with an application? Coming from a Windows background, this is pretty common to avoid DLL hell. Nobody distributes a Windows application that requires the user to go and install some 3rd party library before it'll work. Isolation from each other seemed like one advantage, but that's not even security strength isolation so you can't count on it to protect the host OS from malware in a container. A claim I often see, and that's repeated here, is that containers can run anywhere. But can they really run in any more places than an ordinary application with dependencies included, or even statically compiled into it? You still need Linux and the same CPU architecture, right?
- axlee 10y ago
  > Nobody distributes a Windows application that requires the user to go and install some 3rd party library before it'll work. A-hem, DirectX, VC++...?
  - Hondor 10y ago
    Those things are usually included in the installer, or at least downloaded on the fly so you don't notice. So it's still self-contained from the user's point of view. Yes, sometimes they're installed globally like DirectX, so that can cause yucky interactions between applications.
    - deleted 10y ago
      [deleted]
- raesene6 10y ago
  Containers (and especially multi container orchestration software like docker compose or Kubernetes Pods) let you describe an application that might contain multiple processes (so a really simple example could be a web server with a database backend) in a single file and have that deployed to any system that runs the containerization software. So to that extent its more flexible than a single app. which bundles its dependencies. The other advantage is that you're not reliant on the software vendor or OSS project to create the package, so you as a user of the application, can create your own packages with your own customization. As to where you can run them, yep at the moment the image is tied to an OS and architecture, so either Linux or Windows depending on the Docker engine version, although there are moves afoot for Multi-arch support on the registries (https://github.com/docker/docker/issues/15866 https://github.com/docker/docker/issues/15866)
BjoernKW 10y ago
Containers are often touted is this novel concept that's bound to revolutionise software development and software delivery in particular. The general idea isn't all that new however. Java Applications have been delivered as containers since 1995 (although the concept isn't explicitly named that way with Java applications). Each JAR / WAR is a self-contained application that can run anywhere where there's a JVM (which is pretty much everywhere). From a feature perspective the only real innovation of Docker-style containers probably is that those aren't limited to the JVM but are (largely) language- and runtime-agnostic.
- mirekrusin 10y ago
  JAR/WAR general idea is not that new as well, we had CPAN before!
  - BjoernKW 10y ago
    CPAN is more akin to Maven (or rather public Maven repositories) / RubyGems / NPM in that context. Perl modules are hardly self-contained. Compiling Perl module dependencies could be a real pain at times.
- paulddraper 10y ago
  Actually, JARs are even more compatible than that: they can run against a huge number of OSs and architectures, even in embedded systems.
- geodel 10y ago
  > Each JAR / WAR is a self-contained application that can run anywhere where there's a JVM (which is pretty much everywhere). Jar could be self contained or it could depend on 100s of other jars so it is not as straightforward as one assume. War required a Java application containers to be deployed which is outside a War not inside. Also J2EE deployments can get pretty complicated with its infamous XML usage for configuring anything. JVM is not everywhere by default or work without config by default for non trivial application. It is complex software which can lead to myriad classpath/ version issues related to jar files if not installed very carefully.
llcoolv 10y ago
lol. read two paragraphs => click author's name to verify he's got no actual tech background. life is boring.
- minimaxir 10y ago
  The author has been writing for TechCrunch for a considerable amount of time.
  - llcoolv 10y ago
    I have been singing in the shower for a considerable amount of time. Still no singing expertise. In both cases it is immediatly audible/visible in the output.
AJRF 10y ago
On VM's: "Underneath it all is the host operating system that makes all of these guests believe they are the most important thing in the world. You can see why this is a problem." Two paragraphs later (On containers): "The only operating system on the server is that one host operating system and the containers talk directly to it."
Azeralthefallen 10y ago
I remember going to AWS Reinvent last year and having some beers with a bunch of people who did devops. We started talking about tools, and they were utterly flabbergasted, that we had not embraced docker. They went on an on about how simple docker made HA, and handling fail overs/maintenance. More or less made it seem like it was the greatest thing since sliced bread. Me and a few coworkers decided to try and dockerize some of our serivces, and move our staging ES cluster to docker. For the most part building our own containers was easy enough, for the various services. The biggest issue we had was with Elasticsearch, since we have 4 types of nodes. So we ended up building 4 specialized containers for each node type. Then came the issues: * Making containers work together, for example we use a logging agent, we decided to make that its own container. Then actually getting a way to share a log directory with the agent, was very painful and unclear. (Honestly the single most frustrating thing i recall was asking for advice in irc and more or less being told i am doing it wrong) * Containers would randomly crash under load exiting with error 137 out of memory. Apparently a few of our services would randomly leak memory, but only when running inside a docker container vs ubuntu 14.04. (I never figured this out) * Containers would randomly become unreachable, forcing me to kill them and restart them. * Random hanging issues, suddenly docker exec is hanging. Being told to revert to an older version, or install a newer version is tiring and makes my environment very inconsistent. * Trying to debug why our app in the container isn't working is not fun at all. However the single part that killed me was, i was chatting with one of the people who i met at Reinvent, and i mentioned my issues. He acted like it was completely normal for these kinds of issues. After a solid 2 weeks of random issues and the constant barrage of pagerduty alerts, i just rolled everything back to static EC2 instances for staging, and have ran into 0 issues. I want to try containers again because i want them to work, but i have just had too many issues.
- godzillabrennus 10y ago
  Reminds me of the early days of asterisk and open-vz virtualization. Everyone sane stayed physical but a few of us crazy enough were able to push through and reap the rewards early on.
- cookiecaper 10y ago
  Yes, Docker is still very young and has a ton of issues like these. I think it will take another 2-3 years before that whole ecosystem has emerged, matured, and is ready for a medium-stability deployment. Unfortunately for us, such considerations don't stop tech fads. Because containers can allow many more applications to cohabit on the same "hardware", it has business momentum behind it too (lower infrastructure expenditures). "Docker" will be the buzzword until such time as it's actually practical and intelligent to deploy with it.
zk00006 10y ago
'WTF' really?
ilaksh 10y ago
The reason people don't get the advantage of Docker is because there is a weird (and in my opinion stupid) tabboo against putting all of your deps in one container. This is about 10-100X easier than trying to compose a bunch of containers. Not everyone can do that, but plenty of people could. Except they don't not because of some actual ops requirement (in many cases) but because they don't want someone to say they did it wrong. I am assuming this situation has actually changed now I hope and swarm/compose or whatever is built in is not too hard to use?
- ohdrat 10y ago
  no it's worse now. There's also all the PKI required compounding the pain and suffering. Phusion baseimage-docker seems to help. Worth nothing Atlassian stepped back and does scaleout approach using regular cloud instead of container orchestration technology. https://www.atlassian.com/company/events/summit/2016/watch-sessions/build-deploy/scaling-your-first-1000-containers-with-docker https://www.atlassian.com/company/events/summit/2016/watch-s... Also worth noting you can take away all the PKI and registry pain and have instant scaling with the registry by using cloud store for registry and having Jenkins gatekeeper be the only writer, all dev and production nodes use the same store only readonly.
gtrubetskoy 10y ago
It's amazing how slow the development community en masse has been in "discovering" containers. I remember I worked for web hosting company that offered containers as a web hosting solution back in 1999, I then ran my own little host offering FreeBSD jails and then Linux containers (based on the excellent Linux-VServer project) in 2003, and I do remember how when I tried to explain to (pretty technical) people how this is way more efficient than stuff like Xen they'd go "but it's a hypervisor..." (as if that meant "magic"). I eventually gave up on it and sold my little hosting operation because it was too much work and not enough money, it looks like it was about 10 years ahead of it's time.
- sdsk8 10y ago
  I think the same, i've been deploying containers or things that look like today's containers for the past 12 years, i even did a tool that manages containers with lxc that really look like docker, but 3 years before.
  - cesnja 10y ago
    There were many solutions very similar to Docker, but you have to agree that they made some essential things the right way - UnionFS (e.g. OverlayFS), standardized way of building containers (Dockerfiles) and the right timing.
VLM 10y ago
The analogy in the article is inferior. Here comes the standard HN automobile analogy. The car makers eternally release little light weight cars that are fattened up with cruft until they're as heavy and expensive and complicated as the dinosaurs they were originally meant to replace. At which time car makers birth a new little stripped down lightweight simple compact car line. That design pattern is self similar and fractal in that even car engines and transmissions and car radios undergo a similar pattern of revolution to make something new, then evolution to slowly and methodically make it the opposite of the original goal, repeat forever. This design pattern also applies to computer architecture. To fight the problem of hardware evolution making life pretty hard on programmers, source code compatible OS and libraries were invented that could run the same software on vaguely similar hardware, on mainframes in the late 60s at IBM and on PCs in the 70s CPM era, later extended into the "home computer" series era in the 80s, then into the msdos era. This became unwieldy and too complicated for the end users so it was replaced. The same people implemented the idea of OS packages, again more or less in the 60s on IBM mainframes or the 80s on early unix boxes. The idea is to compile emacs to be integrated very deeply into the OS and cooperate with every other piece of software. This is contemporary. However especially in NON-FOSS it doesn't work and doesn't scale and is very expensive if not impossible to implement, being the only closed thing on an open system is a nightmare for everyone and everything involved. The stereotype in the 90s was only one service installed on one MS windows server, even if that meant it took 20 MS servers to do the job of one unix server. Anyway, expensive, complicated, hard for end users. Again the same people implemented the idea of OS virtualization. Again, IBM mainframes in the 70s with VM, and early PC hardware experiments with TSRs and multiprocessing in the 80s to give "two computers at once". This is also contemporary, enormously more advanced today, of course. It turns out that running 50 OS kernels on a single piece of hardware is kinda wasteful although possible and cooperation gets complicated and unwieldy so time to replace again. Again the same people implemented process jails / chroot on the BSDs and eventually after some decades linux finally caught up resulting in docker. So now your packages don't cooperate or interoperate at all with each other or the OS, which solves all the problems where the previous technology didn't work, and creates massive new problems that never existed mostly where the old technologies worked great. There are of course completely separate new problems. It turns out that a system designed to eliminate interoperability between packages interoperability a huge PITA and there are other problems that make use unwieldy and complicated, hard for end users. Again the same people implemented (this section to be written after 2016). Maybe IOT. Maybe collapse of hardware prices faster than business demand means 20 rasp-pi cluster is cheaper and easier to maintain than a single beefy desktop running 20 virtual images or 20 docker containers. Maybe FPGA on the desktop means people will just synth up whatever matches this hours workload. Maybe cloud will finally work and no one will maintain servers anymore, it'll all be magic, or at least push the magic to someone else who now has the same old problems. Maybe SaaS means we'll all be customers and most productive software will run on internet scale clusters not individual machines, individual machines will be dumb webbrowser terminals. Who knows!
bsenftner 10y ago
Plain and simple: containers violate the KISS principal. While you are developing reams of domain knowledge working with this turd of a technology, others are making progress with non-self-created problems that matter in the real world. This is another case of the tech world developing pointless tech that can be ignored by those solving concrete real world problems.
novaleaf 10y ago
> The way virtual machines work, however, is by packaging the operating system and code together. The operating systems on the virtual machines believes that it has a server to its own, but in reality, it’s sharing the server with a bunch of other virtual machines — all of which run their own operating systems and don’t know of each other. Underneath it all is the host operating system that makes all of these guests believe they are the most important thing in the world. You can see why this is a problem. The guest virtual machines basically run on emulated servers, and that creates a lot of overhead and slows things down (but in return, you could run lots of different operating systems on the same server, too). Wait a second. Isn't this the whole point of hardware virtualization support? So that hypervisors don't have all those VM overhead slowing things down?
realworldview 10y ago
IBM and Sun know.
mmatoscom 10y ago
Hi there, good day fellows I was expecting this kind of thread long ago, thanks guys for sharing your concerns, I am learning a lot from them! IMO we can't compare containers vs VMs like many are doing now - and I was too when first heard about Docker and containers. I hold almost all VMware certs (VCP/VSP/VTSP for 3.1/4.x/5.x and VCDX), I sign the 3 major VMware P2V migrations in BR/LA (287 P2Vs in 2005, 1600/2008 and 2500/2011). I was REALLY into VMware from 2000 to 2010, so I feel confident using it and recommending to many environments. I even manage some of them still today. When we clone or migrate a physical machine to virtual, or whenever deploying a VM from scratch (or even using templates or copying VMdks etc) into production, we aim to develop the environment in order to see it lasting "forever". We want this to be flawless, because even with given most-players virtualization deploying resources (hyper-V, VMware, xen, kVM, vbox+vagrant, etc), nobody wants to troubleshoot a production environment, we cheer to see the VMs always up and running. I remember when P2V in mentioned projects during the night, and needed to fallback physical servers because the cloned VM didn't behaved accordingly. Please VMware, clone it ok, otherwise the troubleshoot for legacy shit will be a pain. On the other side, containers are made to be replaced. They are impermanent. You can tune your images as your app/env needs. You can have an image for each service you have. You can have many images running many containers, and some of them providing services you have. You are able to customize an image in a txt file called Dockerfile, and generate an image from this file. So imagine we got this infrastructure to "Dockerize" , a website with a DB. Does your webserver runs apache? so you can code a Dockerfile, that will deploy an apache instance to you. It could deploy FROM an ubuntu:16.10 or 10.04, depends on what is better for your app. OR, we could just pickup Apache's own image, like in FROM apache. You can save this image as yourid/apache. And you can do the same regardless of what DB you are using, just install it (the mysql using apt-get in a FROM ubuntu/debian based system), or use mysql images directly. You are able to publish the website cloning your site dev repo direct in Dockerfile itself, or you could have the website at some dir in your host, using ADD or CP to make it available in the right container dir (eg /var/www/) You could even use Volumes, to mount some host dir (or a blob, or a storage account, or a partition in your storage, something available in the host itself). This is specially interesting for DB in my opinion. Once you have your Dockerfile ok, you can name it yourid/db. And if you have a main website, and a blog website, you could use the same apache Dockerfile changing only the git clone line, and save them as yourid/apache:website and yourid/apache:blog for example. And when redeploying the container, you will have the same volume data available in the same container dir. Even if you redeployed it from ubuntu:15.10 to ubuntu 16:10.You can use the latest improvements from the freshest image (or patching some vuln), and redeploy all your containers that uses this same image at once. The same goes on, you can test jenkins without knowing what OS jenkins image is made off. You dont have to worry about it. It will just work. You pull the image and run the container and voila. NOW, my Docker instances are like this: I use Docker-machine, and locally I got the default and local envs. I got also an Azure instance at Docker-machine (that runs on Azure), and another instance configured in Docker-cloud using Azure as cloud provider (I use Azure due bizspark credits). So, 4 of them. All those instances, are VMs themselves. Ubuntu VMs to be sure. You just replace the container (probably published in a cluster if you care enough your prod env). Not the same as with VMs at all. I see Docker as a hypervisor for containers the same way VMware and hyperv are to VMs. So I understand my Docker hosts VMs have the same HA, failover, load balance, resources allocation, and so many resources VMs have. I use Docker on those VMs to make easy deploys, images, tune images, really guys I was the VMware guy for so long, I went just crazy in the resources Docker gives to us. Docker has many weak points indeed (NAT networking, privileged containers must be run sometimes, security concerns, etc), but again, I don't see Docker to be erasing VMs from my life and now on I can deploy everything that will run happy forever in containers. We still need HA, failover, load balance, resources allocation and so on. Docker needs to be used together with TOOLS that allows it to run smoothly, and allows us to maintain our environments easier. One of those tools are containers clusters. I work mostly with Google Kubernetes, but there are other as Docker Swarm, Apache Mesos, DCOS... Azure has its Azure Container Services ACS, IBM has its BlueMix containers, etc. Using a cluster and a deploy methodology, you are able to deploy your containers in different namespaces such as DEV / STAGING / PROD. You can use a BUILD environment to read your Dockerfile, build the image and deploy containers to the namespace you need. You can configure this build to trigger with a commit in the git repo for exemple. So lets say we have a developer, and he needs our yourid/apache:website to be deployed, with the new website version. If the website is already updated in your git repo, you just clone it. The Dockerfile would look like this: FROM apache MAINTAINER Your Name <your@email.com> WORKDIR /var/www/ RUN git clone https://github.com/yourid/website/website.git . EXPOSE 80 CMD ["/run.sh"] This would be named as website.Dockerfile. If you change the project git repo to any of your other sites that runs on apache, you can SAVE AS other.site.Dockerfile, and always deploy this service from this specific repo. You can customize your Dockerfile of course and add support to specific codes like installing PHP, Ruby, Python, etc. You could even use Configuration Managers (CMs) as Ansible, Salt, Chef, Puppet, juju etc to apply those changes. Lets say we will start the build now. We are developing this image together. So I just changed my git url on the Dockerfile. When we commit, the autobuild triggers this build in our build system (in my case, Docker-cloud or jenkins). This is what Continuous Integration (CI) and Continuous Deployment (CD) are about. So when build is triggered, it gets the Dockerfile from the repo, builds from its image, deploys the container in the namespace you wanted (our case, DEV). This service could be published as website.dev.mydomain.com for example. Same concerning to staging namespace. And to www.mydomain.com when ready to production, in the PROD kubernetes namespace for example. Kubernetes is a distributed thing, so you could have minions (nodes) splitted across different datacenters, or geo locations. This pretty much reminds me of VMware VMs running inside VMfs storage made available through a set of ESXi servers, all with access to the same luns/partitions/volumes. This is just my point of view, so please feel free to comment and ask me anything. Please, just dont blame Docker because you aren't aware of mainstream techs available nowadays. If you are comparing Docker to VMs, or SSHing inside the containers, and often mad cause your data vanished while redeploying your Docker Containers, believe me you are doing it wrong. Being a pioneer is often the same: in the 90s we had to explain why Linux was good for the enterprise, in the 2000's we had to prove VMware was really going to cluster your legacy systems, and now we have to explain what's possible to do with Docker. And, as the tech is new (I know there were previously solaris zones, google borg, etc), but I see Docker will mature its features relying in other tools (and even copying features from k8s to Swarm eg). Docker is just one skill needed to run your stuff. Cheers! M Matos https://keybase.io/mmatos https://keybase.io/mmatos
alauda 10y ago
Guys, you really want to give a look to hyper.sh, if you are frustrated with running containers in production.

366 points by kawera 10y ago

This is not quite related to Docker/Kubernetes, but a phrase in the article triggered memories about a very good book that I think everyone should read - The Box by Marc Levinson. It really is a fascinating book.

haalcion3 10y ago
Definitely "not quite", as it is about shipping containers, but an interesting read.

bbcbasic 10y ago

I've read before on HN about 'don't use docker in production' can anyone elaborate on that for a newbie?

knowaveragejoe 10y ago
My understanding is that plenty of people do actually use Docker in production, but they use more than just Docker. A docker container isn't quite the same as a VM.
hvidgaard 10y ago
There is a lot of infrastructure around security and availability that isn't yet achiveable with off the shelf software. So ideally you use containers to define dependencies and deployment, and deploy into a proper environment.

simula67 10y ago

Putting a little bit more thought into naming things can help newcomers understand these concepts more easily. The word container is too generic, why not call it an 'application jail' or a 'virtual operating system' ? Another word that is over used in our field is 'context'.

lgas 10y ago
It's based on the idea of container ships where the contents of every container can vary widely but the external interface for dealing with the containers is always identical. It's a pretty apt metaphor and not too generic, IMHO.
stass 10y ago
They were called "jails" long before "containers" became a thing on HN. Somehow "jails" didn't stick, so I'd assume it was even more confusing for newcomers.
- santaclaus 10y ago
  > Somehow "jails" didn't stick I hear chroot jail tossed around a fair amount.
- chii 10y ago
  Jail had no marketing hook. It's a very negative word. I'd guess that's Probably the major reason.
  - hueving 10y ago
    Well that and it wasn't done by a VC backed firm with strong press connections.
- jsjohnst 10y ago
  Jails is typically the FreeBSD name for a very similar concept. Linux "copied" (using the term very loosely) the idea, but not the name.
- jbverschoor 10y ago
  I was running a couple of jails on my FreeBSD server 15 years ago.. Worked like a charm.. But eventually I switched to linux.

dancek 10y ago

I agree that containers (both for shipping and servers) are a great idea. And because I'm tired of always configuring servers, I decided to give it a try some time ago. I wrapped my IRC client (weechat + glowing-bear) in a Docker container. Oh, not a container though, because I also needed https, which meant I needed either a mechanism to build and update letsencrypt certs in the weird format that weechat expects, or to run an nginx instance in the front (and also somehow get the certs, but that's easier with nginx). So two containers. And even though there was a ready-made nginx+letsencrypt https reverse proxy container (actually several), I had a huge amount of headaches to get it actually working. Even with the system set up, I occasionally have the container crash with exit status 137 (IIRC), which I've assumed might be because weechat leaks/consumes memory and eventually the host server kernel kills the process. Maybe. So in my limited experience, comparing Docker containers to shipping containers is a gross simplification. Shipping containers are simple constructions requiring well-defined simple maintenance, while Docker containers seem to be complex thingamabobs that have multiple points of failure.

hvidgaard 10y ago
I wanted to write about my experience with containers. It mimics yours. The idea of a container is beautiful and elegant. The execution, not so much. I'm more or less convinced that to reduce this complexity, we need a limited scope ecosystem. Microsofts .Net would probably be a good place to start, but MS, being enterprise, have a habit of turning things into a mess of configuration.
chewchew 10y ago
Docker gives you the building blocks, but that means you have more pieces to arrange and manage. Take a look at Docker Compose if you haven't already, since the Docker CLI only gets you so far when you're creating apps that consist of multiple containers. I think the best approach for your cert issue is to abstract that into a separate service (nginx is an option, but I'd recommend the Rancher approach below). Yes, that means you have to add another container, but that's just another block in your docker-compose.yml file. Embrace the approach of separating your components into containers and organizing them as a stack. You can easily link containers together, share data volumes, and start/stop individual containers or the stack as a whole. The problems that you're having are pretty easy to fix with some tooling. Rancher (http://rancher.com/ http://rancher.com/) greatly simplifies the cert issue by allowing you to import certs and provide them to the Rancher loadbalancer service (which you can add to any stack). There's also a LetsEncrypt community catalog template that automatically retrieves and imports certificates to Rancher. There are other open source orchestrators like DCOS, but Rancher is probably the simplest to use, and it's the only one I'm very familiar with. There are SaaS options that you can look into, but I don't have experience with them. As for the container crashes, it's trivial to automatically restart them. Just pass the --restart=always flag to the Docker run command. You can also add the flag to a docker-compose.yml file.

dkarapetyan 10y ago

> The promise behind software containers is essentially the same. Instead of shipping around a full operating system and your software (and maybe the software that your software depends on), you simply pack your code and its dependencies into a container that can then run anywhere — and because they are usually pretty small, you can pack lots of containers onto a single computer. Already got it wrong. Current containers are exactly the OS and the kitchen sink for running 'printf("hello world")'.

chewchew 10y ago
It's up to you to make your containers bloated or keep them slim. You can use the alpine versions of the official Dockerhub images. Python on Alpine is 30 MB (vs 267 MB for the debian one). https://hub.docker.com/r/library/python/tags/ https://hub.docker.com/r/library/python/tags/ You can create containers that are just a few MB with compiled languages like Go (5 MB). https://www.iron.io/microcontainers-tiny-portable-containers/ https://www.iron.io/microcontainers-tiny-portable-containers... From the article: "Rather than starting with everything but the kitchen sink, start with the bare minimum and add dependencies on an as needed basis."
- stephenr 10y ago
  If your app is a compiled go binary (so, it runs anywhere) why do you need a container? The whole point of containerisation is to group installed dependencies (as opposed to installable dependencies like with a regular deb or rpm package) and configuration into a 'black box'. If your binary is already a single-file distribution, why lump it in with the crapfest that is docker?
  - chewchew 10y ago
    I don't think that's the whole point. If all you care about is packaging your code in a container, then you don't need Docker. That was solved long ago. Docker simply adds a nice API on top which allows you to package, ship, and manage your apps in the same exact way. By using something like Docker Compose, you can explicitly define each container and the relationships between the containers. Then, you can start/stop groups of containers (an application stack) while still retaining some component isolation and the ability to upgrade and scale containers independently. All of this is defined in a relatively simple YAML file, which can be committed to VCS and tweaked. I can't tell you how awesome it is to find repos on Github that have use docker-compose. Even complex apps with many pieces can often be launched with a single command. It's easy to take these stacks and tweak them to your needs, adding/removing/changing components as necessary. Since Docker provides a standard way for managing any container, orchestrators like Swarm, Rancher (my preference for small-medium clusters), and DC/OS can take that functionality and scale it across many hosts. You can get a birds-eye view of your Docker cluster (all apps) or drill down into individual apps and their components. Each container is a uniform piece and can be controlled, scaled, updated, audited in the same way. Throw a UI in front of it and now you can manage applications that you know nothing about. That's great for developers that manage just a few apps, and it's incredible for enterprises with thousands of them. If you don't want to think about infrastructure at all, then you can use a SaaS Docker provider. Obviously there are pros and cons to each approach, and there are some remaining challenges. Docker isn't perfect, especially in regards to storage and networking. Distributed storage isn't simple, but a lot of progress has been made with volumes and volume drivers. It's not as easy as it needs to be, but the general direction seems to be good and with the proper tooling I think this will be less of an issue.

fenomas 10y ago

I'm only a beginner, but the analogy that makes sense to me is that containers do for app deployment what npm does for Javascript development. That is, the magical part isn't that Docker simulates an operating system and so on - the magic is that it allows a chunk of logic to precisely declare its dependencies - including on other pieces of logic which declare their own dependencies - and then Docker knows how to (in theory anyway) run the logic in such a way that its dependencies are all satisfied. And of course the meta-magic is then that there's a public registry of (in theory) solved problems, which one can build on top of by declaring dependencies against them. I have a pet theory that this "declared dependencies + dependency wrangler + public registry" is a general formula which will keep cropping up as we find new places to apply it.

kpil 10y ago
Not really - Linux package management systems do exactly that. The magic - if there is any - is to combine it all together; separation, discovery, relatively easy packaging and dependencies.
- fenomas 10y ago
  Sure, at a different level. Package managers at the app level, docker at the deployment level, npm at the development level, or something vaguely along those lines. I wasn't suggesting this was unique to Docker - precisely the opposite, that it's a generally useful pattern, being applied here to deployment.
bpizzi 10y ago
To me it's more like a lightweight virtualization: processes inside containers have no clues that there are others processes running in others containers alongside its own, all of that without eating too much memory (at least well under the amount that virtual machines would use). Added benefits: - the 'host' os can be very light and tailored to run Docker and nothing more (cf CoreOS), - we can design orchestration software that handle containers operations across a herd of lightweight hosts (cf Kubernetes). The dependencies/recipes thing is more like a tool that enable those higher goals, but again that's just my point of view.

disposablezero 10y ago

Weak virtualization often lacking security, resource metering/prioritizing/quotas. Hurray, zombie Docker instances needs a whole VM reboot yet again, still not fixed in several years. But look how fast I can deploy millions of containers without SELinux, monitoring, HIDS, SDN, billing, live migration, backup/restore/DR/data lifecycling and all the other things we just pretend to ignore when throwing away sensible production VMs on Type 1 hypervisors devopsec.

hvidgaard 10y ago
The smart use of containers is using it to specify the deployment. You should use proper virtualization as the environment to deploy into. Because you use containers doesn't mean you throw all that away.

sytelus 10y ago

These hand-wavy explanations that constantly avoids explaining how things work at low level are not adequate. Here's short explanation for developers even with moderate understanding of how OSes work: http://stackoverflow.com/questions/16047306/how-is-docker-different-from-a-normal-virtual-machine/34757096#34757096 http://stackoverflow.com/questions/16047306/how-is-docker-di...

deleted 10y ago

[deleted]

parito 10y ago

for the love of god - forget docker, use lxc containers - its simple, secure, goes with its own init, cron, and you dont need to do somersaults to achieve simple tasks. Included with linux kernel. Your own isolated linux system. We use lxc in production for over three years, and we have over 3000 of them. No issues whatsoever.

austinjp 10y ago
Encouraging to hear. Who do you work for, who has these 3,000 LXC containers in production use? And I'm curious, what orchestration system do you use to manage them? Can you outline your toolset?
- parito 10y ago
  We use ansible and bash scripts for orchestration.
  - austinjp 10y ago
    If you'd care to go into more detail, or could point me to your technical docs, I'd be very interested. How, for example, do you handle roll-out, destruction, monitoring, backups, network configs, secrets, etc etc... is there any degree of automation? And why not take advantage of existing orchestration solutions? Was nothing mature enough for your needs? How big is the team managing these 3,000 containers, what sort of traffic are you handling? Asking out of genuine curiosity. I'm keen on understanding how plain LXC can be used robustly in a production environment.
choxi 10y ago
You're right that LXC containers have a similar API compared to Docker, but I think developers often underestimate the benefit of the community around a certain technology. Docker has significantly better documentation, extensions, package management tools, and third-party integrations. Overall, Docker has an incredibly more robust community than LXC or closer competitors like Kubernetes, and those features are just as important as the API for developers.

sztwiorok 10y ago

Docker is the best thing since sliced bread! VM is much more resource and time consuming. # We switched our dev/stage env to containers 2 year ago. # We have made our own standalone app in Docker style. Once again - Easy as pie. if you are a developer you should add Docker + Docker Compose to you working tools.

cZuLi 10y ago
We moved over to Docker at the development process some time ago and it really speed up our work.

jdoliner 10y ago

This is a pretty good analogy for containers but there's an unfortunate conflation of terms. There's actually a distinction with computing containers between the thing that you ship code around as and the thing that you run code in. The latter is the real container while the prior is called a "container image" or often simply an "image." This gets confusing quick if you apply this analogy since you assume that the thing you ship the code as would naturally be called the container.

lottin 10y ago

I guess I'll never get it. Don't most OSs already run processes isolated from each other, have advanced process scheduling mechanisms and manage access to hardware resources? Also with static linking nothing stops you from creating huge binaries that "will run anywhere".

justincormack 10y ago
Not with a private root filesystem, a private loopback network, additional security isolation, no requirement for static linking, no.
greggman 10y ago
In containers a few more things are virtualized. The file system is semi-virtualized. Network ports are too. So from the pov of stuff inside the container nothing else is running. That's not true of processes in general. From outside the containers you can then choose how to map parts of the virtual file systems to parts of the real file system and what real network ports the virtual ports connect to etc... There's more to it. Containers aren't one process theyre as many processes as are launched in that container
- sundarurfriend 10y ago
  So let me try to understand this from a different angle: what's something that a VM can and does do, that container software like Docker can't? TFA makes it sound like legacy systems is the only place for VMs anymore, but I'm guessing that's probably approximation+exaggeration.
  - austinjp 10y ago
    In my (limited) understanding: host OS and guest OSes can be completely unrelated. Whereas in containers, the host and guests share the kernel.
austinjp 10y ago
From my perspective, where I'm planning a hosting service with multiple customers, containers promise to allow me to slice machines into dedicated chunks. VMs could do this too, but they'd consume far more disk-space, which would mean my hosting costs would be comparatively higher. Containers promise to allow me to limit the resources each of my customers can consume without killing the entire box. For example, I can limit RAM, CPU and disk-space per customer. If one customer goes rogue, the shared box remains performant for my other customers. Also there's the data protection: in theory customers would not be able to access each others' data, even if they tried to. There are other considerations as others here have pointed out. This is just my primary concern at the moment.

ChoHag 10y ago

Take an operating system. Remove all the advantages of a shared environment like dynamic libraries, package management, clarity. Stick a chroot before every fork. Boom! Containers.

Hondor 10y ago

I never quite understood containers and this article makes them seem kind of similar to what OSs already do. How is a container different from just installing all the dependencies along with an application? Coming from a Windows background, this is pretty common to avoid DLL hell. Nobody distributes a Windows application that requires the user to go and install some 3rd party library before it'll work. Isolation from each other seemed like one advantage, but that's not even security strength isolation so you can't count on it to protect the host OS from malware in a container. A claim I often see, and that's repeated here, is that containers can run anywhere. But can they really run in any more places than an ordinary application with dependencies included, or even statically compiled into it? You still need Linux and the same CPU architecture, right?

axlee 10y ago
> Nobody distributes a Windows application that requires the user to go and install some 3rd party library before it'll work. A-hem, DirectX, VC++...?
- Hondor 10y ago
  Those things are usually included in the installer, or at least downloaded on the fly so you don't notice. So it's still self-contained from the user's point of view. Yes, sometimes they're installed globally like DirectX, so that can cause yucky interactions between applications.
  - deleted 10y ago
    [deleted]
raesene6 10y ago
Containers (and especially multi container orchestration software like docker compose or Kubernetes Pods) let you describe an application that might contain multiple processes (so a really simple example could be a web server with a database backend) in a single file and have that deployed to any system that runs the containerization software. So to that extent its more flexible than a single app. which bundles its dependencies. The other advantage is that you're not reliant on the software vendor or OSS project to create the package, so you as a user of the application, can create your own packages with your own customization. As to where you can run them, yep at the moment the image is tied to an OS and architecture, so either Linux or Windows depending on the Docker engine version, although there are moves afoot for Multi-arch support on the registries (https://github.com/docker/docker/issues/15866 https://github.com/docker/docker/issues/15866)

BjoernKW 10y ago

Containers are often touted is this novel concept that's bound to revolutionise software development and software delivery in particular. The general idea isn't all that new however. Java Applications have been delivered as containers since 1995 (although the concept isn't explicitly named that way with Java applications). Each JAR / WAR is a self-contained application that can run anywhere where there's a JVM (which is pretty much everywhere). From a feature perspective the only real innovation of Docker-style containers probably is that those aren't limited to the JVM but are (largely) language- and runtime-agnostic.

mirekrusin 10y ago
JAR/WAR general idea is not that new as well, we had CPAN before!
- BjoernKW 10y ago
  CPAN is more akin to Maven (or rather public Maven repositories) / RubyGems / NPM in that context. Perl modules are hardly self-contained. Compiling Perl module dependencies could be a real pain at times.
paulddraper 10y ago
Actually, JARs are even more compatible than that: they can run against a huge number of OSs and architectures, even in embedded systems.
geodel 10y ago
> Each JAR / WAR is a self-contained application that can run anywhere where there's a JVM (which is pretty much everywhere). Jar could be self contained or it could depend on 100s of other jars so it is not as straightforward as one assume. War required a Java application containers to be deployed which is outside a War not inside. Also J2EE deployments can get pretty complicated with its infamous XML usage for configuring anything. JVM is not everywhere by default or work without config by default for non trivial application. It is complex software which can lead to myriad classpath/ version issues related to jar files if not installed very carefully.

llcoolv 10y ago

lol. read two paragraphs => click author's name to verify he's got no actual tech background. life is boring.

minimaxir 10y ago
The author has been writing for TechCrunch for a considerable amount of time.
- llcoolv 10y ago
  I have been singing in the shower for a considerable amount of time. Still no singing expertise. In both cases it is immediatly audible/visible in the output.

AJRF 10y ago

On VM's: "Underneath it all is the host operating system that makes all of these guests believe they are the most important thing in the world. You can see why this is a problem." Two paragraphs later (On containers): "The only operating system on the server is that one host operating system and the containers talk directly to it."

Azeralthefallen 10y ago

I remember going to AWS Reinvent last year and having some beers with a bunch of people who did devops. We started talking about tools, and they were utterly flabbergasted, that we had not embraced docker. They went on an on about how simple docker made HA, and handling fail overs/maintenance. More or less made it seem like it was the greatest thing since sliced bread. Me and a few coworkers decided to try and dockerize some of our serivces, and move our staging ES cluster to docker. For the most part building our own containers was easy enough, for the various services. The biggest issue we had was with Elasticsearch, since we have 4 types of nodes. So we ended up building 4 specialized containers for each node type. Then came the issues: * Making containers work together, for example we use a logging agent, we decided to make that its own container. Then actually getting a way to share a log directory with the agent, was very painful and unclear. (Honestly the single most frustrating thing i recall was asking for advice in irc and more or less being told i am doing it wrong) * Containers would randomly crash under load exiting with error 137 out of memory. Apparently a few of our services would randomly leak memory, but only when running inside a docker container vs ubuntu 14.04. (I never figured this out) * Containers would randomly become unreachable, forcing me to kill them and restart them. * Random hanging issues, suddenly docker exec is hanging. Being told to revert to an older version, or install a newer version is tiring and makes my environment very inconsistent. * Trying to debug why our app in the container isn't working is not fun at all. However the single part that killed me was, i was chatting with one of the people who i met at Reinvent, and i mentioned my issues. He acted like it was completely normal for these kinds of issues. After a solid 2 weeks of random issues and the constant barrage of pagerduty alerts, i just rolled everything back to static EC2 instances for staging, and have ran into 0 issues. I want to try containers again because i want them to work, but i have just had too many issues.

godzillabrennus 10y ago
Reminds me of the early days of asterisk and open-vz virtualization. Everyone sane stayed physical but a few of us crazy enough were able to push through and reap the rewards early on.
cookiecaper 10y ago
Yes, Docker is still very young and has a ton of issues like these. I think it will take another 2-3 years before that whole ecosystem has emerged, matured, and is ready for a medium-stability deployment. Unfortunately for us, such considerations don't stop tech fads. Because containers can allow many more applications to cohabit on the same "hardware", it has business momentum behind it too (lower infrastructure expenditures). "Docker" will be the buzzword until such time as it's actually practical and intelligent to deploy with it.

zk00006 10y ago

'WTF' really?

ilaksh 10y ago

The reason people don't get the advantage of Docker is because there is a weird (and in my opinion stupid) tabboo against putting all of your deps in one container. This is about 10-100X easier than trying to compose a bunch of containers. Not everyone can do that, but plenty of people could. Except they don't not because of some actual ops requirement (in many cases) but because they don't want someone to say they did it wrong. I am assuming this situation has actually changed now I hope and swarm/compose or whatever is built in is not too hard to use?

ohdrat 10y ago
no it's worse now. There's also all the PKI required compounding the pain and suffering. Phusion baseimage-docker seems to help. Worth nothing Atlassian stepped back and does scaleout approach using regular cloud instead of container orchestration technology. https://www.atlassian.com/company/events/summit/2016/watch-sessions/build-deploy/scaling-your-first-1000-containers-with-docker https://www.atlassian.com/company/events/summit/2016/watch-s... Also worth noting you can take away all the PKI and registry pain and have instant scaling with the registry by using cloud store for registry and having Jenkins gatekeeper be the only writer, all dev and production nodes use the same store only readonly.

gtrubetskoy 10y ago

It's amazing how slow the development community en masse has been in "discovering" containers. I remember I worked for web hosting company that offered containers as a web hosting solution back in 1999, I then ran my own little host offering FreeBSD jails and then Linux containers (based on the excellent Linux-VServer project) in 2003, and I do remember how when I tried to explain to (pretty technical) people how this is way more efficient than stuff like Xen they'd go "but it's a hypervisor..." (as if that meant "magic"). I eventually gave up on it and sold my little hosting operation because it was too much work and not enough money, it looks like it was about 10 years ahead of it's time.

sdsk8 10y ago
I think the same, i've been deploying containers or things that look like today's containers for the past 12 years, i even did a tool that manages containers with lxc that really look like docker, but 3 years before.
- cesnja 10y ago
  There were many solutions very similar to Docker, but you have to agree that they made some essential things the right way - UnionFS (e.g. OverlayFS), standardized way of building containers (Dockerfiles) and the right timing.

VLM 10y ago

The analogy in the article is inferior. Here comes the standard HN automobile analogy. The car makers eternally release little light weight cars that are fattened up with cruft until they're as heavy and expensive and complicated as the dinosaurs they were originally meant to replace. At which time car makers birth a new little stripped down lightweight simple compact car line. That design pattern is self similar and fractal in that even car engines and transmissions and car radios undergo a similar pattern of revolution to make something new, then evolution to slowly and methodically make it the opposite of the original goal, repeat forever. This design pattern also applies to computer architecture. To fight the problem of hardware evolution making life pretty hard on programmers, source code compatible OS and libraries were invented that could run the same software on vaguely similar hardware, on mainframes in the late 60s at IBM and on PCs in the 70s CPM era, later extended into the "home computer" series era in the 80s, then into the msdos era. This became unwieldy and too complicated for the end users so it was replaced. The same people implemented the idea of OS packages, again more or less in the 60s on IBM mainframes or the 80s on early unix boxes. The idea is to compile emacs to be integrated very deeply into the OS and cooperate with every other piece of software. This is contemporary. However especially in NON-FOSS it doesn't work and doesn't scale and is very expensive if not impossible to implement, being the only closed thing on an open system is a nightmare for everyone and everything involved. The stereotype in the 90s was only one service installed on one MS windows server, even if that meant it took 20 MS servers to do the job of one unix server. Anyway, expensive, complicated, hard for end users. Again the same people implemented the idea of OS virtualization. Again, IBM mainframes in the 70s with VM, and early PC hardware experiments with TSRs and multiprocessing in the 80s to give "two computers at once". This is also contemporary, enormously more advanced today, of course. It turns out that running 50 OS kernels on a single piece of hardware is kinda wasteful although possible and cooperation gets complicated and unwieldy so time to replace again. Again the same people implemented process jails / chroot on the BSDs and eventually after some decades linux finally caught up resulting in docker. So now your packages don't cooperate or interoperate at all with each other or the OS, which solves all the problems where the previous technology didn't work, and creates massive new problems that never existed mostly where the old technologies worked great. There are of course completely separate new problems. It turns out that a system designed to eliminate interoperability between packages interoperability a huge PITA and there are other problems that make use unwieldy and complicated, hard for end users. Again the same people implemented (this section to be written after 2016). Maybe IOT. Maybe collapse of hardware prices faster than business demand means 20 rasp-pi cluster is cheaper and easier to maintain than a single beefy desktop running 20 virtual images or 20 docker containers. Maybe FPGA on the desktop means people will just synth up whatever matches this hours workload. Maybe cloud will finally work and no one will maintain servers anymore, it'll all be magic, or at least push the magic to someone else who now has the same old problems. Maybe SaaS means we'll all be customers and most productive software will run on internet scale clusters not individual machines, individual machines will be dumb webbrowser terminals. Who knows!

bsenftner 10y ago

Plain and simple: containers violate the KISS principal. While you are developing reams of domain knowledge working with this turd of a technology, others are making progress with non-self-created problems that matter in the real world. This is another case of the tech world developing pointless tech that can be ignored by those solving concrete real world problems.

novaleaf 10y ago

> The way virtual machines work, however, is by packaging the operating system and code together. The operating systems on the virtual machines believes that it has a server to its own, but in reality, it’s sharing the server with a bunch of other virtual machines — all of which run their own operating systems and don’t know of each other. Underneath it all is the host operating system that makes all of these guests believe they are the most important thing in the world. You can see why this is a problem. The guest virtual machines basically run on emulated servers, and that creates a lot of overhead and slows things down (but in return, you could run lots of different operating systems on the same server, too). Wait a second. Isn't this the whole point of hardware virtualization support? So that hypervisors don't have all those VM overhead slowing things down?

realworldview 10y ago

IBM and Sun know.

mmatoscom 10y ago

Hi there, good day fellows I was expecting this kind of thread long ago, thanks guys for sharing your concerns, I am learning a lot from them! IMO we can't compare containers vs VMs like many are doing now - and I was too when first heard about Docker and containers. I hold almost all VMware certs (VCP/VSP/VTSP for 3.1/4.x/5.x and VCDX), I sign the 3 major VMware P2V migrations in BR/LA (287 P2Vs in 2005, 1600/2008 and 2500/2011). I was REALLY into VMware from 2000 to 2010, so I feel confident using it and recommending to many environments. I even manage some of them still today. When we clone or migrate a physical machine to virtual, or whenever deploying a VM from scratch (or even using templates or copying VMdks etc) into production, we aim to develop the environment in order to see it lasting "forever". We want this to be flawless, because even with given most-players virtualization deploying resources (hyper-V, VMware, xen, kVM, vbox+vagrant, etc), nobody wants to troubleshoot a production environment, we cheer to see the VMs always up and running. I remember when P2V in mentioned projects during the night, and needed to fallback physical servers because the cloned VM didn't behaved accordingly. Please VMware, clone it ok, otherwise the troubleshoot for legacy shit will be a pain. On the other side, containers are made to be replaced. They are impermanent. You can tune your images as your app/env needs. You can have an image for each service you have. You can have many images running many containers, and some of them providing services you have. You are able to customize an image in a txt file called Dockerfile, and generate an image from this file. So imagine we got this infrastructure to "Dockerize" , a website with a DB. Does your webserver runs apache? so you can code a Dockerfile, that will deploy an apache instance to you. It could deploy FROM an ubuntu:16.10 or 10.04, depends on what is better for your app. OR, we could just pickup Apache's own image, like in FROM apache. You can save this image as yourid/apache. And you can do the same regardless of what DB you are using, just install it (the mysql using apt-get in a FROM ubuntu/debian based system), or use mysql images directly. You are able to publish the website cloning your site dev repo direct in Dockerfile itself, or you could have the website at some dir in your host, using ADD or CP to make it available in the right container dir (eg /var/www/) You could even use Volumes, to mount some host dir (or a blob, or a storage account, or a partition in your storage, something available in the host itself). This is specially interesting for DB in my opinion. Once you have your Dockerfile ok, you can name it yourid/db. And if you have a main website, and a blog website, you could use the same apache Dockerfile changing only the git clone line, and save them as yourid/apache:website and yourid/apache:blog for example. And when redeploying the container, you will have the same volume data available in the same container dir. Even if you redeployed it from ubuntu:15.10 to ubuntu 16:10.You can use the latest improvements from the freshest image (or patching some vuln), and redeploy all your containers that uses this same image at once. The same goes on, you can test jenkins without knowing what OS jenkins image is made off. You dont have to worry about it. It will just work. You pull the image and run the container and voila. NOW, my Docker instances are like this: I use Docker-machine, and locally I got the default and local envs. I got also an Azure instance at Docker-machine (that runs on Azure), and another instance configured in Docker-cloud using Azure as cloud provider (I use Azure due bizspark credits). So, 4 of them. All those instances, are VMs themselves. Ubuntu VMs to be sure. You just replace the container (probably published in a cluster if you care enough your prod env). Not the same as with VMs at all. I see Docker as a hypervisor for containers the same way VMware and hyperv are to VMs. So I understand my Docker hosts VMs have the same HA, failover, load balance, resources allocation, and so many resources VMs have. I use Docker on those VMs to make easy deploys, images, tune images, really guys I was the VMware guy for so long, I went just crazy in the resources Docker gives to us. Docker has many weak points indeed (NAT networking, privileged containers must be run sometimes, security concerns, etc), but again, I don't see Docker to be erasing VMs from my life and now on I can deploy everything that will run happy forever in containers. We still need HA, failover, load balance, resources allocation and so on. Docker needs to be used together with TOOLS that allows it to run smoothly, and allows us to maintain our environments easier. One of those tools are containers clusters. I work mostly with Google Kubernetes, but there are other as Docker Swarm, Apache Mesos, DCOS... Azure has its Azure Container Services ACS, IBM has its BlueMix containers, etc. Using a cluster and a deploy methodology, you are able to deploy your containers in different namespaces such as DEV / STAGING / PROD. You can use a BUILD environment to read your Dockerfile, build the image and deploy containers to the namespace you need. You can configure this build to trigger with a commit in the git repo for exemple. So lets say we have a developer, and he needs our yourid/apache:website to be deployed, with the new website version. If the website is already updated in your git repo, you just clone it. The Dockerfile would look like this: FROM apache MAINTAINER Your Name <your@email.com> WORKDIR /var/www/ RUN git clone https://github.com/yourid/website/website.git . EXPOSE 80 CMD ["/run.sh"] This would be named as website.Dockerfile. If you change the project git repo to any of your other sites that runs on apache, you can SAVE AS other.site.Dockerfile, and always deploy this service from this specific repo. You can customize your Dockerfile of course and add support to specific codes like installing PHP, Ruby, Python, etc. You could even use Configuration Managers (CMs) as Ansible, Salt, Chef, Puppet, juju etc to apply those changes. Lets say we will start the build now. We are developing this image together. So I just changed my git url on the Dockerfile. When we commit, the autobuild triggers this build in our build system (in my case, Docker-cloud or jenkins). This is what Continuous Integration (CI) and Continuous Deployment (CD) are about. So when build is triggered, it gets the Dockerfile from the repo, builds from its image, deploys the container in the namespace you wanted (our case, DEV). This service could be published as website.dev.mydomain.com for example. Same concerning to staging namespace. And to www.mydomain.com when ready to production, in the PROD kubernetes namespace for example. Kubernetes is a distributed thing, so you could have minions (nodes) splitted across different datacenters, or geo locations. This pretty much reminds me of VMware VMs running inside VMfs storage made available through a set of ESXi servers, all with access to the same luns/partitions/volumes. This is just my point of view, so please feel free to comment and ask me anything. Please, just dont blame Docker because you aren't aware of mainstream techs available nowadays. If you are comparing Docker to VMs, or SSHing inside the containers, and often mad cause your data vanished while redeploying your Docker Containers, believe me you are doing it wrong. Being a pioneer is often the same: in the 90s we had to explain why Linux was good for the enterprise, in the 2000's we had to prove VMware was really going to cluster your legacy systems, and now we have to explain what's possible to do with Docker. And, as the tech is new (I know there were previously solaris zones, google borg, etc), but I see Docker will mature its features relying in other tools (and even copying features from k8s to Swarm eg). Docker is just one skill needed to run your stuff. Cheers! M Matos https://keybase.io/mmatos https://keybase.io/mmatos

alauda 10y ago

Guys, you really want to give a look to hyper.sh, if you are frustrated with running containers in production.