5 ms·

I am very much a fan of hot-takes, but this one is trash -- > The money was wasted on hype. The same will eventually be said of Docker. I’ve yet to hear a sing

I am very much a fan of hot-takes, but this one is trash -- > The money was wasted on hype. The same will eventually be said of Docker. I’ve yet to hear a single benefit attributed to Docker that isn’t also true of other VMs, but standard VMs allow the use of standard operating systems that solved all the hard problems decades ago, whereas Docker is struggling to solve those problems today. Linux containerization (using the word "docker" for everything isn't right either) is an isolation + sandboxing mechanism, NOT a virtual machine. Even if you talk about things like LXC (orchestrated by LXD), that's basically just the addition of the user namespacing feature. A docker container is not a VM, it is a regular process, isolated with the use of cgroups and namespaces, possibly protected (like any other process) with selinux/apparmor/etc. Containerization is almost objectively a better way of running applications -- there's only one question, do you want your process to be isolated, or not. All the other stuff (using Dockerfiles, pulling images, the ease of running languages that require their own interpreters since you package the filesystem) is on top of this basic value propostion. An easy way to tell that someone doesn't know what they're talking about when speaking about containerization is if they call it a VM (and don't qualify/note that they're being fast and loose with terminology). All this said -- I do think Docker will die, and it should die because Docker is no longer the only game in town for reasonably managing (see: podman,crictl) and running containers (see: containerd/cri-o, libcontainer which turned into runc) . [EDIT] - I want to point out that I do not mean the Docker the company or Docker the project will "die" -- they have done amazing things for the community and development as a whole that will literally go down in history as a paradigm shift. What I should have written was that "docker <x>" where x is "image", "container", "registry", etc should be replaced by "container <x>".

chrisdone 7y ago
Deployment of Docker containers is nice, but deploying a VM as in Vagrant was also fine. I avoided learning anything about Docker for about 2 years because I thought it was just a fad. However, I would add that for my own personal use, it's invaluable for development work. All that work that you do _before_ your CI or deployment. 1) When I'm working with a collection of tools that I need but are a complete mess with lots of state (think: compiler tools, LaTeX, things like that), then docker image build with its incremental way of running each command piece by piece, and saving the state after each RUN, is actually a life saver. You follow the steps of some instructions, and of course, as usual, there's one extra step not documented in the manual, so you add that to your Dockerfile. You make a mistake, no big deal, just change the command, the bad state is discarded, and you get to try again. You don't have to run the whole thing all over again. And it's instantaneous. 2) When I have to work with a client's codebase, as a consultant, you'd be surprised how many projects do not have a reproducible build, with Docker or anything else. So I end up building my own Dockerfile. The number of times I've heard "but you just have to run this setup script once" -- well, those scripts never work (why would they? nobody runs them anymore). Especially when it begins with `npm` or `pip` -- almost guaranteed to fail catastrophically, with some g++ compile error, or a segfault, or just a backtrace that means nothing. For example, I recently had to run an `npm` install command and it failed with `npm ERR! write after end`. I re-ran the container again, and again once more, and then it succeeded (https://gist.github.com/chrisdone/ea6e4ba3d8bf2d02f491b4a17fdc4433 https://gist.github.com/chrisdone/ea6e4ba3d8bf2d02f491b4a17f...). npm has a race condition (https://github.com/npm/npm/issues/19989; https://github.com/npm/npm/issues/19989; fixed in the latest version). I wouldn't have been able to confidently share with my client this situation unless I had that reproducibility. 3) It's trivial to share my Dockerfile with anyone else and they can run the same build process. I don't have to share an opaque VM that's hundreds of megs and decide where to put it and how long I want to keep it there, etc. 4) It's a small one; but speed. Spinning up or resuming a VirtualBox machine is just slow. I can run docker containers like scripts, there isn't the same overhead. 5) Popularity is a blessing; the fact that I _can_ share my Dockerfile with someone is a network effect. Like using Git, instead of e.g. darcs or bzr. By the way, you can also do nice container management with systemd and git. There's nothing inherently technologically _new_ about Docker; it's the workflow and network effects; it lets me treat a system's state like a Git repo. Nix has similar advantages, but I see Docker as one small layer above Nix.
ransom1538 7y ago
"All this said -- I do think Docker will die" Harsh. Docker is a nice way to specify how you want your container to work. Sure release Bocker (A better Docker). But Bocker will just basically be Docker: a simple set of instructions to get a machine running. That is all we want. A few instructions to get a system up. Devs are sick of setting up machines. Demand is there.
- hardwaresofton 7y ago
  I agree the way I said it was harsh -- I don't mean Docker the company should die or Docker the project should die or anything of that nature, I mean literally people referring to everything in containerization as "docker" + <whatever the thing is>. Also, Docker itself is very much trying to be more than what it is right now, Docker swarm, compose, and basically everything Docker as a company does to improve their product offering moves them farther away from the way people are using "docker" in conversation right now. Basically, when people say "docker image" they should be saying "container image" (for lack of a better term, at least). > That is all we want. A few instructions to get a system up. Devs are sick of setting up machines. Demand is there. This is pretty vague -- "setting up machines" can mean a lot of things. I want to make it clear that it's not the container runtime's job to set up a single machine, it's to enable a process to pretend that the machine it has access to is it's own. It's a subtle difference but it's worth noting IMO. If you want to fix setting up your machines, you should be looking at tools like Packer, Container Linux distributions (formerly CoreOS, now Atomic I believe), Nix, Guix, and Linuxkit. Docker does not set up machines, it sets up processes, but happens to bring along a filesystem (amongst other things). A linux container is not a machine (virtualized or otherwise), it is an isolated process, I agree that devs are sick of worrying about machine configuration when setting up the required dependencies to run their processes, though, containerization is here to stay -- it was around before docker and will be here after.
gerbilly 7y ago
> do you want your process to be isolated, or not. No, not always. Why?. At work I have a few coworkers pushing hard to dockerize (isolate?) everything. This makes debugging when things go wrong a lot harder. I see isolation as one of several qualities a process could have, that sometimes is valuable enough to be worth the sacrifice. Isolation is not some absolute quality that is without significant tradeoffs.
- kenforthewin 7y ago
  > This makes debugging when things go wrong a lot harder. I've found the opposite to be true. If you have proper observability into your isolated services you can more easily pinpoint the origin of a bug.
- hardwaresofton 7y ago
  I avoided saying that processes should always be isolated because there are sometimes very good reasons to not isolate a processes with the containerization approach we're talking about, performance being one that came to mind quickly. Containerization of processes definitely increases complexity but if you can take the time to understand VMs then you can (and should, IMO) take the time to understand how containers work as well, they are lighter and simpler (for example, you don't need to build a kernel or make an initrd). I would argue that people who think VMs are simpler are actually being fooled by huge advancements in tooling over the years and the fact that it's become "easy", not that it was ever simple. I also want to point out that containers should actually make tracking down some bugs easier, but it does so in a counter-intuitive way -- it removes whole classes of bugs from ever occurring. You'll never have two programs clobber some shared folder or resource, you'll never have programs fight over dependencies, or struggle for locally-bound ports if you're running them in containers. Containerization definitely represents an increase in complexity, but it is well worth the effort, most of the time, granted you understand the tooling.
  - gerbilly 7y ago
    > if you can take the time to understand VMs then you can (and should, IMO) take the time to understand how containers work as well I don't see it as VMs vs containers. We have a good devops process to deploy onto our instances, so we rarely have resource clashes you mention (ports/directories) because none of that is ever configured manually. All our infrastructure is derived from 'scripts', so it hasn't been a problem at all. Aside from python, I see no advantage in containerizing any of our processes at all. As for debugging, I always forget how infuriating it is, till in the heat of the moment I have to open up a shell into someone's badly made docker image and try to use common tools to help diagnose a problem (ps, nslookup, dig, all) all missing from the wonderful little container. It's like being on a big navy ship, stranded in the ocean because the engines broke down, but everyone left all the tools back at the base. Yay!
cmsj 7y ago
I'm not going to support the general thesis of this article, but I want to address something you said. You're right that containers are not VMs, but that's only really relevant as pedantry of technical details. I think that what the author was trying to say (without really understanding it) was a comparison of containers to VMs as units of software deployment. I don't think anyone is credibly using containers as a security measure on Linux, because if they think they are, they are in for several large surprises. Rather, we're seeing the unbundling of software - it used to be that you deployed software to a physical machine with a full OS, then you could deploy it to a virtual machine with a full OS, then you could deploy the process, its dependencies and a minimal OS into a container. I agree that Docker doesn't have a huge and profitable future ahead of it, because it's providing commodity infrastructure. Rather I think it's interesting to think about what the next level of software deployment decomposition will be, and I'd wager that it's FaaS (ie serverless).
- vorpalhex 7y ago
  > You're right that containers are not VMs, but that's only really relevant as pedantry of technical details. That isn't pedantry, it is an extremely critical point and the one most people miss when figuring out Docker. Both from a security context (docker doesn't provide vm level promises about isolation) and from a resource management side (docker is really close to no overhead). It is not uncommon to deploy containers on VMs in the real world...
  - hardwaresofton 7y ago
    This is precisely why I said the fact that someone would gloss over this is a red flag. The point is super critical. VMs are literally so hard to do correctly and in a performant fashion that parts of CPU instruction sets[0], and kernel subsystems (KVM[1]) were created to make them easier to run. Containers, in contrast are literally a few flags and a bunch of in-kernel antics. A few people, notably Liz Rice and Jessie Frazelle have given talks on how to make containers from scratch that are very illuminating for those that are interested: https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=cYsVvV1aVss https://www.youtube.com/watch?v=cYsVvV1aVss [0]: https://en.wikipedia.org/wiki/X86_virtualization https://en.wikipedia.org/wiki/X86_virtualization [1]: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine
adds68 7y ago
Yes! What a load of trash this post is, should be on the last page!
YeGoblynQueenne 7y ago
So, last time I had a web dev job was back in 2016, so the whole container thing kind of passed me by. So I don't know the details- but, on the other hand, I also don't have any baggage about it being "a VM". But, starting from that kind of "clean slate" state I have to say that if it takes exasperated internet posts, like your helpful comment, to explain why containers are not like VMs and how they are not like VMs... well then maybe they are not that much not like VMs to make them such a big new thing. That goes for many things. Like, I don't get the difference between Volley and Beach Volley. One is played on the beach. So it's volley? Played on the beach?
- hardwaresofton 7y ago
  Well the thing is, they aren't a big new thing! But the hype train needed it to be (and to be fair, the ecosystem has benefited greatly from the attention). You rarely see hypetrains for old, established things :)
- linuxftw 7y ago
  Beach volleyball is played typically with 2's or 4's. The rules are slightly different, court size is smaller. There are no positions in 2's, there are no position faults. Feel free to make analogies between two other similar sports, such as Competition Karate and Taekwon-Do.
  - YeGoblynQueenne 7y ago
    I'm sorry but this still sounds like the same game to me. Why is it not called Beach 2-by-2-ball, or somethign similar, if the number of players makes it so very different? If I get a couple of my mates together and we bounce a ball over a net counting points when the ball hits the ground- how is what we're playing not (an informal game of) volleybal?
    - linuxftw 7y ago
      How is basketball different from indoor volleyball? They're both played indoors on a wooden floor court, both have two teams in opposition, playing simultaneously. Both have in bounds and out of bounds. Both have a ball. Both involve passing a ball. Both have scoring dictated by placement of the ball. I mean, we can talk about things that make things similar all day, it doesn't make them the same thing. Is a hamburger a sandwich? Sometimes, the whole is greater than the parts, not everything is perfectly decomposable in life.
nailer 7y ago
Docker famously doesn't isolate very well, as known in infosec circles for years now. If you're unaware, search 'containers don't contain'. MicroVMs start faster and provide better isolation. Meanwhile, none of this is relevant unless you're building your own cloud platform, which is a huge waste of time for most companies. MicroVMs, containers, VMs, zones and bare metal are places to execute code. Serverless makes all those distinctions irrelevant. Sorry if you spent 2015 getting really into Docker. You bet on the wrong horse. It's OK, this happens in tech. Edit: if it's unclear, I don't mean Docker itself is the wrong horse, I mean containerisation tools per se are the wrong horse - and a bad place to invest your time unless you work for a cloud provider
- supermatt 7y ago
  And given recent revelations, VMs don't contain either! Docker is far from betting on the wrong horse. I can build a docker container and deploy/orchestrate it however I wish - via a docker runtime, kubernetes (which can vary in its underlying implementation), microVMs, VMs, or bare metal. But then the containers don't need to be docker either. Theres many options available there too. Either way, there are many options available. And the horse(s) are still in the running. As for "serverless" vs containers/other, if you can provision accordingly, containers/other can be a better option at scale. Horses for courses, I guess.
  - nailer 7y ago
    Sure, having a lot of VM knowledge is also irrelevant unless you're building your own cloud platform too. * Again, most people don't need to build their own cloud platform. Being able to reuse Docker files is nice, you've reinvented CFengine for the eighth time. That's great. The point is Docker is yet another standard way to build boxes and contain them. Serverless / FaaS platforms obviate that need. * obviously an isolated kernel is better than a shared one, but the point of this post (and this comment) is that Docker vs VM vs MicroVMs vs zones etc is irrelevant for companies who are not cloud providers
- hardwaresofton 7y ago
  Your point is actually the point I'm trying to make -- you shouldn't be using docker to try and contain possibly malicious code, it's not for that. It does isolate a process's view of the filesystem, it does isolate a process's view of the PID namespace, and that is valuable. Let's say it's a spectrum: raw processes -------- processes with a certain user ---------- namespace+cgroup isolated processes ---------------- VMs Docker is certainly an improvement for processes that you don't want running completely contained but want somewhat isolated. For isolation in every sense, you want a VM (a lightweight one if you can get it, i.e. some stripped down qemu). > MicroVMs start faster and provide better isolation. Agreed on isolation -- VMs are hands down better at isolating, but I'm a bit skeptical about starting faster. > Meanwhile, none of this is relevant unless you're building your own cloud platform, which is a huge waste of time for most companies. Again, this is exactly why it's relevant, 99% of people aren't building their own cloud platform, so they don't need the full isolation of VMs -- most of the time they're just trying to prevent program A that their devs wrote from clobbering program B that their devs also wrote when they both run on the same machine, and making it easier to deploy the dependencies that come with each. > MicroVMs, containers, VMs, zones and bare metal are places to execute code. Serverless makes all those distinctions irrelevant. OK I don't even really know what this means, you know serverless runs on MicroVMs right? and most of the time it's actually containers in MicroVMs? MicroVMs are just stripped down versions of regular VMs, and no one is in a zone unless they're running Solaris. The distinction between these things is still very important, unless you mean that the future is everyone just deploying functions for their applications? But even if you mean that, cold start is basically the first stumbling block you see and it literally exists because of the distinction between how these technologies work (and how fast they can be started on demand, with how much isolation). > Sorry if you spent 2015 getting really into Docker. You bet on the wrong horse. It's OK, this happens in tech. Is there anyone that spent 2015 really getting into Docker and isn't better for it now? The technologies that have sprung out of this part of computing are very valuable to know and are getting more valuable, not less. You don't have to install a VM to run an isolated postgres instance on your dev machine because containerization exists -- if you're still doing this you should probably look into updating your tooling. Also, don't forget that containerization is how some of the richest and supposedly best (due to their ability to spend money on engineers) companies in the world have been handling deployment for nearly a decade -- 2015 is late for realizing containerization is a good thing, not early.
kazinator 7y ago
> they call it a VM (and don't qualify/note that they're being fast and loose with terminology). It's only confusing to people who are familiar just with the popularized forms of computer science terms. Isolation and sandboxing is virtualization. In a container, the applications seem to have an operating system and machine to themselves. A single Unix process and its address space is also a kind of virtual machine, creating the illusion that the process has a machine all to itself. Thanks to virtual memory, other processes are not even visible; they are in a different name space of pointers. That concept breaks for multi-process applications: processes are aware of each other through manipulations of shared resources like files. Or effects like not being able to bind a networking port because some other process is tying it up. The next level of virtualization is to have namespaces for resource-related namespaces in the system beyond the address space. As far as just the filesystem space goes, we can virtualize with tools like chroot. A group of applications can have their own global /etc configuration, their own version of the C library in /lib and so on. That's the beginning of "containerization".
- hardwaresofton 7y ago
  > It's only confusing to people who are familiar just with the popularized forms of computer science terms. Yeah given how often people mistakenly call containers VMs or assume they have VM-like properties (like the ability to run a different kernel), this is tripping a lot of developers up, not just lay people. > Isolation and sandboxing is virtualization. In a container, the applications seem to have an operating system and machine to themselves. I don't think this is quite true, generally you can isolate and sandbox something without virtualizing anything. In this case to isolate and sandbox without a loss of functionality, we are virtualizing and swapping swap implementations out from underneath an application/process. That is not always the case though -- if you unplug a machine's ethernet cable it is isolated, but you did not virtualize anything, if you deny access to a folder on disk for a process you are isolating without virtualizing, unless you mean that any kernel interaction (like blocking an `fopen()`) is "virtualization". > A single Unix process and its address space is also a kind of virtual machine, creating the illusion that the process has a machine all to itself. Thanks to virtual memory, other processes are not even visible; they are in a different name space of pointers. That concept breaks for multi-process applications: processes are aware of each other through manipulations of shared resources like files. Or effects like not being able to bind a networking port because some other process is tying it up. The next level of virtualization is to have namespaces for resource-related namespaces in the system beyond the address space. As far as just the filesystem space goes, we can virtualize with tools like chroot. A group of applications can have their own global /etc configuration, their own version of the C library in /lib and so on. That's the beginning of "containerization". This is a good summary of how containerization works, and the layers of isolation provided by the kernel -- the approach here is isolation by providing a fake, but that does not mean that isolation + sandboxing = virtualization, that just means that these forms of virtualization can be used to provide isolation & sandboxing.
pixelrevision 7y ago
It’s very easy to underestimate how helpful it can be when you first start working with it. It’s a black box that uses root to do everything and is a pain to debug. Because of this it becomes easy to hand wave it away. Once you have a properly setup project going and your entire build process is mostly repeatable the benefits start becoming more obvious. Yes, you can do all the same things to a certain extent in a vm but it’s really hard to keep that streamlined and up to date. Having a script that sets up your stack in a vm on both windows and Mac then runs on Linux is a pretty big maintenance nightmare. A dockerfile works with a few commands and can be added to your repo. It’s not without trade offs but I think if they can solve the issue of debugging in a better way then we’ll really see things solidify on this concept.
- hardwaresofton 7y ago
  > It’s a black box that uses root to do everything and is a pain to debug. This is less true these days (and on macs that had to use the docker machine "hack" it was barely ever true, per say) -- rootless containers are on the way thanks to user namespaces. For example LXC can run fully rootless containers that act more like VMs themselves (as in they will have systemd as pid 1 inside) -- kernel support, user namespaces, and File systems in Userspace (FUSE) make this possible. IMO One of the biggest benefits to running containers is actually E2E tests -- I don't see it done as often as I should, but it has become drastically easier to run an entire postgres instance for a single local E2E test runs -- I do this on almost every project I start now, I set up E2E tests that spin up the actual world I expect in production (so all the backing services, at the version they will run at), and interact with my application -- this is a huge step forward compared to a huge wiki that has "how to set up the local test VM" -- you can spin up and shutdown these services so fast that you can use them ephemerally (I do) without much worry.
  - panpanna 7y ago
    Lxc/lxd is a fantastic piece of software that I wish more people would try. Especially if people already on Ubuntu, Debian and Arch. You absolutely don't need to be a cloud person to use containers. For example I use lxc on my laptop to keep different projects separate and test new software.
linuxftw 7y ago
I agree that the industry will waste billions on startups. There's not really a market there for 'Docker' containers, and there never was. docker-containers and it's related counterparts are abstractions. Useful abstractions don't necessarily equate to a new line of business. The best attributes of containers (IMO) are packaging and distribution. What business and operators need is a repeatable, easy way to deploy applications across there infrastructure. Containers are one piece of that story. The bigger piece, and IMO, where the business viability is, is the orchestration layer. Containers aren't very useful by themselves, you need a way to get your application online. That's where Kubernetes comes in. You need to understand large organizations and their challenges to see what layer of the containerization stack holds the most value. Long-term, I see 'linux containers' as we know them going away. The industry is going to move to something like [1]: lightweight, hardware-assisted VM/container hybrids. But, no matter what happens at the containerization layer, the orchestration layer is the piece that adds business value to end-users (eg, not AWS or other hosting providers). 1: https://katacontainers.io/ https://katacontainers.io/
SmokeGS 7y ago
I appreciate your nuance and defense in this thread.

chrisdone 7y ago

Deployment of Docker containers is nice, but deploying a VM as in Vagrant was also fine. I avoided learning anything about Docker for about 2 years because I thought it was just a fad. However, I would add that for my own personal use, it's invaluable for development work. All that work that you do _before_ your CI or deployment. 1) When I'm working with a collection of tools that I need but are a complete mess with lots of state (think: compiler tools, LaTeX, things like that), then docker image build with its incremental way of running each command piece by piece, and saving the state after each RUN, is actually a life saver. You follow the steps of some instructions, and of course, as usual, there's one extra step not documented in the manual, so you add that to your Dockerfile. You make a mistake, no big deal, just change the command, the bad state is discarded, and you get to try again. You don't have to run the whole thing all over again. And it's instantaneous. 2) When I have to work with a client's codebase, as a consultant, you'd be surprised how many projects do not have a reproducible build, with Docker or anything else. So I end up building my own Dockerfile. The number of times I've heard "but you just have to run this setup script once" -- well, those scripts never work (why would they? nobody runs them anymore). Especially when it begins with `npm` or `pip` -- almost guaranteed to fail catastrophically, with some g++ compile error, or a segfault, or just a backtrace that means nothing. For example, I recently had to run an `npm` install command and it failed with `npm ERR! write after end`. I re-ran the container again, and again once more, and then it succeeded (https://gist.github.com/chrisdone/ea6e4ba3d8bf2d02f491b4a17fdc4433 https://gist.github.com/chrisdone/ea6e4ba3d8bf2d02f491b4a17f...). npm has a race condition (https://github.com/npm/npm/issues/19989; https://github.com/npm/npm/issues/19989; fixed in the latest version). I wouldn't have been able to confidently share with my client this situation unless I had that reproducibility. 3) It's trivial to share my Dockerfile with anyone else and they can run the same build process. I don't have to share an opaque VM that's hundreds of megs and decide where to put it and how long I want to keep it there, etc. 4) It's a small one; but speed. Spinning up or resuming a VirtualBox machine is just slow. I can run docker containers like scripts, there isn't the same overhead. 5) Popularity is a blessing; the fact that I _can_ share my Dockerfile with someone is a network effect. Like using Git, instead of e.g. darcs or bzr. By the way, you can also do nice container management with systemd and git. There's nothing inherently technologically _new_ about Docker; it's the workflow and network effects; it lets me treat a system's state like a Git repo. Nix has similar advantages, but I see Docker as one small layer above Nix.

ransom1538 7y ago

"All this said -- I do think Docker will die" Harsh. Docker is a nice way to specify how you want your container to work. Sure release Bocker (A better Docker). But Bocker will just basically be Docker: a simple set of instructions to get a machine running. That is all we want. A few instructions to get a system up. Devs are sick of setting up machines. Demand is there.

hardwaresofton 7y ago
I agree the way I said it was harsh -- I don't mean Docker the company should die or Docker the project should die or anything of that nature, I mean literally people referring to everything in containerization as "docker" + <whatever the thing is>. Also, Docker itself is very much trying to be more than what it is right now, Docker swarm, compose, and basically everything Docker as a company does to improve their product offering moves them farther away from the way people are using "docker" in conversation right now. Basically, when people say "docker image" they should be saying "container image" (for lack of a better term, at least). > That is all we want. A few instructions to get a system up. Devs are sick of setting up machines. Demand is there. This is pretty vague -- "setting up machines" can mean a lot of things. I want to make it clear that it's not the container runtime's job to set up a single machine, it's to enable a process to pretend that the machine it has access to is it's own. It's a subtle difference but it's worth noting IMO. If you want to fix setting up your machines, you should be looking at tools like Packer, Container Linux distributions (formerly CoreOS, now Atomic I believe), Nix, Guix, and Linuxkit. Docker does not set up machines, it sets up processes, but happens to bring along a filesystem (amongst other things). A linux container is not a machine (virtualized or otherwise), it is an isolated process, I agree that devs are sick of worrying about machine configuration when setting up the required dependencies to run their processes, though, containerization is here to stay -- it was around before docker and will be here after.

gerbilly 7y ago

> do you want your process to be isolated, or not. No, not always. Why?. At work I have a few coworkers pushing hard to dockerize (isolate?) everything. This makes debugging when things go wrong a lot harder. I see isolation as one of several qualities a process could have, that sometimes is valuable enough to be worth the sacrifice. Isolation is not some absolute quality that is without significant tradeoffs.

kenforthewin 7y ago
> This makes debugging when things go wrong a lot harder. I've found the opposite to be true. If you have proper observability into your isolated services you can more easily pinpoint the origin of a bug.
hardwaresofton 7y ago
I avoided saying that processes should always be isolated because there are sometimes very good reasons to not isolate a processes with the containerization approach we're talking about, performance being one that came to mind quickly. Containerization of processes definitely increases complexity but if you can take the time to understand VMs then you can (and should, IMO) take the time to understand how containers work as well, they are lighter and simpler (for example, you don't need to build a kernel or make an initrd). I would argue that people who think VMs are simpler are actually being fooled by huge advancements in tooling over the years and the fact that it's become "easy", not that it was ever simple. I also want to point out that containers should actually make tracking down some bugs easier, but it does so in a counter-intuitive way -- it removes whole classes of bugs from ever occurring. You'll never have two programs clobber some shared folder or resource, you'll never have programs fight over dependencies, or struggle for locally-bound ports if you're running them in containers. Containerization definitely represents an increase in complexity, but it is well worth the effort, most of the time, granted you understand the tooling.
- gerbilly 7y ago
  > if you can take the time to understand VMs then you can (and should, IMO) take the time to understand how containers work as well I don't see it as VMs vs containers. We have a good devops process to deploy onto our instances, so we rarely have resource clashes you mention (ports/directories) because none of that is ever configured manually. All our infrastructure is derived from 'scripts', so it hasn't been a problem at all. Aside from python, I see no advantage in containerizing any of our processes at all. As for debugging, I always forget how infuriating it is, till in the heat of the moment I have to open up a shell into someone's badly made docker image and try to use common tools to help diagnose a problem (ps, nslookup, dig, all) all missing from the wonderful little container. It's like being on a big navy ship, stranded in the ocean because the engines broke down, but everyone left all the tools back at the base. Yay!

cmsj 7y ago

I'm not going to support the general thesis of this article, but I want to address something you said. You're right that containers are not VMs, but that's only really relevant as pedantry of technical details. I think that what the author was trying to say (without really understanding it) was a comparison of containers to VMs as units of software deployment. I don't think anyone is credibly using containers as a security measure on Linux, because if they think they are, they are in for several large surprises. Rather, we're seeing the unbundling of software - it used to be that you deployed software to a physical machine with a full OS, then you could deploy it to a virtual machine with a full OS, then you could deploy the process, its dependencies and a minimal OS into a container. I agree that Docker doesn't have a huge and profitable future ahead of it, because it's providing commodity infrastructure. Rather I think it's interesting to think about what the next level of software deployment decomposition will be, and I'd wager that it's FaaS (ie serverless).

vorpalhex 7y ago
> You're right that containers are not VMs, but that's only really relevant as pedantry of technical details. That isn't pedantry, it is an extremely critical point and the one most people miss when figuring out Docker. Both from a security context (docker doesn't provide vm level promises about isolation) and from a resource management side (docker is really close to no overhead). It is not uncommon to deploy containers on VMs in the real world...
- hardwaresofton 7y ago
  This is precisely why I said the fact that someone would gloss over this is a red flag. The point is super critical. VMs are literally so hard to do correctly and in a performant fashion that parts of CPU instruction sets[0], and kernel subsystems (KVM[1]) were created to make them easier to run. Containers, in contrast are literally a few flags and a bunch of in-kernel antics. A few people, notably Liz Rice and Jessie Frazelle have given talks on how to make containers from scratch that are very illuminating for those that are interested: https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=HPuvDm8IC-4 https://www.youtube.com/watch?v=cYsVvV1aVss https://www.youtube.com/watch?v=cYsVvV1aVss [0]: https://en.wikipedia.org/wiki/X86_virtualization https://en.wikipedia.org/wiki/X86_virtualization [1]: https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine https://en.wikipedia.org/wiki/Kernel-based_Virtual_Machine

adds68 7y ago

Yes! What a load of trash this post is, should be on the last page!

YeGoblynQueenne 7y ago

So, last time I had a web dev job was back in 2016, so the whole container thing kind of passed me by. So I don't know the details- but, on the other hand, I also don't have any baggage about it being "a VM". But, starting from that kind of "clean slate" state I have to say that if it takes exasperated internet posts, like your helpful comment, to explain why containers are not like VMs and how they are not like VMs... well then maybe they are not that much not like VMs to make them such a big new thing. That goes for many things. Like, I don't get the difference between Volley and Beach Volley. One is played on the beach. So it's volley? Played on the beach?

hardwaresofton 7y ago
Well the thing is, they aren't a big new thing! But the hype train needed it to be (and to be fair, the ecosystem has benefited greatly from the attention). You rarely see hypetrains for old, established things :)
linuxftw 7y ago
Beach volleyball is played typically with 2's or 4's. The rules are slightly different, court size is smaller. There are no positions in 2's, there are no position faults. Feel free to make analogies between two other similar sports, such as Competition Karate and Taekwon-Do.
- YeGoblynQueenne 7y ago
  I'm sorry but this still sounds like the same game to me. Why is it not called Beach 2-by-2-ball, or somethign similar, if the number of players makes it so very different? If I get a couple of my mates together and we bounce a ball over a net counting points when the ball hits the ground- how is what we're playing not (an informal game of) volleybal?
  - linuxftw 7y ago
    How is basketball different from indoor volleyball? They're both played indoors on a wooden floor court, both have two teams in opposition, playing simultaneously. Both have in bounds and out of bounds. Both have a ball. Both involve passing a ball. Both have scoring dictated by placement of the ball. I mean, we can talk about things that make things similar all day, it doesn't make them the same thing. Is a hamburger a sandwich? Sometimes, the whole is greater than the parts, not everything is perfectly decomposable in life.

nailer 7y ago

Docker famously doesn't isolate very well, as known in infosec circles for years now. If you're unaware, search 'containers don't contain'. MicroVMs start faster and provide better isolation. Meanwhile, none of this is relevant unless you're building your own cloud platform, which is a huge waste of time for most companies. MicroVMs, containers, VMs, zones and bare metal are places to execute code. Serverless makes all those distinctions irrelevant. Sorry if you spent 2015 getting really into Docker. You bet on the wrong horse. It's OK, this happens in tech. Edit: if it's unclear, I don't mean Docker itself is the wrong horse, I mean containerisation tools per se are the wrong horse - and a bad place to invest your time unless you work for a cloud provider

supermatt 7y ago
And given recent revelations, VMs don't contain either! Docker is far from betting on the wrong horse. I can build a docker container and deploy/orchestrate it however I wish - via a docker runtime, kubernetes (which can vary in its underlying implementation), microVMs, VMs, or bare metal. But then the containers don't need to be docker either. Theres many options available there too. Either way, there are many options available. And the horse(s) are still in the running. As for "serverless" vs containers/other, if you can provision accordingly, containers/other can be a better option at scale. Horses for courses, I guess.
- nailer 7y ago
  Sure, having a lot of VM knowledge is also irrelevant unless you're building your own cloud platform too. * Again, most people don't need to build their own cloud platform. Being able to reuse Docker files is nice, you've reinvented CFengine for the eighth time. That's great. The point is Docker is yet another standard way to build boxes and contain them. Serverless / FaaS platforms obviate that need. * obviously an isolated kernel is better than a shared one, but the point of this post (and this comment) is that Docker vs VM vs MicroVMs vs zones etc is irrelevant for companies who are not cloud providers
hardwaresofton 7y ago
Your point is actually the point I'm trying to make -- you shouldn't be using docker to try and contain possibly malicious code, it's not for that. It does isolate a process's view of the filesystem, it does isolate a process's view of the PID namespace, and that is valuable. Let's say it's a spectrum: raw processes -------- processes with a certain user ---------- namespace+cgroup isolated processes ---------------- VMs Docker is certainly an improvement for processes that you don't want running completely contained but want somewhat isolated. For isolation in every sense, you want a VM (a lightweight one if you can get it, i.e. some stripped down qemu). > MicroVMs start faster and provide better isolation. Agreed on isolation -- VMs are hands down better at isolating, but I'm a bit skeptical about starting faster. > Meanwhile, none of this is relevant unless you're building your own cloud platform, which is a huge waste of time for most companies. Again, this is exactly why it's relevant, 99% of people aren't building their own cloud platform, so they don't need the full isolation of VMs -- most of the time they're just trying to prevent program A that their devs wrote from clobbering program B that their devs also wrote when they both run on the same machine, and making it easier to deploy the dependencies that come with each. > MicroVMs, containers, VMs, zones and bare metal are places to execute code. Serverless makes all those distinctions irrelevant. OK I don't even really know what this means, you know serverless runs on MicroVMs right? and most of the time it's actually containers in MicroVMs? MicroVMs are just stripped down versions of regular VMs, and no one is in a zone unless they're running Solaris. The distinction between these things is still very important, unless you mean that the future is everyone just deploying functions for their applications? But even if you mean that, cold start is basically the first stumbling block you see and it literally exists because of the distinction between how these technologies work (and how fast they can be started on demand, with how much isolation). > Sorry if you spent 2015 getting really into Docker. You bet on the wrong horse. It's OK, this happens in tech. Is there anyone that spent 2015 really getting into Docker and isn't better for it now? The technologies that have sprung out of this part of computing are very valuable to know and are getting more valuable, not less. You don't have to install a VM to run an isolated postgres instance on your dev machine because containerization exists -- if you're still doing this you should probably look into updating your tooling. Also, don't forget that containerization is how some of the richest and supposedly best (due to their ability to spend money on engineers) companies in the world have been handling deployment for nearly a decade -- 2015 is late for realizing containerization is a good thing, not early.

kazinator 7y ago

> they call it a VM (and don't qualify/note that they're being fast and loose with terminology). It's only confusing to people who are familiar just with the popularized forms of computer science terms. Isolation and sandboxing is virtualization. In a container, the applications seem to have an operating system and machine to themselves. A single Unix process and its address space is also a kind of virtual machine, creating the illusion that the process has a machine all to itself. Thanks to virtual memory, other processes are not even visible; they are in a different name space of pointers. That concept breaks for multi-process applications: processes are aware of each other through manipulations of shared resources like files. Or effects like not being able to bind a networking port because some other process is tying it up. The next level of virtualization is to have namespaces for resource-related namespaces in the system beyond the address space. As far as just the filesystem space goes, we can virtualize with tools like chroot. A group of applications can have their own global /etc configuration, their own version of the C library in /lib and so on. That's the beginning of "containerization".

hardwaresofton 7y ago
> It's only confusing to people who are familiar just with the popularized forms of computer science terms. Yeah given how often people mistakenly call containers VMs or assume they have VM-like properties (like the ability to run a different kernel), this is tripping a lot of developers up, not just lay people. > Isolation and sandboxing is virtualization. In a container, the applications seem to have an operating system and machine to themselves. I don't think this is quite true, generally you can isolate and sandbox something without virtualizing anything. In this case to isolate and sandbox without a loss of functionality, we are virtualizing and swapping swap implementations out from underneath an application/process. That is not always the case though -- if you unplug a machine's ethernet cable it is isolated, but you did not virtualize anything, if you deny access to a folder on disk for a process you are isolating without virtualizing, unless you mean that any kernel interaction (like blocking an `fopen()`) is "virtualization". > A single Unix process and its address space is also a kind of virtual machine, creating the illusion that the process has a machine all to itself. Thanks to virtual memory, other processes are not even visible; they are in a different name space of pointers. That concept breaks for multi-process applications: processes are aware of each other through manipulations of shared resources like files. Or effects like not being able to bind a networking port because some other process is tying it up. The next level of virtualization is to have namespaces for resource-related namespaces in the system beyond the address space. As far as just the filesystem space goes, we can virtualize with tools like chroot. A group of applications can have their own global /etc configuration, their own version of the C library in /lib and so on. That's the beginning of "containerization". This is a good summary of how containerization works, and the layers of isolation provided by the kernel -- the approach here is isolation by providing a fake, but that does not mean that isolation + sandboxing = virtualization, that just means that these forms of virtualization can be used to provide isolation & sandboxing.

pixelrevision 7y ago

It’s very easy to underestimate how helpful it can be when you first start working with it. It’s a black box that uses root to do everything and is a pain to debug. Because of this it becomes easy to hand wave it away. Once you have a properly setup project going and your entire build process is mostly repeatable the benefits start becoming more obvious. Yes, you can do all the same things to a certain extent in a vm but it’s really hard to keep that streamlined and up to date. Having a script that sets up your stack in a vm on both windows and Mac then runs on Linux is a pretty big maintenance nightmare. A dockerfile works with a few commands and can be added to your repo. It’s not without trade offs but I think if they can solve the issue of debugging in a better way then we’ll really see things solidify on this concept.

hardwaresofton 7y ago
> It’s a black box that uses root to do everything and is a pain to debug. This is less true these days (and on macs that had to use the docker machine "hack" it was barely ever true, per say) -- rootless containers are on the way thanks to user namespaces. For example LXC can run fully rootless containers that act more like VMs themselves (as in they will have systemd as pid 1 inside) -- kernel support, user namespaces, and File systems in Userspace (FUSE) make this possible. IMO One of the biggest benefits to running containers is actually E2E tests -- I don't see it done as often as I should, but it has become drastically easier to run an entire postgres instance for a single local E2E test runs -- I do this on almost every project I start now, I set up E2E tests that spin up the actual world I expect in production (so all the backing services, at the version they will run at), and interact with my application -- this is a huge step forward compared to a huge wiki that has "how to set up the local test VM" -- you can spin up and shutdown these services so fast that you can use them ephemerally (I do) without much worry.
- panpanna 7y ago
  Lxc/lxd is a fantastic piece of software that I wish more people would try. Especially if people already on Ubuntu, Debian and Arch. You absolutely don't need to be a cloud person to use containers. For example I use lxc on my laptop to keep different projects separate and test new software.

linuxftw 7y ago

I agree that the industry will waste billions on startups. There's not really a market there for 'Docker' containers, and there never was. docker-containers and it's related counterparts are abstractions. Useful abstractions don't necessarily equate to a new line of business. The best attributes of containers (IMO) are packaging and distribution. What business and operators need is a repeatable, easy way to deploy applications across there infrastructure. Containers are one piece of that story. The bigger piece, and IMO, where the business viability is, is the orchestration layer. Containers aren't very useful by themselves, you need a way to get your application online. That's where Kubernetes comes in. You need to understand large organizations and their challenges to see what layer of the containerization stack holds the most value. Long-term, I see 'linux containers' as we know them going away. The industry is going to move to something like [1]: lightweight, hardware-assisted VM/container hybrids. But, no matter what happens at the containerization layer, the orchestration layer is the piece that adds business value to end-users (eg, not AWS or other hosting providers). 1: https://katacontainers.io/ https://katacontainers.io/

SmokeGS 7y ago

I appreciate your nuance and defense in this thread.