14 ms·

Containers vs. Zones vs. Jails vs. VMs (2017)

351 points by gullyfur 7y ago

mooreds 7y ago
Note this is from 2017. Previous discussion: https://news.ycombinator.com/item?id=13982620 https://news.ycombinator.com/item?id=13982620
- dang 7y ago
  Year added. Thanks!
outworlder 7y ago
> A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. Linux namespaces and cgroups ARE first class objects. NOT containers. Amen. Somewhat tangential note: most developers I have met do not understand what a 'container' is. There's an aura of magic and mystique around them. And a heavy emphasis on Docker. A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it. No mention to storage, or how networking would be impacted, just CPU. That's usually said without measuring the actual performance first. When I press further, what I most commonly get is the sense that they believe that containers are "like VMs, but lighter"(also, I've been told that, literally, a few times, specially when interviewing candidates). To this day, I've heard CGroups being mentioned only once. I wonder if I'm stuck in the wrong bubble, or if this is widespread.
- pojzon 7y ago
  This is widespread unfortunately. Developers changed to „users” and no longer pursue the details of solutions. Im not gonna say its the dominant behaviour in the field right now, but I see it more and more often on various experience levels.
  - jagged-chisel 7y ago
    An experienced software engineer (yeah, developer) has experience engineering software. It’s been less than 10yrs since the advent of containerized deployments, and the space has been fraught with change nearly on par with the front end JavaScript ecosystem. Might as well just stick to writing code. OK, that’s the perception of my own peers, but I assume it scales. DevOps is a recent advent, too, and sounds to me like it should be populated with folks who can participate in development and operations. Most developers I’ve ever known aren’t interested in operations.
    - dielectrikboog 7y ago
      SRE was born of putting software engineers to work building operational software and automation tailored to an organization and application. In contrast, no matter what anyone says, DevOps was objectively born of replacing the operations discipline and career track with a poorly-understood tool economy and ongoing opex to a cloud provider. As you say, typical JavaScript engineers can’t be bothered to understand network capacity planning yet feel they are more qualified to take their application to production by deferring all decisions to cloud providers. Who all employ SREs/PEs, not DevOps Engineers, by the way, and there is a big distinction. We have people who can handle the operational stuff. They’re called systems administrators, network engineers, yes, even SREs, and other folks who are really good at understanding how computers and the Internet actually work, and a webdev bootcamp gives zero context into exactly what they do. None. Then, your ten-head startup suddenly scales to needing a physical footprint because it will literally save 80% of opex, and all your DevOps Engineers say “but why? There’s AWS,” and you’re in Medium hell for weeks arguing about it. Apropos, if I interview you and find you have written a thoughtpiece on Medium about how “sysadmin is evolving” and it’s “time for the gray beards to learn code,” you do not get a call back. That has actually happened, and no, sysadmin is not evolving. I know staff-level “full stack engineers” who can’t tell me what an ASN is. The motions in the industry have merely made those people more in demand at a few companies and served to cement their position as “where computing gets done”. Expect serious, existential operational threats and breaches to rise dramatically as DevOps continues to “win,” and consider it a smart strategic move to avoid DevOps culture in the long term. If you write a req called “DevOps Engineer,” I don’t even know what to say to you.
- wmf 7y ago
  People love to bring this up, but if Linux did have first-class containers, how would the developer's experience be different?
  - lmm 7y ago
    There would maybe be more consistency. E.g. currently if I say an application is running in a container, do you expect there is virtual networking in place, or not?
    - wmf 7y ago
      Jails and Zones probably also have bridged, routed, and NATed modes so I'm not sure that example is that useful. It's true that networking is different in Docker vs. k8s but there are valid reasons for it.
  - cyphar 7y ago
    All system programs could operate and manage the containers running on the system, meaning you can use your existing knowledge to manage a bunch of containers. For instance, you could run your package manager across all containers to see if they have packages with known CVEs. Or manage the filesystems of all containers on the system (the usefulness of this is only clear with filesystems like ZFS and btrfs). This is effectively what you can do with Solaris Zones. These kinds of improvements to usability aren't as sexy now that everyone is really excited about Kubernetes (where you blow away containers at whim) but it is useful for the more traditional container usecases that Zones and Jails. LXC (and LXD) is probably the one Linux container runtime that is closest to that original container model. There's also a very big security argument -- it is (speaking as a maintainer of runc) very hard to construct a secure container on Linux. There are dozens of different facilities you need to individually configure in the right order, with hundreds of individual knobs that some users might want to disable or quirks you need to work around. It's basically impossible to implement a sane and secure container runtime without having read and understood the kernel code which implements the interfaces you're using. If containers were an in-kernel primitive then all of the security design would rest in one single codebase, and all of the policies would be defined by one entity (the kernel).
- mulmen 7y ago
  I think it is widespread because containers are (seemingly) marketed as being some kind of magic. The impression I get is that the benefit to containers is that you don't have to think about them. This may be more a product of Docker but I think containers and Docker have become synonymous. I'm not sure this is a fair comparison but that is my impression. I could be in a bubble too.
  - nurpax 7y ago
    I’d like to think so too. It’s hard to actually understand how Docker works by reading their documentation. Everything looks so simple.. and it magically runs on macOS and Windows too. No wonder people think it’s some sort of a VM.
    - fapjacks 7y ago
      Technically, containers running on MacOS and Windows are a (Linux) VM, under the hood. Now, Microsoft put in a lot of work to support something like namespaces in the Windows kernel, so that it's now possible to run "Windows-native containers" for Windows software. But both Windows and MacOS still use a Linux VM under the hood to run Linux containers. That being said, I'm not familiar with the details of Windows containers because I don't use any Windows software and therefore don't have a reason to run any Windows containers. If you are interested in how containers work at a lower level, you might get more out of the documentation for containerd and runc, which are the underlying container runtimes.
    - ignoramous 7y ago
      > No wonder people think it’s some sort of a VM. From what I read, if one packages a Linux container for Windows, Docker does then run it in HyperV? https://stackoverflow.com/questions/52164563/what-is-the-docker-cross-platform-architecture https://stackoverflow.com/questions/52164563/what-is-the-doc...
  - takeda 7y ago
    It is product of docker, if you would deploy applications by using namespaces and cgroups directly it is very likely you would see things the same way the author does.
- erjiang 7y ago
  This is true for many new waves of popular technologies. 1) A new technology or method becomes popular. 2) Developers find new advantages in using the technology. 3) Understanding of tech and original advantage is somewhat lost. For example: containers are now widely used as part of a scriptable application build process, e.g. the Dockerfile. There are probably many developers out there who care about this and not about how containers are run and how they interact with the kernel. And for their use cases, that is probably the thing that matters most anyways.
  - fiddlerwoaroof 7y ago
    A down side is that people feel like they have to bundle an entire Linux rootfs because they think of a container as a lightweight vm: if they thought of it as a os process running inside various namespaces, they might be more inclined to only ship what they actually need.
- loeg 7y ago
  > Somewhat tangential note: most developers I have met do not understand what a 'container' is. The problem is at least in part that the term is basically meaningless. It's too broad and flexible to be descriptive. As you quoted: > A “container” is just a term people use to describe a combination of Linux namespaces and cgroups.
  - sanderjd 7y ago
    I'm more sympathetic to the mainstream usage. People are (attempting to) use an abstraction: a "container" is an isolated process runtime environment without virtualization overhead. That abstraction seems useful to me. Ideally it would be usable without too much leakiness, in which case its users would not need to be aware of implementation details like cgroups and namespaces. In practice, all abstractions are leaky to some degree and effective use of an abstraction often eventually requires a more sophisticated understanding of the details beneath the veil of the abstraction. But that doesn't mean the abstraction is totally useless or completely a mirage or anything, it's just a leaky abstraction like all others. If you say that a container is not a first class object but cgroups and namespaces are, I can just as easily say that cgroups and namespaces aren't first class objects, they are just terms people use to describe a combination of system calls. It's just abstractions with different amounts of leakage the whole way down.
- tasogare 7y ago
  The "containers are just XX Linux technology" comes regularly in the comments but it’s untrue: Windows containers are obviously not based on any Linux tech proper. Also the overhead intuition exists for a reason: on both macOS and Windows when Linux containers are used, there is actually a whole VM running Linux underneath. And Windows containers come in two flavors, one being Hyper-V based, so again a VM tech comes in play. So there are technical reasons why containers are "misunderstood", it’s because most people don’t run Linux natively, and on their stack containers are more than just cgroups and namespaces.
- matharmin 7y ago
  I feel nowadays containers generally refer to the concept, and cgroups and namespaces are the implementation details of a specific container runtime. These are very important implementation details for security and performance, but it doesn't fundamentally impact how you structure your containerized application. You can take the same container image, and run using Docker, Firecracker, gVisor, or many other container runtimes. Some of them are exactly "like a VM, but lighter".
  - cmckn 7y ago
    Agreed. The post feels a bit pedantic; I don't know any dev doing "cool things" with the underlying namespaces/cgroups. They're just using Docker. De-mystifying containers has value, but so does the abstraction.
    - mav3rick 7y ago
      The abstraction muddles the performance, security etc. impact of these two models. Not knowing them is going to be bad in the long run. Not everyone is a web dev.
    - mav3rick 7y ago
      And I'm a dev "doing cool things" with namespaces / cgroups.
    - moomin 7y ago
      Plenty of them using those features, if they’re using Kubernetes, Docker Swarm or especially Istio. They might not know they are, but that’s besides the point.
      - jdmichal 7y ago
        > They might not know they are, but that’s besides the point. I actually thought that was exactly the point here...
    - bregma 7y ago
      Your personal ignorance makes a poor argument. Not only do I know several devs doing cool things with namespaces and cgroup, but I myself have done cool things with namespaces and cgroups. I played with Docker briefly, but it has no real practical application in my line of work.
- taf2 7y ago
  Thanks I’d have fallen into your category of developers because in large part I never bothered with containers since we have everything running on vms and we’ve already isolated things so I’ve had only partial interest in exploring them ... but now that I know in linux it’s CGroups and namespaces that helps a lot in understanding thanks
- jjtheblunt 7y ago
  widespread in my experience; it reminds me to users of npm not seeming to understand what node is, for example.
- k__ 7y ago
  Doesn't surprise me if I think about all the people who suddendly need a K8s cluster...
- CBLT 7y ago
  > To this day, I've heard CGroups being mentioned only once. See https://www.kernel.org/doc/Documentation/cgroup-v2.txt https://www.kernel.org/doc/Documentation/cgroup-v2.txt > "cgroup" stands for "control group" and is never capitalized. The singular form is used to designate the whole feature and also as a qualifier as in "cgroup controllers". When explicitly referring to multiple individual control groups, the plural form "cgroups" is used. To this day, I've heard cgroup mentioned only once... To put forth a more substantive argument, everybody has a layer of abstraction they do not peek under. You interviewed people that didn't peek under container. You went a layer deeper, but never peeked at the source tree to learn what cgroup really is. Does it really feel that much better to be one level above others?
  - Harlekuin 7y ago
    > everybody has a layer of abstraction they do not peek under Reminds of this xkcd: https://xkcd.com/435/ https://xkcd.com/435/ The beauty of programming is abstraction - you write a function to do one thing, do it well, and then you can abstract that concept away and reuse that abstraction. Although it's only an abstraction - a container is "like" a lightweight VM, and you can use it like that until it doesn't act like a lightweight VM. In which case, you have to dabble one layer deeper just to understand enough to know why the abstraction of your layer isn't a perfect analogy. In which case, if you're looking for an expert in layer n, then basic knowledge of layer n - 1 might be a decent proxy for expertise
  - xelxebar 7y ago
    > To put forth a more substantive argument, everybody has a layer of abstraction they do not peek under. Sure. Though it's reasonable to want your level N developers to have some idea of what goes on at levels N-1 and perhaps N-2, cf. Law of Leaky Abstractions etc. It's similar to wanting your developers to be aware of their users' needs, which are level N+1 concerns.
    - pmichaud 7y ago
      Yeah, I wonder if there's an "optimal target" for the number of layers up and down you'd ideally be aware of. It has to be at least yours, and ones immediately above and below, but I see innovation coming from people with unusually keen insight into layers further away--eg. people making brilliant architectural decisions because they really, really know what the consumers of an api need and how those people think about the domain. Or vice versa, someone making something radically better or faster in a web app because they really get how the linux kernal is implemented. It seems like cases where that deep knowledge is an advantage are rare but also very high value. I wonder how the EV pans out, both for individuals and orgs.
- MuffinFlavored 7y ago
  > A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it. For what it's worth, one of the biggest "containerization" recommendations is to not run your database (example: Postgres) in a container, correct? Due to I/O performance decrease?
  - wmf 7y ago
    No. Docker volumes aka bind mounts have little or no overhead. You don't want to run a database in an ephemeral "cattle" container without some kind of HA because you'd lose data.
    - MuffinFlavored 7y ago
      docker run --rm \ --name postgresql \ -e POSTGRES_PASSWORD=postgres \ -d \ -p 5432:5432 \ --cpuset-cpus="0-1" \ --cpus 2.0 \ -m=1024m \ --mount type=bind,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data \ postgres:12.2 native results on a digitalocean VM: $ pgbench -c 100 -j 2 -T 60 postgres starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 17660 latency average = 342.761 ms tps = 291.748482 (including connections establishing) tps = 291.791293 (excluding connections establishing) in a docker container: starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 13014 latency average = 466.928 ms tps = 214.165822 (including connections establishing) tps = 214.199201 (excluding connections establishing) 214tps in docker, 291tps outside of docker 26% decrease, with a bind mount on ubuntu 18.04
      - takeda 7y ago
        Docker introduces all kinds of performance issues. I for example noticed higher latency even with just pgbouncer. It was very visible when running integration tests, which normally took 15 minutes, but as a container it was 45min - 1h.
- nunez 7y ago
  This is extremely widespread amongst the really large companies I've consulted for (with equally large development teams to boot). "Containers are VMs, but smaller and/or faster" is an extremely common school of thought. I would like to think that this viewpoint is dying somewhat, especially now that many large organizations have at least experimented with orchestrators like Kubernetes or ECS. I can't blame them, however. If you're a dev at a company where such misunderstandings are pervasive, and you're interested in trying to get your app running in this "Docker thing that you've heard of", you will, probably: - need to seek out a Linux dev server because you probably don't have admin rights on your machine, and getting Docker installed onto your machine in a way that doesn't suck is more trouble than its worth, - have engineering management that are being told that containers are like VMs, but smaller, likely from magazines or sales/pre-sales consultants, - Have to wait days/weeks to get SSH credentials to a dev server that has RHEL 7 on it, hoping that it has Docker (a thing you've heard of at this point, but don't really know much about it otherwise), - Have to wait even more time to get Docker installed for you on the dev server by a sysadmin that dislikes Docker because "it's insecure" or something, and - be constantly reminded that your job is shipping features before anything else, usually at the cost of learning new things that make those features more stable The point here is that environments like this are barely conducive for getting engineering done, let alone learning about new things. Moreover, the people that will do anything to scratch that itch usually find themselves out of companies like that eventually. It's a circle of suck for everyone involved. So when I introduce containers (which Docker is, by far, the most common runtime, so I introduce it as "Docker" to avoid confusion) to someone who doesn't know what cgroups or namespaces are, or someone who responds with something about containers being VMs or whatever, I happily meet them where they are at and do my best to show them otherwise.
- fulafel 7y ago
  Docker usage of cgroups is optional and it's not really an integral part of containers. Contrast to people using cgroups without containers too (unlike namespaces).
- cortesoft 7y ago
  Part of it might be that many developers target linux but code on a mac... and on a mac, docker containers DO run in a VM.
- Thaxll 7y ago
  Overhead is close to 0.
- tyingq 7y ago
  Have them look at bocker (docker-like in ~100 lines of bash). It makes it very clear what docker is, and isn't. https://github.com/p8952/bocker https://github.com/p8952/bocker Specifically, the bocker_run function: https://github.com/p8952/bocker/blob/master/bocker#L61 https://github.com/p8952/bocker/blob/master/bocker#L61
  - hadsed 7y ago
    so what's the gap here with docker? what incorrect assumptions would i make from assuming this as a model for containers, if anyone knows
    - cle 7y ago
      The gap is that it's skipping half of what makes Docker powerful, and what isn't really discussed in this conversation at all either. Distribution. E.g. see this line of code: https://github.com/p8952/bocker/blob/master/bocker#L25 https://github.com/p8952/bocker/blob/master/bocker#L25 "But I thought bocker implements Docker?" It doesn't, it only attempts to implement the "Docker daemon" part, and piggy-backs on Docker registries for image distribution. This is a huge part of the power of Docker, and why Docker isn't "just cgroups and namespaces". Cgroups and namespaces are a critical element, but Docker is much more than that too--it's also a set of standards for distributing and administering configuration+data for cgroups and namespaces.
      - barbecue_sauce 7y ago
        So many people don't seem to understand that artifact distribution is the compelling feature of Docker.
        imtringued 7y ago
        And if you want to have artifacts you also need a way to build them... Running containers is just one third of the features that docker offers.
- mav3rick 7y ago
  A process in a namespace is running just like another process being managed by your kernel. Based on how you set up networking, you may face an extra hop to get packets. I don't know what other scalability issues will be there, it's literally a process running similar to other processors. Can you shed light on some of these, maybe I haven't encountered these in my day to day ? (Please note I am not talking about containers running in VMs, which apparently Docker does now).
- dehrmann 7y ago
  > like VMs, but lighter This is both very right and very wrong.
- didibus 7y ago
  I don't know, container is an abstract idea, and that's all. Can you run apps within a contained OS environment? LXC is one way to do so, runc is another way to do so, docker is a third way to do so, all for Linux. Now if you took some other OS, there'd be different solutions, each with slightly different details and thus properties, but same idea. I mean, do you ask people what SQL is? And get frustrated if they don't start talking about MySQL specific details like InnoDB and what not? Don't know, I feel I can't agree with you, do devs feel there's less magic involved in VMs? Honestly, I have less idea what VMs are built on top of than I do for containers.
  - peteretep 7y ago
    > container is an abstract idea In this context, it's not, it's specifically referring to a process running in a cgroup. > LXC is one way to do so, runc is another way to do so, docker is a third way to do so Docker is a suite of tools for managing running LXC or runc, both of which set up processes running under cgroups. > Now if you took some other OS, there'd be different solutions, each with slightly different details and thus properties, but same idea You should read the article.
    - zodiac 7y ago
      "container" can mean other things in other contexts though, e.g. I'm running a configuration of Docker on my machine right now such that the CLI talks about "containers" but it is actually running VirtualBox in the background
- BiteCode_dev 7y ago
  With the new dev ops crave, we expect people to understand backend dev, and front end dev and design and sysadmin and networking and project management and infra and product ownership. Not to mention be proficient with tools related to those things. The result is not people getting experts at all those things, but getting capable of producing something with all those things. Obviously, to do so, people must take shortcut, and container ~= docker ~= "like VMs, but lighter" is good enough as a simplification for producing something. Now there is something to be said about the value, quality, durability and ethics of what is produced. But that's a choice companies make.
- ailideex 7y ago
  > A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. And those people should stop because that would be inaccurate. More specifically, a "container" can be any of the following: - Someone who contains; something that contains. An item in which objects, materials or data can be stored or transported. - (transport) A very large, typically metal, box used for transporting goods. - (by extension) Someone who holds people in their seats or in a (reasonably) calm state. - (computing) A file format that can hold various types of data. - (object-oriented programming) An abstract data type whose instances are collections of other objects. - (computing, graphical user interface) Any user interface component that can hold further (child) components. ... If we are talking about a Docker Container (here Docker Container is a proper noun), on the other hand - then clearly you are still wrong - as Docker have worked at various times on Windows and FreeBSD. Now trying to grasp at even more straws to find some the "container" definition of the gaps that may give some justification to your claims, we can look at the other proper noun "OCI Container" ... but alas ... https://www.opencontainers.org/faq https://www.opencontainers.org/faq > Will the runtime and image format specs support multiple platforms? > Yes. For example, take a look at the runtime-specification configuration where it mentions example Linux, Windows and Solaris configurations. There are also multiple implementations of the runtime-specification that you can take a look at. It seems then that you are just wrong. Plain and simple. I'm seriously concerned with whoever is having you interview candidates.
- kqr 7y ago
  > A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. Linux namespaces and cgroups ARE first class objects. NOT containers. Wow. I have always wondered in which cases one would use "containers". I have asked so many Docker enthusiasts, "Why would I use containers instead of the strong process separation functionality that's built into the operating system, like users, cgroups, SELinux, etc?" The answer has freakishly invariably been "Containers are lighter than virtual machines. Virtual machines use up all your resources. Containers can run just a single application, where virtual machines must emulate a whole operating system." Now you're probably going, "How is that an answer to the question?" And I know! It's not. These Docker enthusiasts have been impossible to get an answer out of. They have appeared like they're just parroting whatever they've heard from someone else as long as it seemed vaguely related to the question. Now that I finally have an answer to my original question, it all makes sense again. And I'm inclined to agree that if you're stuck in the wrong bubble, we're both stuck in the same bubble.
  - marcus_holmes 7y ago
    I figure Docker wraps stuff up into a nice "thing" that you can use, with documentation, a logo, and mindspace. You can put "Docker" on your CV and there's some hope that a recruiter will know what that means.
  - Jnr 7y ago
    I started using Linux containers long time ago and I have used it to achieve compatibility and ease of use. Before LXC was introduced it was somewhat painful to manage multiple environments using chroot, managing network, etc. But running the same software in a single environment wasn't always easy. You had to take care of different software versions. And it wasn't uncommon for things to break frequently because of that. While things like python venv, ruby rvm, etc. helped dealing with it, there was no universal tool for whole filesystem environments besides virtualization. When LXC came out, I started using it for everything. Nowadays I use LXD and sometimes Docker and it is so nice and requires minimal effort. I know that without those tools it would be very inconvenient to manage my own servers. I have separate auto-updating containers for everything and if one thing breaks, it doesn't take everything down with it. And when everything is contained and each system has minimal set of packages set up, over the years rarely anything ever breaks. And let's not forget that these Linux features also enabled the universal packages (flatpaks, snaps, etc.) which make it easier for Desktop users to get up to date software easily. Of course I know that it is not virtualization. But why do people say "containers are not really containers"? It still contains different environments. No one said it is about containing memory or something else.
- fluffything 7y ago
  I think how you are asking candidates the question might be unfortunate. A FreeBSD, MacOSX, or Windows dev that know their OSes might never tell you "namespace + cgroup". Hell if you try to explain containers to 99% of the world programmers by saying "namespace + cgroup", I'd bet you that 0% of them will understand what you mean. Instead, if you tell them that's "like a VM, but faster, because it "reuses" the host's kernel", you might be able to reach a substantial amount of them, and that level of abstraction would be enough for most of them to do something useful with containers. Maybe the question you should be asking your candidates is: "How would you implement a Docker-like container app on Linux?". That's a question that specifically constraints the level of abstraction of the answer to at least one layer below Docker, and also specifies that you are interested on hearing the details for Linux.
deleted 7y ago
[deleted]
moonchild 7y ago
I'm a bit disappointed it didn't go into detail into the way jails differ from zones. VMs I understand, but it seemed like the main point of the post was to distinguish containers from the other three.
- deleted 7y ago
  [deleted]
- nickik 7y ago
  All the detail you could possible want: https://www.youtube.com/watch?v=hgN8pCMLI2U https://www.youtube.com/watch?v=hgN8pCMLI2U
nfoz 7y ago
So.... are any or all of these what you would call a process "sandbox"? Do operating systems make it easy to sandbox an application from causing harm to the system? What more could be done to make that a natural, first-class feature? Like, let's say you found some binary and you don't know what it does, and don't want it to mess anything up. Is there an easy way to run it securely? Why not? And how about giving it specific, opt-in permissions, like limited network or filesystem access.
dirtydroog 7y ago
For my workload I've struggled to see the advantage containers would give me. Maybe someone here can convince me, rather than the current justification of 'docker all the things'. We have servers, they handle a lot of traffic. It's the only thing running on the machines and takes over all the resources of the machine. It will need all the RAM, and all 16 vCPUs are at ~90%. It's running on GCP. To rollout we have a jenkins job that builds a tag, creates a package (dpkg) and builds an image. There's another jenkins job that deploys the new image to all regions and starts the update process, autoscaling and all that. Can containers help me here?
- pbecotte 7y ago
  If you already have all of that working, why would you change? Containers are valuable for a couple things- 1. Packaging and distribution- it's very easy to set up a known good filesystem using docker images and reuse that. There are other solutions- dpkg plus ansible would be an example. 2. Standardized control- all apps using 'docker run' vs a mix of systemd and shell scripts can simplify things. 3. Let's you tie into higher level orchestration layers like k8s where you can view your app instances as a single thing. There are other solutions here as well. 4. Can use the same image on dev machines as prod instead of needing two parallel setup schemes. If you already are happy with your infra, certainly don't change it. I think once you know containers they are a convenient solution to those problems, but if stuff is setup they already missed their shot.
codeape 7y ago
I do not understand docker on windows. If I understand correctly, when I run a docker image on Linux then the dockerized processes's syscalls are all executed by the host kernel (since - again if I understand correctly - the dockerized process executes more or less like a normal process, just in isolated process and filesystem namespace). Is this correct? But how does docker on windows work?
- codeape 7y ago
  Found this: https://stackoverflow.com/questions/41550727/how-does-docker-for-windows-run-linux-containers https://stackoverflow.com/questions/41550727/how-does-docker... And: https://dockercon.docker.com/watch/U7Bxp66uKmemZssjCTyXkm https://dockercon.docker.com/watch/U7Bxp66uKmemZssjCTyXkm
deg4uss3r 7y ago
My only problem with this article is there is no such thing as "Legos". Jess is brilliant and explains things super well here.

351 points by gullyfur 7y ago

Note this is from 2017. Previous discussion: https://news.ycombinator.com/item?id=13982620 https://news.ycombinator.com/item?id=13982620

dang 7y ago
Year added. Thanks!

outworlder 7y ago

> A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. Linux namespaces and cgroups ARE first class objects. NOT containers. Amen. Somewhat tangential note: most developers I have met do not understand what a 'container' is. There's an aura of magic and mystique around them. And a heavy emphasis on Docker. A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it. No mention to storage, or how networking would be impacted, just CPU. That's usually said without measuring the actual performance first. When I press further, what I most commonly get is the sense that they believe that containers are "like VMs, but lighter"(also, I've been told that, literally, a few times, specially when interviewing candidates). To this day, I've heard CGroups being mentioned only once. I wonder if I'm stuck in the wrong bubble, or if this is widespread.

pojzon 7y ago
This is widespread unfortunately. Developers changed to „users” and no longer pursue the details of solutions. Im not gonna say its the dominant behaviour in the field right now, but I see it more and more often on various experience levels.
- jagged-chisel 7y ago
  An experienced software engineer (yeah, developer) has experience engineering software. It’s been less than 10yrs since the advent of containerized deployments, and the space has been fraught with change nearly on par with the front end JavaScript ecosystem. Might as well just stick to writing code. OK, that’s the perception of my own peers, but I assume it scales. DevOps is a recent advent, too, and sounds to me like it should be populated with folks who can participate in development and operations. Most developers I’ve ever known aren’t interested in operations.
  - dielectrikboog 7y ago
    SRE was born of putting software engineers to work building operational software and automation tailored to an organization and application. In contrast, no matter what anyone says, DevOps was objectively born of replacing the operations discipline and career track with a poorly-understood tool economy and ongoing opex to a cloud provider. As you say, typical JavaScript engineers can’t be bothered to understand network capacity planning yet feel they are more qualified to take their application to production by deferring all decisions to cloud providers. Who all employ SREs/PEs, not DevOps Engineers, by the way, and there is a big distinction. We have people who can handle the operational stuff. They’re called systems administrators, network engineers, yes, even SREs, and other folks who are really good at understanding how computers and the Internet actually work, and a webdev bootcamp gives zero context into exactly what they do. None. Then, your ten-head startup suddenly scales to needing a physical footprint because it will literally save 80% of opex, and all your DevOps Engineers say “but why? There’s AWS,” and you’re in Medium hell for weeks arguing about it. Apropos, if I interview you and find you have written a thoughtpiece on Medium about how “sysadmin is evolving” and it’s “time for the gray beards to learn code,” you do not get a call back. That has actually happened, and no, sysadmin is not evolving. I know staff-level “full stack engineers” who can’t tell me what an ASN is. The motions in the industry have merely made those people more in demand at a few companies and served to cement their position as “where computing gets done”. Expect serious, existential operational threats and breaches to rise dramatically as DevOps continues to “win,” and consider it a smart strategic move to avoid DevOps culture in the long term. If you write a req called “DevOps Engineer,” I don’t even know what to say to you.
wmf 7y ago
People love to bring this up, but if Linux did have first-class containers, how would the developer's experience be different?
- lmm 7y ago
  There would maybe be more consistency. E.g. currently if I say an application is running in a container, do you expect there is virtual networking in place, or not?
  - wmf 7y ago
    Jails and Zones probably also have bridged, routed, and NATed modes so I'm not sure that example is that useful. It's true that networking is different in Docker vs. k8s but there are valid reasons for it.
- cyphar 7y ago
  All system programs could operate and manage the containers running on the system, meaning you can use your existing knowledge to manage a bunch of containers. For instance, you could run your package manager across all containers to see if they have packages with known CVEs. Or manage the filesystems of all containers on the system (the usefulness of this is only clear with filesystems like ZFS and btrfs). This is effectively what you can do with Solaris Zones. These kinds of improvements to usability aren't as sexy now that everyone is really excited about Kubernetes (where you blow away containers at whim) but it is useful for the more traditional container usecases that Zones and Jails. LXC (and LXD) is probably the one Linux container runtime that is closest to that original container model. There's also a very big security argument -- it is (speaking as a maintainer of runc) very hard to construct a secure container on Linux. There are dozens of different facilities you need to individually configure in the right order, with hundreds of individual knobs that some users might want to disable or quirks you need to work around. It's basically impossible to implement a sane and secure container runtime without having read and understood the kernel code which implements the interfaces you're using. If containers were an in-kernel primitive then all of the security design would rest in one single codebase, and all of the policies would be defined by one entity (the kernel).
mulmen 7y ago
I think it is widespread because containers are (seemingly) marketed as being some kind of magic. The impression I get is that the benefit to containers is that you don't have to think about them. This may be more a product of Docker but I think containers and Docker have become synonymous. I'm not sure this is a fair comparison but that is my impression. I could be in a bubble too.
- nurpax 7y ago
  I’d like to think so too. It’s hard to actually understand how Docker works by reading their documentation. Everything looks so simple.. and it magically runs on macOS and Windows too. No wonder people think it’s some sort of a VM.
  - fapjacks 7y ago
    Technically, containers running on MacOS and Windows are a (Linux) VM, under the hood. Now, Microsoft put in a lot of work to support something like namespaces in the Windows kernel, so that it's now possible to run "Windows-native containers" for Windows software. But both Windows and MacOS still use a Linux VM under the hood to run Linux containers. That being said, I'm not familiar with the details of Windows containers because I don't use any Windows software and therefore don't have a reason to run any Windows containers. If you are interested in how containers work at a lower level, you might get more out of the documentation for containerd and runc, which are the underlying container runtimes.
  - ignoramous 7y ago
    > No wonder people think it’s some sort of a VM. From what I read, if one packages a Linux container for Windows, Docker does then run it in HyperV? https://stackoverflow.com/questions/52164563/what-is-the-docker-cross-platform-architecture https://stackoverflow.com/questions/52164563/what-is-the-doc...
- takeda 7y ago
  It is product of docker, if you would deploy applications by using namespaces and cgroups directly it is very likely you would see things the same way the author does.
erjiang 7y ago
This is true for many new waves of popular technologies. 1) A new technology or method becomes popular. 2) Developers find new advantages in using the technology. 3) Understanding of tech and original advantage is somewhat lost. For example: containers are now widely used as part of a scriptable application build process, e.g. the Dockerfile. There are probably many developers out there who care about this and not about how containers are run and how they interact with the kernel. And for their use cases, that is probably the thing that matters most anyways.
- fiddlerwoaroof 7y ago
  A down side is that people feel like they have to bundle an entire Linux rootfs because they think of a container as a lightweight vm: if they thought of it as a os process running inside various namespaces, they might be more inclined to only ship what they actually need.
loeg 7y ago
> Somewhat tangential note: most developers I have met do not understand what a 'container' is. The problem is at least in part that the term is basically meaningless. It's too broad and flexible to be descriptive. As you quoted: > A “container” is just a term people use to describe a combination of Linux namespaces and cgroups.
- sanderjd 7y ago
  I'm more sympathetic to the mainstream usage. People are (attempting to) use an abstraction: a "container" is an isolated process runtime environment without virtualization overhead. That abstraction seems useful to me. Ideally it would be usable without too much leakiness, in which case its users would not need to be aware of implementation details like cgroups and namespaces. In practice, all abstractions are leaky to some degree and effective use of an abstraction often eventually requires a more sophisticated understanding of the details beneath the veil of the abstraction. But that doesn't mean the abstraction is totally useless or completely a mirage or anything, it's just a leaky abstraction like all others. If you say that a container is not a first class object but cgroups and namespaces are, I can just as easily say that cgroups and namespaces aren't first class objects, they are just terms people use to describe a combination of system calls. It's just abstractions with different amounts of leakage the whole way down.
tasogare 7y ago
The "containers are just XX Linux technology" comes regularly in the comments but it’s untrue: Windows containers are obviously not based on any Linux tech proper. Also the overhead intuition exists for a reason: on both macOS and Windows when Linux containers are used, there is actually a whole VM running Linux underneath. And Windows containers come in two flavors, one being Hyper-V based, so again a VM tech comes in play. So there are technical reasons why containers are "misunderstood", it’s because most people don’t run Linux natively, and on their stack containers are more than just cgroups and namespaces.
matharmin 7y ago
I feel nowadays containers generally refer to the concept, and cgroups and namespaces are the implementation details of a specific container runtime. These are very important implementation details for security and performance, but it doesn't fundamentally impact how you structure your containerized application. You can take the same container image, and run using Docker, Firecracker, gVisor, or many other container runtimes. Some of them are exactly "like a VM, but lighter".
- cmckn 7y ago
  Agreed. The post feels a bit pedantic; I don't know any dev doing "cool things" with the underlying namespaces/cgroups. They're just using Docker. De-mystifying containers has value, but so does the abstraction.
  - mav3rick 7y ago
    The abstraction muddles the performance, security etc. impact of these two models. Not knowing them is going to be bad in the long run. Not everyone is a web dev.
  - mav3rick 7y ago
    And I'm a dev "doing cool things" with namespaces / cgroups.
  - moomin 7y ago
    Plenty of them using those features, if they’re using Kubernetes, Docker Swarm or especially Istio. They might not know they are, but that’s besides the point.
    - jdmichal 7y ago
      > They might not know they are, but that’s besides the point. I actually thought that was exactly the point here...
  - bregma 7y ago
    Your personal ignorance makes a poor argument. Not only do I know several devs doing cool things with namespaces and cgroup, but I myself have done cool things with namespaces and cgroups. I played with Docker briefly, but it has no real practical application in my line of work.
taf2 7y ago
Thanks I’d have fallen into your category of developers because in large part I never bothered with containers since we have everything running on vms and we’ve already isolated things so I’ve had only partial interest in exploring them ... but now that I know in linux it’s CGroups and namespaces that helps a lot in understanding thanks
jjtheblunt 7y ago
widespread in my experience; it reminds me to users of npm not seeming to understand what node is, for example.
k__ 7y ago
Doesn't surprise me if I think about all the people who suddendly need a K8s cluster...
CBLT 7y ago
> To this day, I've heard CGroups being mentioned only once. See https://www.kernel.org/doc/Documentation/cgroup-v2.txt https://www.kernel.org/doc/Documentation/cgroup-v2.txt > "cgroup" stands for "control group" and is never capitalized. The singular form is used to designate the whole feature and also as a qualifier as in "cgroup controllers". When explicitly referring to multiple individual control groups, the plural form "cgroups" is used. To this day, I've heard cgroup mentioned only once... To put forth a more substantive argument, everybody has a layer of abstraction they do not peek under. You interviewed people that didn't peek under container. You went a layer deeper, but never peeked at the source tree to learn what cgroup really is. Does it really feel that much better to be one level above others?
- Harlekuin 7y ago
  > everybody has a layer of abstraction they do not peek under Reminds of this xkcd: https://xkcd.com/435/ https://xkcd.com/435/ The beauty of programming is abstraction - you write a function to do one thing, do it well, and then you can abstract that concept away and reuse that abstraction. Although it's only an abstraction - a container is "like" a lightweight VM, and you can use it like that until it doesn't act like a lightweight VM. In which case, you have to dabble one layer deeper just to understand enough to know why the abstraction of your layer isn't a perfect analogy. In which case, if you're looking for an expert in layer n, then basic knowledge of layer n - 1 might be a decent proxy for expertise
- xelxebar 7y ago
  > To put forth a more substantive argument, everybody has a layer of abstraction they do not peek under. Sure. Though it's reasonable to want your level N developers to have some idea of what goes on at levels N-1 and perhaps N-2, cf. Law of Leaky Abstractions etc. It's similar to wanting your developers to be aware of their users' needs, which are level N+1 concerns.
  - pmichaud 7y ago
    Yeah, I wonder if there's an "optimal target" for the number of layers up and down you'd ideally be aware of. It has to be at least yours, and ones immediately above and below, but I see innovation coming from people with unusually keen insight into layers further away--eg. people making brilliant architectural decisions because they really, really know what the consumers of an api need and how those people think about the domain. Or vice versa, someone making something radically better or faster in a web app because they really get how the linux kernal is implemented. It seems like cases where that deep knowledge is an advantage are rare but also very high value. I wonder how the EV pans out, both for individuals and orgs.
MuffinFlavored 7y ago
> A sizable fraction will be concerned about 'container overhead' (and "scalability issues") when asked to move workloads to containers. They are usually not able to explain what the overhead would be, and what could potentially be causing it. For what it's worth, one of the biggest "containerization" recommendations is to not run your database (example: Postgres) in a container, correct? Due to I/O performance decrease?
- wmf 7y ago
  No. Docker volumes aka bind mounts have little or no overhead. You don't want to run a database in an ephemeral "cattle" container without some kind of HA because you'd lose data.
  - MuffinFlavored 7y ago
    docker run --rm \ --name postgresql \ -e POSTGRES_PASSWORD=postgres \ -d \ -p 5432:5432 \ --cpuset-cpus="0-1" \ --cpus 2.0 \ -m=1024m \ --mount type=bind,source=$HOME/docker/volumes/postgres,target=/var/lib/postgresql/data \ postgres:12.2 native results on a digitalocean VM: $ pgbench -c 100 -j 2 -T 60 postgres starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 17660 latency average = 342.761 ms tps = 291.748482 (including connections establishing) tps = 291.791293 (excluding connections establishing) in a docker container: starting vacuum...end. transaction type: <builtin: TPC-B (sort of)> scaling factor: 1 query mode: simple number of clients: 100 number of threads: 2 duration: 60 s number of transactions actually processed: 13014 latency average = 466.928 ms tps = 214.165822 (including connections establishing) tps = 214.199201 (excluding connections establishing) 214tps in docker, 291tps outside of docker 26% decrease, with a bind mount on ubuntu 18.04
    - takeda 7y ago
      Docker introduces all kinds of performance issues. I for example noticed higher latency even with just pgbouncer. It was very visible when running integration tests, which normally took 15 minutes, but as a container it was 45min - 1h.
nunez 7y ago
This is extremely widespread amongst the really large companies I've consulted for (with equally large development teams to boot). "Containers are VMs, but smaller and/or faster" is an extremely common school of thought. I would like to think that this viewpoint is dying somewhat, especially now that many large organizations have at least experimented with orchestrators like Kubernetes or ECS. I can't blame them, however. If you're a dev at a company where such misunderstandings are pervasive, and you're interested in trying to get your app running in this "Docker thing that you've heard of", you will, probably: - need to seek out a Linux dev server because you probably don't have admin rights on your machine, and getting Docker installed onto your machine in a way that doesn't suck is more trouble than its worth, - have engineering management that are being told that containers are like VMs, but smaller, likely from magazines or sales/pre-sales consultants, - Have to wait days/weeks to get SSH credentials to a dev server that has RHEL 7 on it, hoping that it has Docker (a thing you've heard of at this point, but don't really know much about it otherwise), - Have to wait even more time to get Docker installed for you on the dev server by a sysadmin that dislikes Docker because "it's insecure" or something, and - be constantly reminded that your job is shipping features before anything else, usually at the cost of learning new things that make those features more stable The point here is that environments like this are barely conducive for getting engineering done, let alone learning about new things. Moreover, the people that will do anything to scratch that itch usually find themselves out of companies like that eventually. It's a circle of suck for everyone involved. So when I introduce containers (which Docker is, by far, the most common runtime, so I introduce it as "Docker" to avoid confusion) to someone who doesn't know what cgroups or namespaces are, or someone who responds with something about containers being VMs or whatever, I happily meet them where they are at and do my best to show them otherwise.
fulafel 7y ago
Docker usage of cgroups is optional and it's not really an integral part of containers. Contrast to people using cgroups without containers too (unlike namespaces).
cortesoft 7y ago
Part of it might be that many developers target linux but code on a mac... and on a mac, docker containers DO run in a VM.
Thaxll 7y ago
Overhead is close to 0.
tyingq 7y ago
Have them look at bocker (docker-like in ~100 lines of bash). It makes it very clear what docker is, and isn't. https://github.com/p8952/bocker https://github.com/p8952/bocker Specifically, the bocker_run function: https://github.com/p8952/bocker/blob/master/bocker#L61 https://github.com/p8952/bocker/blob/master/bocker#L61
- hadsed 7y ago
  so what's the gap here with docker? what incorrect assumptions would i make from assuming this as a model for containers, if anyone knows
  - cle 7y ago
    The gap is that it's skipping half of what makes Docker powerful, and what isn't really discussed in this conversation at all either. Distribution. E.g. see this line of code: https://github.com/p8952/bocker/blob/master/bocker#L25 https://github.com/p8952/bocker/blob/master/bocker#L25 "But I thought bocker implements Docker?" It doesn't, it only attempts to implement the "Docker daemon" part, and piggy-backs on Docker registries for image distribution. This is a huge part of the power of Docker, and why Docker isn't "just cgroups and namespaces". Cgroups and namespaces are a critical element, but Docker is much more than that too--it's also a set of standards for distributing and administering configuration+data for cgroups and namespaces.
    - barbecue_sauce 7y ago
      So many people don't seem to understand that artifact distribution is the compelling feature of Docker.
      - imtringued 7y ago
        And if you want to have artifacts you also need a way to build them... Running containers is just one third of the features that docker offers.
mav3rick 7y ago
A process in a namespace is running just like another process being managed by your kernel. Based on how you set up networking, you may face an extra hop to get packets. I don't know what other scalability issues will be there, it's literally a process running similar to other processors. Can you shed light on some of these, maybe I haven't encountered these in my day to day ? (Please note I am not talking about containers running in VMs, which apparently Docker does now).
dehrmann 7y ago
> like VMs, but lighter This is both very right and very wrong.
didibus 7y ago
I don't know, container is an abstract idea, and that's all. Can you run apps within a contained OS environment? LXC is one way to do so, runc is another way to do so, docker is a third way to do so, all for Linux. Now if you took some other OS, there'd be different solutions, each with slightly different details and thus properties, but same idea. I mean, do you ask people what SQL is? And get frustrated if they don't start talking about MySQL specific details like InnoDB and what not? Don't know, I feel I can't agree with you, do devs feel there's less magic involved in VMs? Honestly, I have less idea what VMs are built on top of than I do for containers.
- peteretep 7y ago
  > container is an abstract idea In this context, it's not, it's specifically referring to a process running in a cgroup. > LXC is one way to do so, runc is another way to do so, docker is a third way to do so Docker is a suite of tools for managing running LXC or runc, both of which set up processes running under cgroups. > Now if you took some other OS, there'd be different solutions, each with slightly different details and thus properties, but same idea You should read the article.
  - zodiac 7y ago
    "container" can mean other things in other contexts though, e.g. I'm running a configuration of Docker on my machine right now such that the CLI talks about "containers" but it is actually running VirtualBox in the background
BiteCode_dev 7y ago
With the new dev ops crave, we expect people to understand backend dev, and front end dev and design and sysadmin and networking and project management and infra and product ownership. Not to mention be proficient with tools related to those things. The result is not people getting experts at all those things, but getting capable of producing something with all those things. Obviously, to do so, people must take shortcut, and container ~= docker ~= "like VMs, but lighter" is good enough as a simplification for producing something. Now there is something to be said about the value, quality, durability and ethics of what is produced. But that's a choice companies make.
ailideex 7y ago
> A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. And those people should stop because that would be inaccurate. More specifically, a "container" can be any of the following: - Someone who contains; something that contains. An item in which objects, materials or data can be stored or transported. - (transport) A very large, typically metal, box used for transporting goods. - (by extension) Someone who holds people in their seats or in a (reasonably) calm state. - (computing) A file format that can hold various types of data. - (object-oriented programming) An abstract data type whose instances are collections of other objects. - (computing, graphical user interface) Any user interface component that can hold further (child) components. ... If we are talking about a Docker Container (here Docker Container is a proper noun), on the other hand - then clearly you are still wrong - as Docker have worked at various times on Windows and FreeBSD. Now trying to grasp at even more straws to find some the "container" definition of the gaps that may give some justification to your claims, we can look at the other proper noun "OCI Container" ... but alas ... https://www.opencontainers.org/faq https://www.opencontainers.org/faq > Will the runtime and image format specs support multiple platforms? > Yes. For example, take a look at the runtime-specification configuration where it mentions example Linux, Windows and Solaris configurations. There are also multiple implementations of the runtime-specification that you can take a look at. It seems then that you are just wrong. Plain and simple. I'm seriously concerned with whoever is having you interview candidates.
kqr 7y ago
> A “container” is just a term people use to describe a combination of Linux namespaces and cgroups. Linux namespaces and cgroups ARE first class objects. NOT containers. Wow. I have always wondered in which cases one would use "containers". I have asked so many Docker enthusiasts, "Why would I use containers instead of the strong process separation functionality that's built into the operating system, like users, cgroups, SELinux, etc?" The answer has freakishly invariably been "Containers are lighter than virtual machines. Virtual machines use up all your resources. Containers can run just a single application, where virtual machines must emulate a whole operating system." Now you're probably going, "How is that an answer to the question?" And I know! It's not. These Docker enthusiasts have been impossible to get an answer out of. They have appeared like they're just parroting whatever they've heard from someone else as long as it seemed vaguely related to the question. Now that I finally have an answer to my original question, it all makes sense again. And I'm inclined to agree that if you're stuck in the wrong bubble, we're both stuck in the same bubble.
- marcus_holmes 7y ago
  I figure Docker wraps stuff up into a nice "thing" that you can use, with documentation, a logo, and mindspace. You can put "Docker" on your CV and there's some hope that a recruiter will know what that means.
- Jnr 7y ago
  I started using Linux containers long time ago and I have used it to achieve compatibility and ease of use. Before LXC was introduced it was somewhat painful to manage multiple environments using chroot, managing network, etc. But running the same software in a single environment wasn't always easy. You had to take care of different software versions. And it wasn't uncommon for things to break frequently because of that. While things like python venv, ruby rvm, etc. helped dealing with it, there was no universal tool for whole filesystem environments besides virtualization. When LXC came out, I started using it for everything. Nowadays I use LXD and sometimes Docker and it is so nice and requires minimal effort. I know that without those tools it would be very inconvenient to manage my own servers. I have separate auto-updating containers for everything and if one thing breaks, it doesn't take everything down with it. And when everything is contained and each system has minimal set of packages set up, over the years rarely anything ever breaks. And let's not forget that these Linux features also enabled the universal packages (flatpaks, snaps, etc.) which make it easier for Desktop users to get up to date software easily. Of course I know that it is not virtualization. But why do people say "containers are not really containers"? It still contains different environments. No one said it is about containing memory or something else.
fluffything 7y ago
I think how you are asking candidates the question might be unfortunate. A FreeBSD, MacOSX, or Windows dev that know their OSes might never tell you "namespace + cgroup". Hell if you try to explain containers to 99% of the world programmers by saying "namespace + cgroup", I'd bet you that 0% of them will understand what you mean. Instead, if you tell them that's "like a VM, but faster, because it "reuses" the host's kernel", you might be able to reach a substantial amount of them, and that level of abstraction would be enough for most of them to do something useful with containers. Maybe the question you should be asking your candidates is: "How would you implement a Docker-like container app on Linux?". That's a question that specifically constraints the level of abstraction of the answer to at least one layer below Docker, and also specifies that you are interested on hearing the details for Linux.

deleted 7y ago

[deleted]

moonchild 7y ago

I'm a bit disappointed it didn't go into detail into the way jails differ from zones. VMs I understand, but it seemed like the main point of the post was to distinguish containers from the other three.

deleted 7y ago
[deleted]
nickik 7y ago
All the detail you could possible want: https://www.youtube.com/watch?v=hgN8pCMLI2U https://www.youtube.com/watch?v=hgN8pCMLI2U

nfoz 7y ago

So.... are any or all of these what you would call a process "sandbox"? Do operating systems make it easy to sandbox an application from causing harm to the system? What more could be done to make that a natural, first-class feature? Like, let's say you found some binary and you don't know what it does, and don't want it to mess anything up. Is there an easy way to run it securely? Why not? And how about giving it specific, opt-in permissions, like limited network or filesystem access.

dirtydroog 7y ago

For my workload I've struggled to see the advantage containers would give me. Maybe someone here can convince me, rather than the current justification of 'docker all the things'. We have servers, they handle a lot of traffic. It's the only thing running on the machines and takes over all the resources of the machine. It will need all the RAM, and all 16 vCPUs are at ~90%. It's running on GCP. To rollout we have a jenkins job that builds a tag, creates a package (dpkg) and builds an image. There's another jenkins job that deploys the new image to all regions and starts the update process, autoscaling and all that. Can containers help me here?

pbecotte 7y ago
If you already have all of that working, why would you change? Containers are valuable for a couple things- 1. Packaging and distribution- it's very easy to set up a known good filesystem using docker images and reuse that. There are other solutions- dpkg plus ansible would be an example. 2. Standardized control- all apps using 'docker run' vs a mix of systemd and shell scripts can simplify things. 3. Let's you tie into higher level orchestration layers like k8s where you can view your app instances as a single thing. There are other solutions here as well. 4. Can use the same image on dev machines as prod instead of needing two parallel setup schemes. If you already are happy with your infra, certainly don't change it. I think once you know containers they are a convenient solution to those problems, but if stuff is setup they already missed their shot.

codeape 7y ago

I do not understand docker on windows. If I understand correctly, when I run a docker image on Linux then the dockerized processes's syscalls are all executed by the host kernel (since - again if I understand correctly - the dockerized process executes more or less like a normal process, just in isolated process and filesystem namespace). Is this correct? But how does docker on windows work?

codeape 7y ago
Found this: https://stackoverflow.com/questions/41550727/how-does-docker-for-windows-run-linux-containers https://stackoverflow.com/questions/41550727/how-does-docker... And: https://dockercon.docker.com/watch/U7Bxp66uKmemZssjCTyXkm https://dockercon.docker.com/watch/U7Bxp66uKmemZssjCTyXkm

deg4uss3r 7y ago

My only problem with this article is there is no such thing as "Legos". Jess is brilliant and explains things super well here.