4 ms·

Folks interested in this should also take note of the projects that target Xen [1,2 are ones I'm aware of]. Startup times that can be tuned to be on the order

by amirmc 13y ago

Folks interested in this should also take note of the projects that target Xen [1,2 are ones I'm aware of]. Startup times that can be tuned to be on the order of 10s of milliseconds (see comment in Mirage paper). [1] Mirage - http://openmirage.org http://openmirage.org (paper at http://anil.recoil.org/papers/2013-asplos-mirage.pdf http://anil.recoil.org/papers/2013-asplos-mirage.pdf) [2] Zerg - http://zerg.erlangonxen.org http://zerg.erlangonxen.org

marshray 13y ago
I agree, these things are the future. Containers may be useful for single-tenant partitioning, but no one has managed to produce an OS kernel without an order of magnitude more local root security vulnerabilities than the HVM or PV virutualization systems.
- contingencies 13y ago
  Honestly, on Linux we can reach a similar point now by careful application of existing tools. For example, by whitelisting syscalls, devices, hardware and network resources and access patterns, in combination with containers. Most of the challenge is automating that process adequately, which is really nothing more than traditional code profiling meets existing policy-based kernel-level security toolkits meets containers meets development processes (ie. build/test/release). After ~3 years' with LXC, I believe that most of the attack surfaces are reasonably well understood and/or not required for most code to function. Sure, there will be kernel-level vulnerabilities, but they won't be exposed to such a broad range of userland applications. With security you also get so much more insight, power and predictability with regards to resource profiling than with paravirtualized solutions. Real kernel-level metrics versus load! Think of the impact for operations/capacity planning. So... short soundbite to summarize? Docker is a toy compared to where we are going.
  - shykes 13y ago
    Hi, I'm the author of Docker. I agree with everything you said about lxc, but I'm not sure how you reached the conclusion that docker is a toy. Since Docker is a portability layer on top of lxc, it automatically benefits from all the awesome work happening in the kernel. Or do you just use "toy" as a synonymous for "userland"?
    - contingencies 13y ago
      I mean no disrespect to docker! It's a great step, though has its limitations. I am talking conceptually about the whole direction the lightweight virt/app-level portability (relative infrastructure independence) thing is going. My point is that the impact that continuing developments in this general area are going to have on security, development and operations processes is probably extreme... and I believe seriously under-considered by the community at present.
      - shykes 13y ago
        Author's susceptibility aside :) You are totally right. Now that lightweight virt is broadly available on the server, we are going to see a very, very big change in how computing is done on the server, and all of today's tools (docker included) are going to look cute and primitive compared to what we'll have a in a couple years. I can't wait :)
        contingencies 13y ago
        I have been working in the same direction in private for my current employer, are near a solution encompassing all of the above, and may open source at some point. In terms of potential collaboration, what I would really like from docker is a single page, frequently revised summary page of features and efforts within the docker world. That would help me to get in touch with people to share info if we are facing similar challenges. Right now it seems I would have to spend a lot of time monitoring things; time which doesn't exist. (He says, off to a sitar lesson...)
        shykes 13y ago
        Yes, we need that. For now here is what we have: 1) An ad-hoc compilation of cool stuff maintained by Ken Cochrane: https://github.com/dotcloud/docker/issues/614 https://github.com/dotcloud/docker/issues/614 2) All sorts of interesting software projects distributed as docker containers: http://index.docker.io http://index.docker.io 3) A highly active IRC channel, #docker@freenode where many cool applications of docker are being discussed
contingencies 13y ago
I believe your two examples are actually two entirely different paradigms. The first one is based on the exokernel notion. A bit muddy, but it seems that nobody has ever used a pure exokernel successfully at scale in industry, presumably because it seems to call for the kernel exposing hardware directly to userland code whilst simultaneously managing multiple tenants. Basically, the idea seems like a sort of early and poorly articulated call to modern OS level resource management (eg. cgroups). Your example two, the zerg system, seems very similar to LXC. Other than the exoticism of Erlang, their only big feature vs. normal paravirtualization seems to be startup time, which they see as a groundbreaking change. Well, I regularly start whole systems in LXC in fractions of a second. See https://github.com/globalcitizen/lxc-gentoo https://github.com/globalcitizen/lxc-gentoo In summary, these two projects are not really any different to the topic under discussion, ie. LXC (~cgroups/namespaces). Well, except that unlike cgroups/namespaces they're not necessarily able to run everywhere Linux runs!
- vidarh 13y ago
  The Mirage link is confusing matters. Mirage appears to be an exo/micro kernel for running in a para-virtualized system. That's very different from an exo-kernel running on bare hardware. With Mirage, what the apps run on seems to be "para-virtualized bare metal", so rather than using the POSIX syscall interface as its ABI, it's using the paravirtualized "hardware" as its ABI. That makes Mirage very similar to Zerg - it's the same concept, only OCaml vs. Erlang. As for LXC, I agree LXC offers a lot of this, especially since you don't need to boot full Linux environments - you can spawn individual applications and fine-tune what isolation you need. Or you can even build applications that explicitly use cgroups themselves to isolate sub-processes. So their "spawn one Xen Dom-U per request" demo could similarly have been a "spawn one new process isolated in its own cgroups". But these systems are appealing to me in that they bring VM level isolation with startup performance and resource utilization more in line with container systems. That's particularly appealing if we start seeing cloud systems that can handle it. Imagine being able to create new EC2 instances at sub-second times...

by amirmc 13y ago

marshray 13y ago

I agree, these things are the future. Containers may be useful for single-tenant partitioning, but no one has managed to produce an OS kernel without an order of magnitude more local root security vulnerabilities than the HVM or PV virutualization systems.

contingencies 13y ago
Honestly, on Linux we can reach a similar point now by careful application of existing tools. For example, by whitelisting syscalls, devices, hardware and network resources and access patterns, in combination with containers. Most of the challenge is automating that process adequately, which is really nothing more than traditional code profiling meets existing policy-based kernel-level security toolkits meets containers meets development processes (ie. build/test/release). After ~3 years' with LXC, I believe that most of the attack surfaces are reasonably well understood and/or not required for most code to function. Sure, there will be kernel-level vulnerabilities, but they won't be exposed to such a broad range of userland applications. With security you also get so much more insight, power and predictability with regards to resource profiling than with paravirtualized solutions. Real kernel-level metrics versus load! Think of the impact for operations/capacity planning. So... short soundbite to summarize? Docker is a toy compared to where we are going.
- shykes 13y ago
  Hi, I'm the author of Docker. I agree with everything you said about lxc, but I'm not sure how you reached the conclusion that docker is a toy. Since Docker is a portability layer on top of lxc, it automatically benefits from all the awesome work happening in the kernel. Or do you just use "toy" as a synonymous for "userland"?
  - contingencies 13y ago
    I mean no disrespect to docker! It's a great step, though has its limitations. I am talking conceptually about the whole direction the lightweight virt/app-level portability (relative infrastructure independence) thing is going. My point is that the impact that continuing developments in this general area are going to have on security, development and operations processes is probably extreme... and I believe seriously under-considered by the community at present.
    - shykes 13y ago
      Author's susceptibility aside :) You are totally right. Now that lightweight virt is broadly available on the server, we are going to see a very, very big change in how computing is done on the server, and all of today's tools (docker included) are going to look cute and primitive compared to what we'll have a in a couple years. I can't wait :)
      - contingencies 13y ago
        I have been working in the same direction in private for my current employer, are near a solution encompassing all of the above, and may open source at some point. In terms of potential collaboration, what I would really like from docker is a single page, frequently revised summary page of features and efforts within the docker world. That would help me to get in touch with people to share info if we are facing similar challenges. Right now it seems I would have to spend a lot of time monitoring things; time which doesn't exist. (He says, off to a sitar lesson...)
        shykes 13y ago
        Yes, we need that. For now here is what we have: 1) An ad-hoc compilation of cool stuff maintained by Ken Cochrane: https://github.com/dotcloud/docker/issues/614 https://github.com/dotcloud/docker/issues/614 2) All sorts of interesting software projects distributed as docker containers: http://index.docker.io http://index.docker.io 3) A highly active IRC channel, #docker@freenode where many cool applications of docker are being discussed

contingencies 13y ago

I believe your two examples are actually two entirely different paradigms. The first one is based on the exokernel notion. A bit muddy, but it seems that nobody has ever used a pure exokernel successfully at scale in industry, presumably because it seems to call for the kernel exposing hardware directly to userland code whilst simultaneously managing multiple tenants. Basically, the idea seems like a sort of early and poorly articulated call to modern OS level resource management (eg. cgroups). Your example two, the zerg system, seems very similar to LXC. Other than the exoticism of Erlang, their only big feature vs. normal paravirtualization seems to be startup time, which they see as a groundbreaking change. Well, I regularly start whole systems in LXC in fractions of a second. See https://github.com/globalcitizen/lxc-gentoo https://github.com/globalcitizen/lxc-gentoo In summary, these two projects are not really any different to the topic under discussion, ie. LXC (~cgroups/namespaces). Well, except that unlike cgroups/namespaces they're not necessarily able to run everywhere Linux runs!

vidarh 13y ago
The Mirage link is confusing matters. Mirage appears to be an exo/micro kernel for running in a para-virtualized system. That's very different from an exo-kernel running on bare hardware. With Mirage, what the apps run on seems to be "para-virtualized bare metal", so rather than using the POSIX syscall interface as its ABI, it's using the paravirtualized "hardware" as its ABI. That makes Mirage very similar to Zerg - it's the same concept, only OCaml vs. Erlang. As for LXC, I agree LXC offers a lot of this, especially since you don't need to boot full Linux environments - you can spawn individual applications and fine-tune what isolation you need. Or you can even build applications that explicitly use cgroups themselves to isolate sub-processes. So their "spawn one Xen Dom-U per request" demo could similarly have been a "spawn one new process isolated in its own cgroups". But these systems are appealing to me in that they bring VM level isolation with startup performance and resource utilization more in line with container systems. That's particularly appealing if we start seeing cloud systems that can handle it. Imagine being able to create new EC2 instances at sub-second times...