6 ms·

This is a very interesting development! But doesn't that mean we are merely reimplementing old concepts on a new level? Almost everything we do on VM level we

by vog 9y ago

This is a very interesting development! But doesn't that mean we are merely reimplementing old concepts on a new level? Almost everything we do on VM level we already did on process level: - Read-only VM images are essentially like statically linked binaries, just better encapsulated when they run. - Tiny, fast loading VM images are essentially stripped static binaries with dead code elimination. - Starting those by incoming network request is just like old "inetd" versions started custom handler programs on each request. - Same with CGI, just on HTTP rather than TCP level. - Manual maintaining the lifecycle of those VMs like FastCGI in nginx. - Maintaining the lifecycle of such a started VM is essentially like FastCGI in Apache (or nginx + auto-spawner). The only real difference is the improved encapsulation (controlled network access and resource consumption) of the running code, isn't it? What might come next? - Multiple VMs could share the same images to save memory, essentially reinventing shared libraries and dynamic linking. - Network traffic into and out of a VM is controlled more tightly, reinventing SELinux. (Perhaps followed by noticing that these profiles are hard to maintain, rediscovering the maintenance issues of SELinux systems.) - Providing the network traffic shape directly with the image, so these can be maintained at a single place, by the original developers. This would reinvent Pledge. - Introducing permissions for network traffic, e.g. such that just one group of application VMs can communicate with the database VMs. Ideally with an ACL system as used for managing file permissions. This would reinvent Unix Sockets.

catern 9y ago
Yes, all this "container per request" or "unikernel per request" stuff is essentially the same as what everyone was doing 20 years ago with basic CGI. But this stuff also has far worse performance than CGI's "process per request", since processes can be started very fast, in microseconds. It would be more interesting if we just made more sandboxing powers available for traditional processes, though there is already quite a lot of sandboxing available for Linux processes...
- anilgulecha 9y ago
  > made more sandboxing powers available for traditional processes You just described containers :)
  - catern 9y ago
    It depends on your definition of "container". The popular Docker-style container is substantially heavier than "just a process"; it creates cgroups, bind mounts things, creates new pid/user/mount/etc namespaces, all kinds of stuff. Such containers aren't just used for sandboxing; they are also used as a packaging mechanism (install whatever software versions you want) and deployment mechanism (special networking setups to connect containers) which partially explains why they are so heavy. Most of that is not really necessary if you are just coming from a standpoint of "I want to sandbox a process". Indeed, if you just want to sandbox a process, just run it under a dedicated non-root user account, and it won't be able to unduly interfere with the rest of the system.
    - aseipp 9y ago
      > Indeed, if you just want to sandbox a process, just run it under a dedicated non-root user account, and it won't be able to unduly interfere with the rest of the system. This is simply not true, unless your definition of "sandboxing" only means "run it under another account" and absolutely nothing more -- but under this definition, non-root processes can 100% interfere with the rest of the system without proper control being put into place. "Intereference" can mean a lot more than "writes to my ~ dir" -- for example, monopolizing a system resource (e.g. IOPS or network bandwidth), adding code to the kernel (module autoloading) or simply using a resource at all when it shouldn't, or rogue problems like termination of a hostile/stupid set of processes (cgroups and namespaces are one of the few reliable ways to kill process groups in non-racey ways, etc). > Most of that is not really necessary if you are just coming from a standpoint of "I want to sandbox a process". No, a substantial amount of it is necessary if you actually want to sandbox a process in a way that's remotely isolated from the host system. Use cases like Flatpak and Docker are nice, but merely byproducts of the design that allow custom mount namespaces, etc. The original use cases for namespacing, cgroups, etc was precisely to give more fine-grained resource and isolation control over existing applications, in a hierarchical manner.
      - catern 9y ago
        >monopolizing a system resource (e.g. IOPS or network bandwidth) People have been placing ulimits on users to limit their resource usage for decades, using cgroups to do it is not a novel concept, just more effective. >rogue problems like termination of a hostile/stupid set of processes (cgroups and namespaces are one of the few reliable ways to kill process groups in non-racey ways, etc). The ability for a user to leave processes around on the system has been fixed for years now, with systemd-logind which puts each user in its own cgroup. >adding code to the kernel (module autoloading) This is a concern both for container systems and regular user isolation, and the only reliable way to fix it for either is to disable module autoloading, or at least restrict the available modules to well-maintained ones. >The original use cases for namespacing, cgroups, etc was precisely to give more fine-grained resource and isolation control over existing applications, in a hierarchical manner. Yes, and they failed. The design of Linux namespaces and cgroups is awful. You can perceive this by the fact that they require privileges to use. Hopefully in a few years all this Flatpak/Docker churn will output some namespacing APIs for Linux that are actually good.
- user5994461 9y ago
  A process takes multiple milliseconds to start, not microseconds. Creating a process is really expensive. CGI starts a process per request. fastCGI can reuse processes. Problem already found and solved 20 years ago.
  - catern 9y ago
    catern@home ~ $ { timeout 1 sh -c 'while :; do /bin/echo foo; done'; echo; } | wc -l 1542 So at least sub-millisecond. And this is less efficient than possible because it's also writing to a pipe. Also, I quite agree that CGI is a bad way to do things, but that's what the original post is proposing. :)
- nimish 9y ago
  Containers are basically processes with extra sandboxing.
forgotpassagan 9y ago
It's all part of the endless fight against proprietary systems. When cloud providers started using 'optimized' images the response was to move everything above the VM to 'container' level. The same thing happened in the VM boom once hardware makers started adding binary blob drivers to bare metal. We'll just keep building layers as manufacturers try to lock the lower ones down :) See WeChat in China for a good example on Android

by vog 9y ago

catern 9y ago

Yes, all this "container per request" or "unikernel per request" stuff is essentially the same as what everyone was doing 20 years ago with basic CGI. But this stuff also has far worse performance than CGI's "process per request", since processes can be started very fast, in microseconds. It would be more interesting if we just made more sandboxing powers available for traditional processes, though there is already quite a lot of sandboxing available for Linux processes...

anilgulecha 9y ago
> made more sandboxing powers available for traditional processes You just described containers :)
- catern 9y ago
  It depends on your definition of "container". The popular Docker-style container is substantially heavier than "just a process"; it creates cgroups, bind mounts things, creates new pid/user/mount/etc namespaces, all kinds of stuff. Such containers aren't just used for sandboxing; they are also used as a packaging mechanism (install whatever software versions you want) and deployment mechanism (special networking setups to connect containers) which partially explains why they are so heavy. Most of that is not really necessary if you are just coming from a standpoint of "I want to sandbox a process". Indeed, if you just want to sandbox a process, just run it under a dedicated non-root user account, and it won't be able to unduly interfere with the rest of the system.
  - aseipp 9y ago
    > Indeed, if you just want to sandbox a process, just run it under a dedicated non-root user account, and it won't be able to unduly interfere with the rest of the system. This is simply not true, unless your definition of "sandboxing" only means "run it under another account" and absolutely nothing more -- but under this definition, non-root processes can 100% interfere with the rest of the system without proper control being put into place. "Intereference" can mean a lot more than "writes to my ~ dir" -- for example, monopolizing a system resource (e.g. IOPS or network bandwidth), adding code to the kernel (module autoloading) or simply using a resource at all when it shouldn't, or rogue problems like termination of a hostile/stupid set of processes (cgroups and namespaces are one of the few reliable ways to kill process groups in non-racey ways, etc). > Most of that is not really necessary if you are just coming from a standpoint of "I want to sandbox a process". No, a substantial amount of it is necessary if you actually want to sandbox a process in a way that's remotely isolated from the host system. Use cases like Flatpak and Docker are nice, but merely byproducts of the design that allow custom mount namespaces, etc. The original use cases for namespacing, cgroups, etc was precisely to give more fine-grained resource and isolation control over existing applications, in a hierarchical manner.
    - catern 9y ago
      >monopolizing a system resource (e.g. IOPS or network bandwidth) People have been placing ulimits on users to limit their resource usage for decades, using cgroups to do it is not a novel concept, just more effective. >rogue problems like termination of a hostile/stupid set of processes (cgroups and namespaces are one of the few reliable ways to kill process groups in non-racey ways, etc). The ability for a user to leave processes around on the system has been fixed for years now, with systemd-logind which puts each user in its own cgroup. >adding code to the kernel (module autoloading) This is a concern both for container systems and regular user isolation, and the only reliable way to fix it for either is to disable module autoloading, or at least restrict the available modules to well-maintained ones. >The original use cases for namespacing, cgroups, etc was precisely to give more fine-grained resource and isolation control over existing applications, in a hierarchical manner. Yes, and they failed. The design of Linux namespaces and cgroups is awful. You can perceive this by the fact that they require privileges to use. Hopefully in a few years all this Flatpak/Docker churn will output some namespacing APIs for Linux that are actually good.
user5994461 9y ago
A process takes multiple milliseconds to start, not microseconds. Creating a process is really expensive. CGI starts a process per request. fastCGI can reuse processes. Problem already found and solved 20 years ago.
- catern 9y ago
  catern@home ~ $ { timeout 1 sh -c 'while :; do /bin/echo foo; done'; echo; } | wc -l 1542 So at least sub-millisecond. And this is less efficient than possible because it's also writing to a pipe. Also, I quite agree that CGI is a bad way to do things, but that's what the original post is proposing. :)
nimish 9y ago
Containers are basically processes with extra sandboxing.

forgotpassagan 9y ago

It's all part of the endless fight against proprietary systems. When cloud providers started using 'optimized' images the response was to move everything above the VM to 'container' level. The same thing happened in the VM boom once hardware makers started adding binary blob drivers to bare metal. We'll just keep building layers as manufacturers try to lock the lower ones down :) See WeChat in China for a good example on Android