3 ms·

What exactly is the problem with process sandboxing and language level VMs? The industry is tackling all the wrong problems. So have we given up on process sand

What exactly is the problem with process sandboxing and language level VMs? The industry is tackling all the wrong problems. So have we given up on process sandboxing with capabilities? The whole containerization movement is one giant hammer to kill a fly kinda business these days. While you guys are figuring all this out I'm gonna stick with BEAM, JVM, and other tried and true methods. I'll check back in another 5 years to see what this mess has turned into. Maybe that's enough time to figure out how all the overlay networking is working out because, you know, we need a few more overlays between the hardware and the VM.

derefr 9y ago
Because then each piece of software you develop is limited to a single process written in a single language. How do you use "process sandboxing with capabilities" to isolate, say, an Erlang process that uses a NIF embedding a shared-memory section to interact with a companion OpenCL C process? With containers (or VMs), the answer is obvious; with POSIX-level isolation primitives, not so much.
- dkarapetyan 9y ago
  http://man7.org/linux/man-pages/man7/capabilities.7.html http://man7.org/linux/man-pages/man7/capabilities.7.html There are several capabilities in that list to address your problem. Since your question is concerned about memory you can search for "memory" in that page and note the exact capabilities you will need. OpenBSD has even more features (https://en.wikipedia.org/wiki/OpenBSD_security_features https://en.wikipedia.org/wiki/OpenBSD_security_features) alongside the capability model. My gripe is that instead of learning how to properly use their tools and platforms people have started looking for golden hammers like docker and now we have a mess like RancherOS. If someone can explain to me how isolating the system services in docker containers (which by the way are not isolated since they are privileged) does anything above and beyond what capabilities and the security features in OpenBSD provide then I'll concede the point. My guess is there are no advantages and people are jumping on bandwagons they know nothing about and since the previous generation of tools was not utilized to its full extent neither will all the new docker hotness. People are hammering square pegs into round holes.
  - derefr 9y ago
    Note that my point wasn't about how "your process" can be non-isolated in certain ways using capabilities, such that you could, in theory, sandbox both of those processes individually and have them still do whatever IPC you want; yes, this is certainly possible, and obviously more sensible if one of those processes isn't so much "your" process as it is some other process managed by some other party that you're interacting with. My point was more about the "developer UX" of needing to isolate things that way. Containers have the semantics of isolating a group of processes, but not performing any internal isolation between the processes in the container. This is almost always what you want—you want "your app" to be able to have multiple processes, and to not have any security boundary between the parts of "your app", just between "your app" and "other apps" or the OS. In other words, you want to have a single "process"—from the OS security subsystem's perspective—whose threads happens to be composed of multiple PIDs, multiple binaries, and multiple virtual-memory mappings. You want to fork(2) + exec(2) without creating a new security context in the process. Sandboxing would make perfect sense if single processes were always the granularity that "apps" existed at. Sometimes they are. Sometimes they're not, and people do complex things with IPC-capability-objects. And sometimes, when they're not, that fact pushes people toward avoiding multi-process architectures in favor of monolithic apps that reimplement functionality that already exists in some other program in themselves, in order to bring that functionality into their platform/runtime so it can live in their process. Containers let people avoid this decision, by just applying things like capabilities at the container level, rather than at the process level. > If someone can explain to me how isolating the system services in docker containers (which by the way are not isolated since they are privileged) does anything above and beyond what capabilities and the security features in OpenBSD provide then I'll concede the point. Completely apart from the above, my understanding of Rancher is that it's the Docker part of "Docker container", not the container part, that provides the benefit there. Docker is a packaging and service-management system; that its packages use containers is frequently beside the point. Rancher's system services are Docker images (i.e. Docker "packages"), and so you use Docker tooling to create, distribute, manage and upgrade them. If your own application on such a system is managed through Docker, this provides a neat solution to unifying your operations—you just do everything through the docker(1) command.
justincormack 9y ago
Because process sandboxing turned out to be insufficient, and we need layered sandboxing with several levels for protection for security.
ChuckMcM 9y ago
Well at Google where a lot of development on containers took place the problems with process sandboxing and language level VMs were machine resource allocation, both in memory and in I/O bandwidth. Lets say you have three "systems" on the box, one is a collection of processes providing elements of a file system, one is a collection of processes providing computation, and one is a collection of processes providing chat services. Now you want to allocate half of the disk i/o to the file services, and a half to the compute system, then 75 percent of the network bandwidth to the chat system, 20 percent to the file system, and the remaining plus any "unused" to the compute system. You want half of the memory in the system to go to the compute system and give the rest to the network file system processes. That is a complex mix of services running on a machine, some sharing the same flavor of VM, and you're allocating fractions of the total available resource capability to different components. If you cannot make hard allocations that are enforced by the kernel you cannot accurately reason about how the system will perform under load, and if one of your missions is to get your total system utilization to be quite high, you have to run things at the edge.

derefr 9y ago

Because then each piece of software you develop is limited to a single process written in a single language. How do you use "process sandboxing with capabilities" to isolate, say, an Erlang process that uses a NIF embedding a shared-memory section to interact with a companion OpenCL C process? With containers (or VMs), the answer is obvious; with POSIX-level isolation primitives, not so much.

dkarapetyan 9y ago
http://man7.org/linux/man-pages/man7/capabilities.7.html http://man7.org/linux/man-pages/man7/capabilities.7.html There are several capabilities in that list to address your problem. Since your question is concerned about memory you can search for "memory" in that page and note the exact capabilities you will need. OpenBSD has even more features (https://en.wikipedia.org/wiki/OpenBSD_security_features https://en.wikipedia.org/wiki/OpenBSD_security_features) alongside the capability model. My gripe is that instead of learning how to properly use their tools and platforms people have started looking for golden hammers like docker and now we have a mess like RancherOS. If someone can explain to me how isolating the system services in docker containers (which by the way are not isolated since they are privileged) does anything above and beyond what capabilities and the security features in OpenBSD provide then I'll concede the point. My guess is there are no advantages and people are jumping on bandwagons they know nothing about and since the previous generation of tools was not utilized to its full extent neither will all the new docker hotness. People are hammering square pegs into round holes.
- derefr 9y ago
  Note that my point wasn't about how "your process" can be non-isolated in certain ways using capabilities, such that you could, in theory, sandbox both of those processes individually and have them still do whatever IPC you want; yes, this is certainly possible, and obviously more sensible if one of those processes isn't so much "your" process as it is some other process managed by some other party that you're interacting with. My point was more about the "developer UX" of needing to isolate things that way. Containers have the semantics of isolating a group of processes, but not performing any internal isolation between the processes in the container. This is almost always what you want—you want "your app" to be able to have multiple processes, and to not have any security boundary between the parts of "your app", just between "your app" and "other apps" or the OS. In other words, you want to have a single "process"—from the OS security subsystem's perspective—whose threads happens to be composed of multiple PIDs, multiple binaries, and multiple virtual-memory mappings. You want to fork(2) + exec(2) without creating a new security context in the process. Sandboxing would make perfect sense if single processes were always the granularity that "apps" existed at. Sometimes they are. Sometimes they're not, and people do complex things with IPC-capability-objects. And sometimes, when they're not, that fact pushes people toward avoiding multi-process architectures in favor of monolithic apps that reimplement functionality that already exists in some other program in themselves, in order to bring that functionality into their platform/runtime so it can live in their process. Containers let people avoid this decision, by just applying things like capabilities at the container level, rather than at the process level. > If someone can explain to me how isolating the system services in docker containers (which by the way are not isolated since they are privileged) does anything above and beyond what capabilities and the security features in OpenBSD provide then I'll concede the point. Completely apart from the above, my understanding of Rancher is that it's the Docker part of "Docker container", not the container part, that provides the benefit there. Docker is a packaging and service-management system; that its packages use containers is frequently beside the point. Rancher's system services are Docker images (i.e. Docker "packages"), and so you use Docker tooling to create, distribute, manage and upgrade them. If your own application on such a system is managed through Docker, this provides a neat solution to unifying your operations—you just do everything through the docker(1) command.

justincormack 9y ago

Because process sandboxing turned out to be insufficient, and we need layered sandboxing with several levels for protection for security.

ChuckMcM 9y ago

Well at Google where a lot of development on containers took place the problems with process sandboxing and language level VMs were machine resource allocation, both in memory and in I/O bandwidth. Lets say you have three "systems" on the box, one is a collection of processes providing elements of a file system, one is a collection of processes providing computation, and one is a collection of processes providing chat services. Now you want to allocate half of the disk i/o to the file services, and a half to the compute system, then 75 percent of the network bandwidth to the chat system, 20 percent to the file system, and the remaining plus any "unused" to the compute system. You want half of the memory in the system to go to the compute system and give the rest to the network file system processes. That is a complex mix of services running on a machine, some sharing the same flavor of VM, and you're allocating fractions of the total available resource capability to different components. If you cannot make hard allocations that are enforced by the kernel you cannot accurately reason about how the system will perform under load, and if one of your missions is to get your total system utilization to be quite high, you have to run things at the edge.