3 ms·

Statically linking the OS is quite common nowadays [1], better known as containerization. For quite a while there has been a movement that every program or "exe

by dtech 8y ago

Statically linking the OS is quite common nowadays [1], better known as containerization. For quite a while there has been a movement that every program or "executable" is statically linked. In your example, the "OS" is a program on it's own, it's controlled by a single vendor and released/updated at the same time. So dynamic linking it fine there For a downloadable program statically linking Qt would seem prudent however, otherwise an incompatible version on the OS would render the program unusable. You see this design decision nearly everywhere where legacy is not too big a concern: Go and Rust statically link its runtime and dedendencies in executables, JVM consumer apps are usually bundled with a specific JVM, containerization etc. [1] edit: masklinn correctly pointed out that usually the kernel is still shared between containers and the host OS

masklinn 8y ago
> Statically linking the kernel and OS is quite common nowadays, better known as containerization. For quite a while there has been a movement that every program or "executable" is statically linked. Containers specifically do not statically link the kernel, the entire point is to have multiple isolated userlands use the same underlying kernel. That's why they're so cheap.
- mapcars 8y ago
  >the entire point is to have multiple isolated userlands use the same underlying kernel. That's why they're so cheap. Ironically, this is what Plan9 does out-of-the-box. And even cheaper.
  - soraminazuki 8y ago
    Do you have any evidence to support your claim? My understanding is that containerized processes are theoretically the same as any other process on the system in terms of runtime performance.
    - mapcars 8y ago
      In plan9 they are exactly the same, not theoretically. Isolation happens on a file system level (everything is a file, but truly, not a "special device file"). While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance. >Linux namespaces were inspired by the more general namespace functionality used heavily throughout Plan 9 from Bell Labs. https://en.wikipedia.org/wiki/Cgroups https://en.wikipedia.org/wiki/Cgroups
      - masklinn 8y ago
        > While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance. It really doesn't. For all we know, because the plan9 version has to be more general it is much slower than the ad-hoc version.
- dtech 8y ago
  Fair enough, although it is the case if the host OS is different then the container, VM would be a better name then I guess. So maybe not kernel, but the entire userland/OS of the app is still statically linked with containers.
  - masklinn 8y ago
    > Fair enough, although it is the case if the host OS is different then the container, VM would be a better name then I guess. If the host OS is different than the container's you need either an intermediate hypervisor / VM (pretty sure that's what happens for docker on OSX, and why it's much more expensive than on linux) or for the host to provide an ABI compatible with the VM's OS/expectations (WSL, smartos's branded zones). Either adds even more "dynamic linking" to the pile. > So maybe not kernel, but the entire userland/OS of the app is still statically linked with containers. Yes, a container's userland should be statically linked.
- skybrian 8y ago
  Ok, yes, but that's just the kernel, and it's cheap only relative to a full VM. A container may or may not be cheap depending on the language runtime and whatever helper apps you're bundling. Bundling the python interpreter or a Java VM into every app isn't cheap. This reminds me of early web servers and CGI's. A CGI can be relatively cheap to run as long as you write it in C and don't link in too much; not so much for a scripting language. Redundancy can happen at multiple layers, so deduplicating at one layer might not be enough, depending on the situation.
Go0the0gophers 8y ago
That respects https://12factor.net/dependencies https://12factor.net/dependencies
MrBuddyCasino 8y ago
What you describe are unikernels, some argue a technology that lost to containerization.
segfaultbuserr 8y ago
> Statically linking the OS is quite common nowadays, better known as containerization. Shipping prepackaged images with self-supplied libraries is functionally equivalent to shipping a blob linked statically. But the implementation is different, in a container, each program still loads the self-supplied libraries using the loader, and this difference matters. Only dynamically-loaded libraries with PIE executables can enjoy the fully benefit of Address Space Layout Randomization. But statically-linked executables cannot, and they leave fixed offsets that can be exploited directly by an attacker. > Security measures like load address randomization cannot be used. With statically linked applications, only the stack and heap address can be randomized. All text has a fixed address in all invocations. With dynamically linked applications, the kernel has the ability to load all DSOs at arbitrary addresses, independent from each other. In case the application is built as a position independent executable (PIE) even this code can be loaded at random addresses. Fixed addresses (or even only fixed offsets) are the dreams of attackers. And no, it is not possible in general to generate PIEs with static linking. On IA-32 it is possible to use code compiled without -fpic and -fpie in PIEs (although with a cost) but this is not true for other architectures, including x86-64. Unless we stop writing major programs in C one day, and/or formally verify all programs in the system and provide a high-assurance system that most people would use, using exploit mitigation techniques is still the last line of defense to limit the damage created by a memory-safety bug. After thinking for a while, I think should change my stance, and support containerized applications such as Docker or Flatpak despite it's an ugly blob. A buggy and ugly container with prepacked applications linked dynamically to vulnerable blob with PIE and ASLR, is less of a evil than a buggy and ugly executable, linked statically to vulnerable blob with fixed addresses.

by dtech 8y ago

masklinn 8y ago

> Statically linking the kernel and OS is quite common nowadays, better known as containerization. For quite a while there has been a movement that every program or "executable" is statically linked. Containers specifically do not statically link the kernel, the entire point is to have multiple isolated userlands use the same underlying kernel. That's why they're so cheap.

mapcars 8y ago
>the entire point is to have multiple isolated userlands use the same underlying kernel. That's why they're so cheap. Ironically, this is what Plan9 does out-of-the-box. And even cheaper.
- soraminazuki 8y ago
  Do you have any evidence to support your claim? My understanding is that containerized processes are theoretically the same as any other process on the system in terms of runtime performance.
  - mapcars 8y ago
    In plan9 they are exactly the same, not theoretically. Isolation happens on a file system level (everything is a file, but truly, not a "special device file"). While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance. >Linux namespaces were inspired by the more general namespace functionality used heavily throughout Plan 9 from Bell Labs. https://en.wikipedia.org/wiki/Cgroups https://en.wikipedia.org/wiki/Cgroups
    - masklinn 8y ago
      > While it's practically not possible to compare performance, just the fact that this is a part of the initial design and a core of Plan9 vs adhoc feature (actually ported from Plan to Linux) is enough to speculate on performance. It really doesn't. For all we know, because the plan9 version has to be more general it is much slower than the ad-hoc version.
dtech 8y ago
Fair enough, although it is the case if the host OS is different then the container, VM would be a better name then I guess. So maybe not kernel, but the entire userland/OS of the app is still statically linked with containers.
- masklinn 8y ago
  > Fair enough, although it is the case if the host OS is different then the container, VM would be a better name then I guess. If the host OS is different than the container's you need either an intermediate hypervisor / VM (pretty sure that's what happens for docker on OSX, and why it's much more expensive than on linux) or for the host to provide an ABI compatible with the VM's OS/expectations (WSL, smartos's branded zones). Either adds even more "dynamic linking" to the pile. > So maybe not kernel, but the entire userland/OS of the app is still statically linked with containers. Yes, a container's userland should be statically linked.
skybrian 8y ago
Ok, yes, but that's just the kernel, and it's cheap only relative to a full VM. A container may or may not be cheap depending on the language runtime and whatever helper apps you're bundling. Bundling the python interpreter or a Java VM into every app isn't cheap. This reminds me of early web servers and CGI's. A CGI can be relatively cheap to run as long as you write it in C and don't link in too much; not so much for a scripting language. Redundancy can happen at multiple layers, so deduplicating at one layer might not be enough, depending on the situation.

Go0the0gophers 8y ago

That respects https://12factor.net/dependencies https://12factor.net/dependencies

MrBuddyCasino 8y ago

What you describe are unikernels, some argue a technology that lost to containerization.

segfaultbuserr 8y ago

> Statically linking the OS is quite common nowadays, better known as containerization. Shipping prepackaged images with self-supplied libraries is functionally equivalent to shipping a blob linked statically. But the implementation is different, in a container, each program still loads the self-supplied libraries using the loader, and this difference matters. Only dynamically-loaded libraries with PIE executables can enjoy the fully benefit of Address Space Layout Randomization. But statically-linked executables cannot, and they leave fixed offsets that can be exploited directly by an attacker. > Security measures like load address randomization cannot be used. With statically linked applications, only the stack and heap address can be randomized. All text has a fixed address in all invocations. With dynamically linked applications, the kernel has the ability to load all DSOs at arbitrary addresses, independent from each other. In case the application is built as a position independent executable (PIE) even this code can be loaded at random addresses. Fixed addresses (or even only fixed offsets) are the dreams of attackers. And no, it is not possible in general to generate PIEs with static linking. On IA-32 it is possible to use code compiled without -fpic and -fpie in PIEs (although with a cost) but this is not true for other architectures, including x86-64. Unless we stop writing major programs in C one day, and/or formally verify all programs in the system and provide a high-assurance system that most people would use, using exploit mitigation techniques is still the last line of defense to limit the damage created by a memory-safety bug. After thinking for a while, I think should change my stance, and support containerized applications such as Docker or Flatpak despite it's an ugly blob. A buggy and ugly container with prepacked applications linked dynamically to vulnerable blob with PIE and ASLR, is less of a evil than a buggy and ugly executable, linked statically to vulnerable blob with fixed addresses.