11 ms·

This is starting to smell like a system on top of a system to fix something that could be fixed in the system. Kind-of like implementing a filesystem on top op

This is starting to smell like a system on top of a system to fix something that could be fixed in the system. Kind-of like implementing a filesystem on top op a filesystem... or putting a database on a filesystem to run another filesystem inside the database, or using a webbrowser as a runtime instead of an operating system.

zerocrates 10y ago
This kind of thing happens when the base system is ubiquitous and therefore hard to change. It's easier to layer something on top of the base, where people can "opt in" and there's a large preexisting compatible audience. Changing the base layer itself at a minimum requires people to upgrade, and now you don't have that advantage of the preexisting audience anymore. If your improvement requires a breaking change, then you're in a real pickle. So people will stack on more and more until it becomes more or less unbearable.
- user5994461 10y ago
  There is an advantage in layering components, or building new software on top of existing components. However, a minor improvement or bugfix should be done in the component that is responsible for it. Creating another layer instead of fixing the problem is just creating more problems.
  - numbsafari 10y ago
    You can go whole hog with "containers" by just adopting unikernels. But, unless you are willing to abandon Linux/Unix, this is kinda where you are left.
    - hueving 10y ago
      How would that work when containers make kernel calls?
      - sp332 10y ago
        Where containers would make kernel calls, unikernels make calls to the VM hypervisor. (Not exactly, but kinda.)
- omgam 10y ago
  A useful analogy for understanding meatspace ecosystems. Software is a bit like pond scum.
- duncan_bayne 10y ago
  Which is why highly flexible, powerful base systems are usually preferable to less flexible base systems optimised towards specific goals. Common Lisp is my go to example of this. It's possible to extend the language to support entirely new paradigms without breaking it, because of the power of the macro system.
- slackingoff2017 10y ago
  The reason a new layer is being built is because people don't want vendor lock in and VM's aren't portable anymore. You can't take an AWS VM and fire it up on DigitalOcean without trial by fire. VM's should be portable but they're not because cloud providers don't want them to be. For a while everyone was locked into AWS but now that there's other options some companies want to hedge their bets, or even run parts of their workload locally. Since the VM space is all proprietary now people are moving one level up and building a consistent environment above it. I fully expect docker to become proprietary in a couple years and the cycle will begin again.
  - rsync 10y ago
    "You can't take an AWS VM and fire it up on DigitalOcean without trial by fire. VM's should be portable but they're not because cloud providers don't want them to be." You can't ? I have never used DO, but I have deployed linux and FreeBSD systems on EC2 and moving them to bare metal was just a tar command away ... you can even pipeline 'dd' over ssh if you want to be fancy ...
    - fapjacks 10y ago
      You could say that it's as easy as using 'rsync'...
    - slackingoff2017 10y ago
      My friend pointed out today that he can't see my comments on HN. I think we're both hellbanned, just letting you know.
lostmsu 10y ago
The whole notion of containers is basically this. That's why I am not sure why not just fix the OS. If there's anything to fix in the first place.
- SomeStupidPoint 10y ago
  Containers are just unifying the concepts of namespaces and cgroups in to environments -- I'm not sure that there's a better way to do that at the OS level, as I enjoy the primitives being separate and having the potential to remix them as my understanding of containerization evolves. (Okay, so there's a few other things mixed in like SELinux settings.) I think there's interesting ideas of environments in other operating systems, but I'm not seeing how you'd make things better by flattening containers into the OS, per se.
  - derefr 10y ago
    One obvious tweak would be to make it so that every bit of isolation that containers now get by default, you instead get per POSIX process-group/session or somesuch, so you don't have to think in terms of containers to get the benefit of containers—they're just something the OS does transparently whenever you make it clear that a set of processes forms a distinct, separate cluster. Making existing programs compatible with such a paradigm probably wouldn't be any more work than e.g. adding SELinux/AppArmor support.
    - digi_owl 10y ago
      Nah, that would be too easy. Let's instead introduce yet another notion of sessions (logind).
- gumby 10y ago
  I always felt, perhaps uncharitably, that the point of containers was "those other programmers are idiots so we need to encapsulate everything for the sake of defense"
  - 10y ago
trhway 10y ago
>Kind-of like implementing a filesystem on top op a filesystem yep, when you want a distributed filesystem building it on top of a good and already working local filesystem is a pretty robust and cheap approach. >smell like a system on top of a system to fix something that could be fixed in the system. container layer is basically a distributed OS. Making a distributed OS by "fixing in the system" is pretty much non-starter ... or a huge academia project.
- michaelmrose 10y ago
  How is docker a distributed OS?
deleted 10y ago
[deleted]
gerdesj 10y ago
It's simply billed as the easiest way to run Docker, well: "The smallest, easiest way to run Docker in production at scale." I might give it a crack based on that. If you are hell bent on running Docker then an equivalent of the Ubuntu minimal install seems a good way to start.
developer2 10y ago
It's nothing like that at all. It's just a minimal operating system with a small install image. The only special thing it does is replace the init process with a system-docker process to reduce the overhead of resources used by the operating system. The point isn't really to "run more Docker". It's to eliminate as much operating system overhead as possible, so that nearly every CPU cycle and byte of memory usage is dedicated to your containers.
- dkarapetyan 10y ago
  You realize how you're not making any sense. Docker is another abstraction so it can not be better in terms of resource usage than running whatever process you are now running inside docker. You literally have more overhead with the docker approach in terms of all resource utilization. You now have all the overhead of an OS and then you are layering docker on top of it. The OS has not gone away. It is still managing processes, memory, files, sockets, networking, etc.
  - elcritch 10y ago
    Rather it's replacing systems like systemd and various other system daemons with the docker equivalents. For example why run a network supervision daemon if dockerd or equivalent handle all the important complex pieces of networking via container orchestration? Why have a local package manager, or system port mapper.
    - dkarapetyan 10y ago
      I'm pretty horrified by the argument you're making. The reason all those things are separate things is because they serve orthogonal functions. By bundling all of that into a single binary how have you improved things? You've increased the attack surface, reduced stability, increased complexity, and made things a lot harder to test and verify.
      - developer2 10y ago
        Have you ever used Rancher OS? Not trying to sound like a judgemental jerk, but the arguments you are making are clearly from someone who is commenting based on a preconceived notion of what it is, without knowing what it really is. Instead of combating others in comments because they can't succinctly describe it to you, go research it instead. If you don't care enough to look into it, you shouldn't be caring to argue in the comments.
skrebbel 10y ago
Your comment made me more interested in this solution than I was at first glance. I tend to prefer pragmatism over elegance, which is why I like using a webbrowser as a runtime, for instance. It'll work everywhere. If this idea has similar pragmatic advantages over the would-be best solution, then I'm game.
czeidler 10y ago
Have a look at http://genode.org/ http://genode.org/

zerocrates 10y ago

This kind of thing happens when the base system is ubiquitous and therefore hard to change. It's easier to layer something on top of the base, where people can "opt in" and there's a large preexisting compatible audience. Changing the base layer itself at a minimum requires people to upgrade, and now you don't have that advantage of the preexisting audience anymore. If your improvement requires a breaking change, then you're in a real pickle. So people will stack on more and more until it becomes more or less unbearable.

user5994461 10y ago
There is an advantage in layering components, or building new software on top of existing components. However, a minor improvement or bugfix should be done in the component that is responsible for it. Creating another layer instead of fixing the problem is just creating more problems.
- numbsafari 10y ago
  You can go whole hog with "containers" by just adopting unikernels. But, unless you are willing to abandon Linux/Unix, this is kinda where you are left.
  - hueving 10y ago
    How would that work when containers make kernel calls?
    - sp332 10y ago
      Where containers would make kernel calls, unikernels make calls to the VM hypervisor. (Not exactly, but kinda.)
omgam 10y ago
A useful analogy for understanding meatspace ecosystems. Software is a bit like pond scum.
duncan_bayne 10y ago
Which is why highly flexible, powerful base systems are usually preferable to less flexible base systems optimised towards specific goals. Common Lisp is my go to example of this. It's possible to extend the language to support entirely new paradigms without breaking it, because of the power of the macro system.
slackingoff2017 10y ago
The reason a new layer is being built is because people don't want vendor lock in and VM's aren't portable anymore. You can't take an AWS VM and fire it up on DigitalOcean without trial by fire. VM's should be portable but they're not because cloud providers don't want them to be. For a while everyone was locked into AWS but now that there's other options some companies want to hedge their bets, or even run parts of their workload locally. Since the VM space is all proprietary now people are moving one level up and building a consistent environment above it. I fully expect docker to become proprietary in a couple years and the cycle will begin again.
- rsync 10y ago
  "You can't take an AWS VM and fire it up on DigitalOcean without trial by fire. VM's should be portable but they're not because cloud providers don't want them to be." You can't ? I have never used DO, but I have deployed linux and FreeBSD systems on EC2 and moving them to bare metal was just a tar command away ... you can even pipeline 'dd' over ssh if you want to be fancy ...
  - fapjacks 10y ago
    You could say that it's as easy as using 'rsync'...
  - slackingoff2017 10y ago
    My friend pointed out today that he can't see my comments on HN. I think we're both hellbanned, just letting you know.

lostmsu 10y ago

The whole notion of containers is basically this. That's why I am not sure why not just fix the OS. If there's anything to fix in the first place.

SomeStupidPoint 10y ago
Containers are just unifying the concepts of namespaces and cgroups in to environments -- I'm not sure that there's a better way to do that at the OS level, as I enjoy the primitives being separate and having the potential to remix them as my understanding of containerization evolves. (Okay, so there's a few other things mixed in like SELinux settings.) I think there's interesting ideas of environments in other operating systems, but I'm not seeing how you'd make things better by flattening containers into the OS, per se.
- derefr 10y ago
  One obvious tweak would be to make it so that every bit of isolation that containers now get by default, you instead get per POSIX process-group/session or somesuch, so you don't have to think in terms of containers to get the benefit of containers—they're just something the OS does transparently whenever you make it clear that a set of processes forms a distinct, separate cluster. Making existing programs compatible with such a paradigm probably wouldn't be any more work than e.g. adding SELinux/AppArmor support.
  - digi_owl 10y ago
    Nah, that would be too easy. Let's instead introduce yet another notion of sessions (logind).
gumby 10y ago
I always felt, perhaps uncharitably, that the point of containers was "those other programmers are idiots so we need to encapsulate everything for the sake of defense"
- 10y ago

trhway 10y ago

>Kind-of like implementing a filesystem on top op a filesystem yep, when you want a distributed filesystem building it on top of a good and already working local filesystem is a pretty robust and cheap approach. >smell like a system on top of a system to fix something that could be fixed in the system. container layer is basically a distributed OS. Making a distributed OS by "fixing in the system" is pretty much non-starter ... or a huge academia project.

michaelmrose 10y ago
How is docker a distributed OS?

deleted 10y ago

[deleted]

gerdesj 10y ago

It's simply billed as the easiest way to run Docker, well: "The smallest, easiest way to run Docker in production at scale." I might give it a crack based on that. If you are hell bent on running Docker then an equivalent of the Ubuntu minimal install seems a good way to start.

developer2 10y ago

It's nothing like that at all. It's just a minimal operating system with a small install image. The only special thing it does is replace the init process with a system-docker process to reduce the overhead of resources used by the operating system. The point isn't really to "run more Docker". It's to eliminate as much operating system overhead as possible, so that nearly every CPU cycle and byte of memory usage is dedicated to your containers.

dkarapetyan 10y ago
You realize how you're not making any sense. Docker is another abstraction so it can not be better in terms of resource usage than running whatever process you are now running inside docker. You literally have more overhead with the docker approach in terms of all resource utilization. You now have all the overhead of an OS and then you are layering docker on top of it. The OS has not gone away. It is still managing processes, memory, files, sockets, networking, etc.
- elcritch 10y ago
  Rather it's replacing systems like systemd and various other system daemons with the docker equivalents. For example why run a network supervision daemon if dockerd or equivalent handle all the important complex pieces of networking via container orchestration? Why have a local package manager, or system port mapper.
  - dkarapetyan 10y ago
    I'm pretty horrified by the argument you're making. The reason all those things are separate things is because they serve orthogonal functions. By bundling all of that into a single binary how have you improved things? You've increased the attack surface, reduced stability, increased complexity, and made things a lot harder to test and verify.
    - developer2 10y ago
      Have you ever used Rancher OS? Not trying to sound like a judgemental jerk, but the arguments you are making are clearly from someone who is commenting based on a preconceived notion of what it is, without knowing what it really is. Instead of combating others in comments because they can't succinctly describe it to you, go research it instead. If you don't care enough to look into it, you shouldn't be caring to argue in the comments.

skrebbel 10y ago

Your comment made me more interested in this solution than I was at first glance. I tend to prefer pragmatism over elegance, which is why I like using a webbrowser as a runtime, for instance. It'll work everywhere. If this idea has similar pragmatic advantages over the would-be best solution, then I'm game.

czeidler 10y ago

Have a look at http://genode.org/ http://genode.org/