6 ms·

Why do I keep getting this distinct impression that people use containers to keep reinventing OS processes, but "in the cloud!"? Next thing, you'll see announc

Why do I keep getting this distinct impression that people use containers to keep reinventing OS processes, but "in the cloud!"? Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers - to avoid including them multiple times in your application containers that share the same version, and we'll have come full circle. Is there a subtle detail about packaging software for distribution as isolated executables that I'm missing?

rkeene2 4y ago
It's the same reason people keep inventing VMs. Every process is also a VM (and can only do what the hypervisor... kernel... allows), but it's a more convenient abstraction to run a new kernel and its processes in a hardware emulating VM within that VM for many kinds of workloads, despite the overhead of emulating hardware. Containers are a middle-ground, and a useful abstraction -- where you can have a group of processes that can share some resources and be accounted for together, but without the need to emulate hardware or a second hypervisor (kernel). For lots of workloads, though, processes would be enough. Some more could be done with additional work to kernels to help group processes together (Linux cgroups, Solaris contracts and zone facilities) and more container-aware schedulers (Not sure what Linux has, Solaris has a 2-layer scheduler for its containers). I was heavily into containers in the 90s and 2000s, and then VMs when they became more viable with Xen 2, but I've been moving things back to just processes.
- liotier 4y ago
  And then, one day, single-system-image will be back in fashion and processes will migrate across the cluster - like in the good old days of Openmosix, but maybe with Kubernetes-like robustness...
brimble 4y ago
No, you're basically right. To the extent that containers solve a problem, it's mostly a problem with package management, and with UI for process isolation/permissions/hardening.
qbasic_forever 4y ago
> Is there a subtle detail about packaging software for distribution as isolated executables that I'm missing? In practice it's a nightmare, particularly for legacy software that demands a ton of dynamic linking to system-installed libraries.
- TuringTest 4y ago
  Yeah, but that's a problem with dynamically linked dependencies. If you statically link all your dependencies into a single executable, doesn't it work conceptually the same as a Docker container with all your tech stack contained in a single package?
  - deleted 4y ago
    [deleted]
  - paulgb 4y ago
    Even a statically linked program can have dependencies on the system it’s running on, like system fonts and root certificates. Containers ensure a standard environment that accounts for those differences, in a language-agnostic way.
  - qbasic_forever 4y ago
    Yep, but when you're dealing with something to run your desktop or other software inside most of that software is dynamically linked tools and utilities.
- hackerfromthefu 4y ago
  Whether its a nightmare or not depends on what you practise. Dependency heavy environments such as e.g. node or python make it a nightmare. But large stable base library environments with selectively chosen dependencies are usually easy to manage, e.g. .NET.
chrisshroba 4y ago
If I give someone a sample docker-compose file, they can immediately run my service regardless of OS. If I distribute manually, I need to provide instructions for setting up the proper dev environment and packages for several common OSes and distros (brew maybe, apt, rpm, etc.). Speaking personally, I know how to write a proper Dockerfile, (and it's a skill that's learnable in a couple hours). I have no idea how to distribute packages through other formats and avoid footguns.
- pid-1 4y ago
  Docker's managed VM strategy for Windows and MacOS is really powerful, I'm not sure why other applications don't do that.
- cmroanirgo 4y ago
  There's a lot more to running a service than getting it running easily. Longevity/stability through applying updates should be engineered too. The problem is when using Docker, you can't tell what the dependencies are for your app. Telling me i need node or Go or php and/or what database or redis, etc etc gives me an instant feel for how the deployment, as well as how security updates, need to be applied. Docker is just a black box by comparison... at least to me. All my attempts at running docker solutions on small vms have ended badly (ports opened publically, rampant disk use, poor log files management, lack of security updates...). Seriously, I wish devs would at least list the tech stacks they're using in their apps in the readme. However, i do grok that people who've embraced the ecosystem would like it that way... but not all techs like it. For me, the best projects are those that lay it all out, warts and all and then have a separate repo that manages the docker state.
  - dwild 4y ago
    > The problem is when using Docker, you can't tell what the dependencies are for your app. That's all in the Dockerfile though... it's a simple and standard way to show how to install something. It's just like a makefile, but it's one that always works whatever the environment you are in, whatever the dependencies that you already got in your system and in most case, whether it has been maintained or not. > ports opened publically ... and that wouldn't have happened if you used something else? How so? Unlike any other solution where each application choose how ports are configured, on Docker you actually need to be aware of the port to open and specify it. I means sure you could have not known that by default it does it over 0.0.0.0 but that would be true for almost any applications (and that's when you are even aware of the ports, that's a basic CTF challenge to have an unknown port open). > rampant disk use That's a good point, I agree that the storage can be quite annoying, but at the same time, you handle that the same way you would handle that with any other software, simply by knowing where the storage goes and why. > poor log files management I love how log is handled with Docker, there's a single log output and that's it. It's the source of truth for logging, easy peasy. You want your logs to be pushed to another system? Well connect it to the Docker logging system and that's it. > lack of security updates Could you develop that one? You are responsible to keep updating whatever you use, whether it's a docker container or an application. One or the other doesn't change that. Maybe you means that it's so easy to make a Docker image that it's just as easy to stop updating it thus making it less secure for whoever use it? I means sure... that would make it less secure but it's still possible for any application to be abandoned, it is your responsibility to make sure whatever you use will be maintained. > Seriously, I wish devs would at least list the tech stacks they're using in their apps in the readme. I haven't seen many Docker images that doesn't do that. I do have seen a few that doesn't go into depth on how to setup the environment, but the tech stack, that's mostly a given. Any good examples of that?
johnmarcus 4y ago
No, there are blatantly obvious things about packaging software that you are missing. - you will need a copy of each platform running in order to build the binary - it's X-times++ as much work to package for X number of platforms. - The code you chose might not compile well on all platforms without code changes. - Dependency conflicts can be a pain. oh boy, i could go on.
hedora 4y ago
You can dynamically link container services from other containers. Look up "docker layers".
- puetzk 4y ago
  You can dynamically link one service from another container. Choose wisely.
a9h74j 4y ago
> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers DLL --> JCL ... Joinable Container Libraries
ElectricalUnion 4y ago
> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers - to avoid including them multiple times in your application containers that share the same version, and we'll have come full circle. That unfortunately already exists; Kubernetes namespaces and Helm Charts.
Sophistifunk 4y ago
No you're not wrong, it's the constant churn of "our current isolation between users/processes/apps isn't as good as we hoped, so lets just run virtualise the whole OS" that has been driving multiprocessing since the 70s.
hda111 4y ago
> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers I think that wouldn’t be a bad idea. It’s good for RAM and disk space usage while still isolating processes. Would be useful for IoT devices without much memory or space that still have to run docker.

rkeene2 4y ago

It's the same reason people keep inventing VMs. Every process is also a VM (and can only do what the hypervisor... kernel... allows), but it's a more convenient abstraction to run a new kernel and its processes in a hardware emulating VM within that VM for many kinds of workloads, despite the overhead of emulating hardware. Containers are a middle-ground, and a useful abstraction -- where you can have a group of processes that can share some resources and be accounted for together, but without the need to emulate hardware or a second hypervisor (kernel). For lots of workloads, though, processes would be enough. Some more could be done with additional work to kernels to help group processes together (Linux cgroups, Solaris contracts and zone facilities) and more container-aware schedulers (Not sure what Linux has, Solaris has a 2-layer scheduler for its containers). I was heavily into containers in the 90s and 2000s, and then VMs when they became more viable with Xen 2, but I've been moving things back to just processes.

liotier 4y ago
And then, one day, single-system-image will be back in fashion and processes will migrate across the cluster - like in the good old days of Openmosix, but maybe with Kubernetes-like robustness...

brimble 4y ago

No, you're basically right. To the extent that containers solve a problem, it's mostly a problem with package management, and with UI for process isolation/permissions/hardening.

qbasic_forever 4y ago

> Is there a subtle detail about packaging software for distribution as isolated executables that I'm missing? In practice it's a nightmare, particularly for legacy software that demands a ton of dynamic linking to system-installed libraries.

TuringTest 4y ago
Yeah, but that's a problem with dynamically linked dependencies. If you statically link all your dependencies into a single executable, doesn't it work conceptually the same as a Docker container with all your tech stack contained in a single package?
- deleted 4y ago
  [deleted]
- paulgb 4y ago
  Even a statically linked program can have dependencies on the system it’s running on, like system fonts and root certificates. Containers ensure a standard environment that accounts for those differences, in a language-agnostic way.
- qbasic_forever 4y ago
  Yep, but when you're dealing with something to run your desktop or other software inside most of that software is dynamically linked tools and utilities.
hackerfromthefu 4y ago
Whether its a nightmare or not depends on what you practise. Dependency heavy environments such as e.g. node or python make it a nightmare. But large stable base library environments with selectively chosen dependencies are usually easy to manage, e.g. .NET.

chrisshroba 4y ago

If I give someone a sample docker-compose file, they can immediately run my service regardless of OS. If I distribute manually, I need to provide instructions for setting up the proper dev environment and packages for several common OSes and distros (brew maybe, apt, rpm, etc.). Speaking personally, I know how to write a proper Dockerfile, (and it's a skill that's learnable in a couple hours). I have no idea how to distribute packages through other formats and avoid footguns.

pid-1 4y ago
Docker's managed VM strategy for Windows and MacOS is really powerful, I'm not sure why other applications don't do that.
cmroanirgo 4y ago
There's a lot more to running a service than getting it running easily. Longevity/stability through applying updates should be engineered too. The problem is when using Docker, you can't tell what the dependencies are for your app. Telling me i need node or Go or php and/or what database or redis, etc etc gives me an instant feel for how the deployment, as well as how security updates, need to be applied. Docker is just a black box by comparison... at least to me. All my attempts at running docker solutions on small vms have ended badly (ports opened publically, rampant disk use, poor log files management, lack of security updates...). Seriously, I wish devs would at least list the tech stacks they're using in their apps in the readme. However, i do grok that people who've embraced the ecosystem would like it that way... but not all techs like it. For me, the best projects are those that lay it all out, warts and all and then have a separate repo that manages the docker state.
- dwild 4y ago
  > The problem is when using Docker, you can't tell what the dependencies are for your app. That's all in the Dockerfile though... it's a simple and standard way to show how to install something. It's just like a makefile, but it's one that always works whatever the environment you are in, whatever the dependencies that you already got in your system and in most case, whether it has been maintained or not. > ports opened publically ... and that wouldn't have happened if you used something else? How so? Unlike any other solution where each application choose how ports are configured, on Docker you actually need to be aware of the port to open and specify it. I means sure you could have not known that by default it does it over 0.0.0.0 but that would be true for almost any applications (and that's when you are even aware of the ports, that's a basic CTF challenge to have an unknown port open). > rampant disk use That's a good point, I agree that the storage can be quite annoying, but at the same time, you handle that the same way you would handle that with any other software, simply by knowing where the storage goes and why. > poor log files management I love how log is handled with Docker, there's a single log output and that's it. It's the source of truth for logging, easy peasy. You want your logs to be pushed to another system? Well connect it to the Docker logging system and that's it. > lack of security updates Could you develop that one? You are responsible to keep updating whatever you use, whether it's a docker container or an application. One or the other doesn't change that. Maybe you means that it's so easy to make a Docker image that it's just as easy to stop updating it thus making it less secure for whoever use it? I means sure... that would make it less secure but it's still possible for any application to be abandoned, it is your responsibility to make sure whatever you use will be maintained. > Seriously, I wish devs would at least list the tech stacks they're using in their apps in the readme. I haven't seen many Docker images that doesn't do that. I do have seen a few that doesn't go into depth on how to setup the environment, but the tech stack, that's mostly a given. Any good examples of that?

johnmarcus 4y ago

No, there are blatantly obvious things about packaging software that you are missing. - you will need a copy of each platform running in order to build the binary - it's X-times++ as much work to package for X number of platforms. - The code you chose might not compile well on all platforms without code changes. - Dependency conflicts can be a pain. oh boy, i could go on.

hedora 4y ago

You can dynamically link container services from other containers. Look up "docker layers".

puetzk 4y ago
You can dynamically link one service from another container. Choose wisely.

a9h74j 4y ago

> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers DLL --> JCL ... Joinable Container Libraries

ElectricalUnion 4y ago

> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers - to avoid including them multiple times in your application containers that share the same version, and we'll have come full circle. That unfortunately already exists; Kubernetes namespaces and Helm Charts.

Sophistifunk 4y ago

No you're not wrong, it's the constant churn of "our current isolation between users/processes/apps isn't as good as we hoped, so lets just run virtualise the whole OS" that has been driving multiprocessing since the 70s.

hda111 4y ago

> Next thing, you'll see announced a platform for reusable container services that can be dynamically linked from other containers I think that wouldn’t be a bad idea. It’s good for RAM and disk space usage while still isolating processes. Would be useful for IoT devices without much memory or space that still have to run docker.