6 ms·

I often wonder, why isn't systemd-nspawn[0] used more often? It's self-described as "chroot on steroids". IME it pretty much lives up to that name. Makes it rea

I often wonder, why isn't systemd-nspawn[0] used more often? It's self-described as "chroot on steroids". IME it pretty much lives up to that name. Makes it really easy to containerize things and since it integrates well with systemd you basically don't have to learn new things. I totally get these are different tools and I don't think nspawn makes docker or podman useless, but I do find it interesting that it isn't more used, especially in things you're using completely locally. Say, your random self-hosted server thing that isn't escaping your LAN (e.g. Jellyfin or anything like this) [0] https://wiki.archlinux.org/title/Systemd-nspawn https://wiki.archlinux.org/title/Systemd-nspawn

deleted 2y ago
[deleted]
cmeacham98 2y ago
Because Docker/OCI/etc got the most important part right (or at least much better than the alternatives): distribution. All you need to start running a Docker container is a location and tag (or hash). To update, all you do is bump the tag (or hash). If a little more complicated setup is necessary (environment variables, volumes, ports, etc) - this can all be easily represented in common formats like Docker compose or Kubernetes manifests. How do you start running a system-nspawn container? Well first, you bootstrap an entire OS, then deal with that OS's package manager to install the application. You have to manage updates with the package manager yourself (which likely aren't immutable). There's no easy declarative config - you'll probably end up writing a shell script or using a third party tool like Ansible. There have been many container/chroot concepts in the past. Docker's idea was not novel, but they did building and distribution far better than any alternative when it first released, and it still holds up well today.
- ranger207 2y ago
  Yeah, this. Docker/container's greatest feature is less the sandboxing than the distribution. The sandboxing is essential to making the distribution work well, but it's a side feature most of the time
  - placardloop 2y ago
    It’s kind of funny that people think of “sandboxing” as the main feature of containers, or even as a feature at all. The distribution benefits have always been the entire point of Docker. The logo of Docker is a ship with a bunch of shipping containers on it (the original logo was clearer, but the current logo still shows this). “Containers” has never been about “containment”, but about modularity and portability.
    - shykes 2y ago
      Docker introduced an ambiguity in the meaning of the word "container". The word existed before Docker, and it was about sandboxing. Docker introduced the analogy of the shipping container, which as ranger207 says, is about sandboxing at the service of distribution. The two meanings - sandboxing and distribution - have coexisted ever since, sometimes causing misunderstandings and frustration.
    - globular-toast 2y ago
      It's not about sandboxing or distribution, it's about having a regular interface. This is why the container analogy works. In the analogy the ship is a computer and the containers are programs. Containers provide a regular interface such that a computer can run any program that is packaged up into a container. That's how things like Kubernetes work. They don't care what's in the container, just give them a container and they can run it. This is as opposed to the "old world" where computers needed to be specifically provisioned for running said program (like having interpreters and libraries available etc.), which is like shipping prior to containers: ships were more specialised to carrying particular loads. The analogy should not be extended to the ship moving and transporting stuff. That has nothing to do with it. The internet, URLs and tarballs have existed for decades.
  - guappa 2y ago
    Docker containers ran as root by default for a great number of years. I'm not even sure if it has now finally been changed. They provided no sandboxing whatsoever.
- godelski 2y ago
  Sorry. I agree, but that's a different question. I'll circle back to that then. Why don't technical people make these interfaces, giving the same love to user experience that something like Docker gets. As you said, it is scriptable, and I think -- us all being programmers here -- we all know that means you can just make the interface easier.
  - prmoustache 2y ago
    Are you implying that docker or podman hasn't been made by _technical people_?
    - godelski 2y ago
      No? I'm not sure I follow. I wouldn't say Apple wasn't made by technical people either. Saying technical people frequently ignore the importance of design does not mean that anyone who recognizes the importance of design is non technical
vaylian 2y ago
> I often wonder, why isn't systemd-nspawn[0] used more often? I think most people simply don't know about it. A lot of people also don't know that there are alternatives to Docker. I use both, systemd-nspawn and podman containers. They serve different purposes: systemd-nspawn: Run a complete operation system in a container. Updates are applied in-place. The whole system is writeable. I manage this system myself. I also use the -M switch for the systemctl and journalctl commands on the host to peek into my nspawn-containers. I create the system with debootstrap. podman: Run a stripped down operating system or just a contained executable with some supporting files. Most of the system is read-only with some writeable volumes mounted at well-defined locations in the file system tree. I don't manage the container image myself and I have activated auto-updates via the quadlet definition file. I create the container based on an image from a public container registry. Both solutions have their place. systemd-nspawn is a good choice if you want to create a long-lived linux system with lots of components. podman/docker containers are a good choice if you want to containerize an application with standard requirements. systemd-nspawn is good for pet containers. podman is good for cattle containers.
- fuhsnn 2y ago
  I just started learning to setup containers and found nspawn a total convenience, just create ./usr, throw some static-linked binaries to ./bin and systemd-nspawn -D would handle the rest including network pass-through.
bionsystem 2y ago
What is the advantage of nspawn vs lxc ? I use lxc extensively at the moment to test ansible recipes and it works pretty well even though I'm constrained on an old version.
- godelski 2y ago
  It uses the host kernel so you can get better performance. You can also do all the resource and capability management that you can do with systemd, so you have a bit more control over the level of consainerization and resource management
- mrbluecoat 2y ago
  or nspawn vs apptainer for that matter
wvh 2y ago
I used this extensively at the time Docker was up and coming. It worked well, much faster than Docker volumes, but required a lot of scripting and clean-up. What Docker got right, apart from distribution, is better separation of host system and whatever mess you are creating. You do not want to make a mistake bootstrapping an OS or forgetting to `chroot` to the right volume.
magicalhippo 2y ago
> Say, your random self-hosted server thing that isn't escaping your LAN (e.g. Jellyfin or anything like this) I tried reading your link but I'm none the wiser, so perhaps you could provide the docker-equivalent one-liner to start a Jellyfin instance using systemd-nspawn?
- godelski 2y ago
  There isn't a one liner because no one has built it. Which you be clear, this also had to be done for docker. I'll admit, the documentation to really anything systemd kinda sucks but awareness can help change that
  - magicalhippo 2y ago
    Ok, so I misread your question. You're asking why hasn't anyone made something like Docker but with systemd-nspawn as the runtime or "engine". edit: Found this article[1], which tries to do just that. Still not as convenient as Docker, but doesn't look terrible either. [1]: https://benjamintoll.com/2022/02/04/on-running-systemd-nspawn-containers/ https://benjamintoll.com/2022/02/04/on-running-systemd-nspaw...
    - godelski 2y ago
      Yeah, definitely there is a big difference between something being technically better (or worse) and the actual usability of a thing. We have a long history of products that are not technically better winning out (for many reasons). I'm confident nspawn doesn't have nearly the attention and even few people know about it. Docs definitely suck. But we're also on a very technical forum, not a general audience one, so I kinda assume a context that people here are not as concerned about the user interface.
      - magicalhippo 2y ago
        > But we're also on a very technical forum, not a general audience one, so I kinda assume a context that people here are not as concerned about the user interface. I think that's a common mistake. I'm fairly highly technical compared to your average user, but I don't have that much higher tolerance to friction for stuff that's not my core concern. Poor UX is definitely friction, and system administration is seldom my core concern. I'm fairly certain I'm not unique.

deleted 2y ago

[deleted]

cmeacham98 2y ago

Because Docker/OCI/etc got the most important part right (or at least much better than the alternatives): distribution. All you need to start running a Docker container is a location and tag (or hash). To update, all you do is bump the tag (or hash). If a little more complicated setup is necessary (environment variables, volumes, ports, etc) - this can all be easily represented in common formats like Docker compose or Kubernetes manifests. How do you start running a system-nspawn container? Well first, you bootstrap an entire OS, then deal with that OS's package manager to install the application. You have to manage updates with the package manager yourself (which likely aren't immutable). There's no easy declarative config - you'll probably end up writing a shell script or using a third party tool like Ansible. There have been many container/chroot concepts in the past. Docker's idea was not novel, but they did building and distribution far better than any alternative when it first released, and it still holds up well today.

ranger207 2y ago
Yeah, this. Docker/container's greatest feature is less the sandboxing than the distribution. The sandboxing is essential to making the distribution work well, but it's a side feature most of the time
- placardloop 2y ago
  It’s kind of funny that people think of “sandboxing” as the main feature of containers, or even as a feature at all. The distribution benefits have always been the entire point of Docker. The logo of Docker is a ship with a bunch of shipping containers on it (the original logo was clearer, but the current logo still shows this). “Containers” has never been about “containment”, but about modularity and portability.
  - shykes 2y ago
    Docker introduced an ambiguity in the meaning of the word "container". The word existed before Docker, and it was about sandboxing. Docker introduced the analogy of the shipping container, which as ranger207 says, is about sandboxing at the service of distribution. The two meanings - sandboxing and distribution - have coexisted ever since, sometimes causing misunderstandings and frustration.
  - globular-toast 2y ago
    It's not about sandboxing or distribution, it's about having a regular interface. This is why the container analogy works. In the analogy the ship is a computer and the containers are programs. Containers provide a regular interface such that a computer can run any program that is packaged up into a container. That's how things like Kubernetes work. They don't care what's in the container, just give them a container and they can run it. This is as opposed to the "old world" where computers needed to be specifically provisioned for running said program (like having interpreters and libraries available etc.), which is like shipping prior to containers: ships were more specialised to carrying particular loads. The analogy should not be extended to the ship moving and transporting stuff. That has nothing to do with it. The internet, URLs and tarballs have existed for decades.
- guappa 2y ago
  Docker containers ran as root by default for a great number of years. I'm not even sure if it has now finally been changed. They provided no sandboxing whatsoever.
godelski 2y ago
Sorry. I agree, but that's a different question. I'll circle back to that then. Why don't technical people make these interfaces, giving the same love to user experience that something like Docker gets. As you said, it is scriptable, and I think -- us all being programmers here -- we all know that means you can just make the interface easier.
- prmoustache 2y ago
  Are you implying that docker or podman hasn't been made by _technical people_?
  - godelski 2y ago
    No? I'm not sure I follow. I wouldn't say Apple wasn't made by technical people either. Saying technical people frequently ignore the importance of design does not mean that anyone who recognizes the importance of design is non technical

vaylian 2y ago

> I often wonder, why isn't systemd-nspawn[0] used more often? I think most people simply don't know about it. A lot of people also don't know that there are alternatives to Docker. I use both, systemd-nspawn and podman containers. They serve different purposes: systemd-nspawn: Run a complete operation system in a container. Updates are applied in-place. The whole system is writeable. I manage this system myself. I also use the -M switch for the systemctl and journalctl commands on the host to peek into my nspawn-containers. I create the system with debootstrap. podman: Run a stripped down operating system or just a contained executable with some supporting files. Most of the system is read-only with some writeable volumes mounted at well-defined locations in the file system tree. I don't manage the container image myself and I have activated auto-updates via the quadlet definition file. I create the container based on an image from a public container registry. Both solutions have their place. systemd-nspawn is a good choice if you want to create a long-lived linux system with lots of components. podman/docker containers are a good choice if you want to containerize an application with standard requirements. systemd-nspawn is good for pet containers. podman is good for cattle containers.

fuhsnn 2y ago
I just started learning to setup containers and found nspawn a total convenience, just create ./usr, throw some static-linked binaries to ./bin and systemd-nspawn -D would handle the rest including network pass-through.

bionsystem 2y ago

What is the advantage of nspawn vs lxc ? I use lxc extensively at the moment to test ansible recipes and it works pretty well even though I'm constrained on an old version.

godelski 2y ago
It uses the host kernel so you can get better performance. You can also do all the resource and capability management that you can do with systemd, so you have a bit more control over the level of consainerization and resource management
mrbluecoat 2y ago
or nspawn vs apptainer for that matter

wvh 2y ago

I used this extensively at the time Docker was up and coming. It worked well, much faster than Docker volumes, but required a lot of scripting and clean-up. What Docker got right, apart from distribution, is better separation of host system and whatever mess you are creating. You do not want to make a mistake bootstrapping an OS or forgetting to `chroot` to the right volume.

magicalhippo 2y ago

> Say, your random self-hosted server thing that isn't escaping your LAN (e.g. Jellyfin or anything like this) I tried reading your link but I'm none the wiser, so perhaps you could provide the docker-equivalent one-liner to start a Jellyfin instance using systemd-nspawn?

godelski 2y ago
There isn't a one liner because no one has built it. Which you be clear, this also had to be done for docker. I'll admit, the documentation to really anything systemd kinda sucks but awareness can help change that
- magicalhippo 2y ago
  Ok, so I misread your question. You're asking why hasn't anyone made something like Docker but with systemd-nspawn as the runtime or "engine". edit: Found this article[1], which tries to do just that. Still not as convenient as Docker, but doesn't look terrible either. [1]: https://benjamintoll.com/2022/02/04/on-running-systemd-nspawn-containers/ https://benjamintoll.com/2022/02/04/on-running-systemd-nspaw...
  - godelski 2y ago
    Yeah, definitely there is a big difference between something being technically better (or worse) and the actual usability of a thing. We have a long history of products that are not technically better winning out (for many reasons). I'm confident nspawn doesn't have nearly the attention and even few people know about it. Docs definitely suck. But we're also on a very technical forum, not a general audience one, so I kinda assume a context that people here are not as concerned about the user interface.
    - magicalhippo 2y ago
      > But we're also on a very technical forum, not a general audience one, so I kinda assume a context that people here are not as concerned about the user interface. I think that's a common mistake. I'm fairly highly technical compared to your average user, but I don't have that much higher tolerance to friction for stuff that's not my core concern. Poor UX is definitely friction, and system administration is seldom my core concern. I'm fairly certain I'm not unique.