12 ms·

Walkthrough for Systemd Portable Services

152 points by type0 8y ago

mathnmusic 8y ago
Yet another tool for containerization technology :-) While this is interesting, the containers that I'd appreciate, are: multiple versions of programming languages & packages to be installed by each user without affecting the system globally. Every language has their own separate answer (virtualenv/venv, rbenv, node version manager etc). Just today I struggled with "brew install python3" which complained with "Error: python 2.7.14_2 is already installed" and the only option offered was to upgrade. Whereas what I wanted was parallel installations of python 2.7 and python3. :-/
- viraptor 8y ago
  If you want to preserve python 2, you `brew install python@2`. You can have both that way. (Updated homebrew required)
- vbezhenar 8y ago
  There shouldn't be need for containers to use different versions of programming toolkits. I'm Java developer and in our world it's extremely easy: JDK is just a directory with bunch of files. I can have (and I actually have) around 10 versions of JDKs, from Java 1.1 to Java 10. There's no need to install anything, just extract the archive. To use a particular Java usually JAVA_HOME environment variable is enough. That works for every operating system from Linux to Windows and macOS identically. That's why I don't ever use distro Java, it's a mess, just download Oracle JDK and start work.
  - foota 8y ago
    Java's missing a decent packaging solution though.
    - shub 8y ago
      I've never had a problem using wars or fat jars for server deployments. Client distribution is another matter.
    - gitgud 8y ago
      Gradle works pretty well in my opinion
    - jamietanna 8y ago
      I've not had any issues with Maven before
- clhodapp 8y ago
  That's exactly what Nix does! Nix is wonderful tech that unfortunately seems doomed to low adoption due to poor usability. My belief is that this stems the fact that maintaining the package tree is obscenely costly of contributor time, leaving almost no time or energy to make progress on "niceties" like improving the core CLI experience and the continuous integration system.
  - rspeer 8y ago
    I once started trying to learn about Nix. All the up-to-date documentation I could find, however, was about NixOS. I would like to use Nix just to manage my local development environment on Ubuntu, not to replace my entire operating system. Is that something the developers are still interested in? Is there documentation?
    - jumblemuddle 8y ago
      Have a look here: https://nixos.org/nix/manual/ https://nixos.org/nix/manual/ There's a dropdown in the top-left corner to switch between nix-related projects. (NixOS, NixOps, etc.)
      - andreabedini 8y ago
        I found nix-pills more useful than the manual: https://nixos.org/nixos/nix-pills/index.html https://nixos.org/nixos/nix-pills/index.html
  - manveru 8y ago
    The Nix team (in particular grahamc with https://github.com/NixOS/ofborg https://github.com/NixOS/ofborg) is working hard on making CI better . But it's true that the on-boarding experience and CLI could be much improved. It's quite frustrating to me, because after I've been using Nix & NixOS for about 3 years now, there's no way I'd go back to the insanity out there. I never had the amount of stability, control, and predictability on any other platform, from configuring my OS, editors, WM, shell, random ricing, or building VMs, Servers, Docker containers, production deploys, development environments, there's really not much Nix can't handle. But getting familiar with Nix took some serious effort I'm sure not many people can afford. I begin to understand how frustrated people in the Lisp and Smalltalk communities must feel while people slowly reinvent everything they've enjoyed for decades (and Nix is still in its teens). Teaching people about the value of having just a single package manager, no matter what OS or language they're using, is mostly futile because there's so much value attributed to being "mainstream" that now we have hundreds of mainstream package managers with varying degrees of sophistication, security, predictability, and ability to _not_ try and take over your system. I use it to build and develop Go, Ruby, JS, Crystal, Elm, Mint, Haskell, Perl, Bash, VimL, Elisp, Guile, and whatever else comes my way. All I have to do to get a working and isolated dev env is go into that directory and let nix-shell do the rest. On NixOS you also get nixos-container with all its benefits, so I can test whole networks or just spin up some DBs, all behaving exactly the same they'll be once deployed by simply reusing that configuration. Of course it also manages Systemd, so I guess we'll just write yet another function to slap a checksum on those containers and control them as well, it's just a shame that humanity loses so much time reinventing the wheel every few months. On the other hand, there's still a lot of opportunities for start-ups in that space, like https://nixcloud.io https://nixcloud.io, https://www.tweag.io https://www.tweag.io (they just hired Eelco Dolstra so he can work full-time on the Nix core), https://www.packet.net https://www.packet.net (they sponsor almost all our new CI infrastructure), or https://vpsfree.org/ https://vpsfree.org/ (first one to offer their own NixOS based distro specifically made for VPSs), which are all doing well.
- paxys 8y ago
  This is pretty much the reason why Docker is so popular.
- TheAceOfHearts 8y ago
  I use asdf [0] for this. It lets you install and run different versions of any language. I keep a sensible global default if I'm just experimenting with stuff, but for any serious project I'd create a .tool-versions file in its root to specify the exact language version I'm developing and deploying against. [0] https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf
  - mcdan 8y ago
    Agreed I love asdf I even use it on my Jenkins machines cause it’s a nice consistent way to enforce exactly what tool will be used where.
- CMCDragonkai 8y ago
  Just use Nix!
- 101km 8y ago
  You actually want to "brew install pyenv" And then "pyenv install 3.7" or whatever version you're after. Multiple versions of programming languages don't require docker, containers, or Nix, that is utter insanity.
  - matthewmacleod 8y ago
    Not quite utter insanity - isn’t it quite weird that every programming language has its own tool for installing multiple different versions of itself? nvm, pyenv, rbenv, rustup, etc. Maybe it’s nice to think of a common solution to this!
- pmontra 8y ago
  Try a language manager. One I know to work on MacOS is asdf https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf Then, there is something wrong with either brew or MacOS or both because we can install both Python 2 and 3 in parallel on any Linux with the system package manager.
- xorcist 8y ago
  > multiple versions of programming languages & packages to be installed by each user without affecting the system What language runtime do you have in mind? I haven't seen one that doesn't support this. A normal user should not be allowed to affect the system, so installing language runtimes under your home directory is normal and expected. Most languages use an environment variable to point to the packages being used, so it should be possible to have multiple versions installed under your home. This is how the language developers work. They need to have many development versions installed for testing. Containers are useful for a lot of things, but a much more complicated set up than just installing a couple of language runtimes. You should have no problems doing that the normal way.
cagenut 8y ago
This is cool. I'm betting hard on k8s at $dayjob, but I often think of how much less I could be happy with. A simple daemon that used some gossip/broadcast protocol to work out a list of systemd service states would be juuust enough.
- navaati 8y ago
  You mean exactly like CoreOS Fleet ?
  - irq-1 8y ago
    > fleet is no longer actively developed or maintained by CoreOS. CoreOS instead recommends Kubernetes for cluster orchestration. https://coreos.com/fleet/docs/latest/launching-containers-fleet.html https://coreos.com/fleet/docs/latest/launching-containers-fl...
    - nickik 8y ago
      While they no want to do cluster orchestration, as a distributed systemd it can still be useful. The core functionality is there and well tested. No longer actively maintained is a problem, but I think it still has a quite a few users.
- ttwwmm 8y ago
  Have you checked out https://www.habitat.sh/ https://www.habitat.sh/? It has a gossip-based discovery/coordination layer. I've been meaning to give it a try --- I too have Docker fatigue.
deleted 8y ago
[deleted]
yarrel 8y ago
The software that is just a better init system continues to not be.
- donbright 8y ago
  "Because the Portable Service concepts introduces zero new metadata and just builds on existing security and resource bundling features of systemd it's implemented in a set of distinct tools, relatively disconnected from the rest of systemd. " clearly, this is a messy situation. lots of different pieces doing different things in different ways. in order to harmonize this, and simplify it, I believe we need to have systemdd, a daemon for systemd, so that all of it's various pieces can be in one, neat, centralized location.
  - chris_wot 8y ago
    What a ridiculous comment. Keeping these tools relatively decoupled is a good thing.
    - dogecoinbase 8y ago
      I can't speak for the GP, but I believe their comment is parody. Keeping tools decoupled is, of course, very good -- systemd is the antithesis of that norm.
      - chris_wot 8y ago
        Ah, if it was sarcasm then apologies - it's not something that one can easily pick up through text.
        donbright 8y ago
        That is no problem. We had our comments scattered out into separate pieces, each doing their own thing in a non-standard comment idiom. In order to simplify and organize things better, I have created commentd which will combine them all into one comment. To save valuable disk space I have also compressed them and put them in binary format. begin 644 - M0F%S:6PZ($QI<W1E;BP@9&]N)W0@;65N=&EO;B!T:&4@=V%R(2!)(&UE;G1I M;VYE9"!I="!O;F-E+"!B=70@22!T:&EN:R!)(&=O="!A=V%Y('=I=&@@:70@ )86QR:6=H="X* ` end
CuriousSkeptic 8y ago
I just spent a few days trying to navigate all this. Perhaps some one here could enlighten me. If I just want to download some semi-trusted source code. And Configure; make; run; it in an environment where it can’t access anything unless I explicitly white list it (in particular, it should not be allowed to access the internet or any network resources, but I should be able to send http requests to it) Which variant of all these container stuff would make things simplest to set up? Docker seems way overkill (and seems to have the wrong defaults for this need anyways) but handcrafting things with iptables or ip netns seems a bit to low level. (And, well, not “contained”... I would prefer something declarative with automatic setup/teardown) Ubuntu snaps looks like an interesting middle ground. But it also looks completely dead. Any tips? Edit: should probably rtfa before asking. it actually looks like a good fit ;)
- sjellis 8y ago
  If you want the portability of Docker/OCI images without running a service etc., you might take a look at Buildah: https://www.projectatomic.io/blog/2017/06/introducing-buildah/ https://www.projectatomic.io/blog/2017/06/introducing-builda... It's a command-line tool for building and running container images.
- 112233 8y ago
  Not a container, but IF you manage to get Qubes running, it's really easy to create temp VM off fedora or debian that cannot access your other stuffs. You can even swap distro from under your homedir later.
- mongol 8y ago
  I think the systemd containerization features are worthwhile to explore. Both portable services and nspawn. They will be around on most Linux systems "out of the box" which is a big benefit. It will be like bash, maybe not the best shell, but the one that is around and that you can count on.
- leni536 8y ago
  Firejail could be a good fit.
sandGorgon 8y ago
Interestingly, systemd maintainers have consistently refused to make available a Docker compatible version of systemd to run as the init 1. Everyone brings their own here (e.g. http://phusion.github.io/baseimage-docker/ http://phusion.github.io/baseimage-docker/) like supervisord, runit,etc. What would have been ideal is for a docker-compatible systemd to run inside the container. IMHO the maintainers are bent on creating a competing standard (like this one) and don't want to build anything that brings the advantages of systemd to the Docker ecosystem.
- TheDong 8y ago
  > Interestingly, systemd maintainers have consistently refused to make available a Docker compatible version of systemd to run as the init 1. Please provide evidence of that. I will provide counter-evidence, proving that is not the case unless you have very compelling evidence. 1. https://lists.freedesktop.org/archives/systemd-devel/2014-May/019005.html https://lists.freedesktop.org/archives/systemd-devel/2014-Ma... Poettering writes: > To say this explicitly: we are really interested in making sure that systemd runs out-of-the-box in containers, and docker is just one implementation of that. 2. A PR to make systemd-logind work better in a docker container is accepted with no issue: https://github.com/systemd/systemd/pull/4154 https://github.com/systemd/systemd/pull/4154 3. Running systemd as pid1 in docker (privileged or not) works, so clearly the maintainers didn't do a good job of preventing it from happening. (just google, there's plenty of info out there on how to do this) I'm sure there are many other examples too, but this is with a quick glance around. I'd be interested in your reference before you spread such strange FUD.
  - sandGorgon 8y ago
    there you go - https://bugs.freedesktop.org/show_bug.cgi?id=88986 https://bugs.freedesktop.org/show_bug.cgi?id=88986 this is a bug I filed. Lennart's reply: We had support for this in systemd in the initial versions, but since nobody was using and testing this, and the semantics were different from the system version in subtle ways we removed support for it. Also note that systemd requires kernel 3.7 as minimal version right now. You cannot run it on older kernels, and hence really old operating systems anyway. Sorry, but this is nothing we want or can support! I admit that it is 2 years old - and if it has changed, then that would be nice. But there is an explicit reply there.. and I'm not spreading FUD. There's nothing more that I would love than to replace all the supervisord stuff in Docker with systemd.
king_nothing 8y ago
I’ll take s8 and some sort of log-structured FS for local buffering of structured messages logged to stderr, TYVM. No pid files, no lock files, no log files to rotate awkwardly and no unstructured log files. PS: log events shouldn’t be perceived of as lines of text but structured data messages, so that they don’t need special parsing.
InTheArena 8y ago
Oh yeah. Systemd and containers. Two technologies certain militant technical users love to complaint about :-) Do we really need a non-OCi image format and tool chain here?! I get immutability benefits, district benefits, and isolation. having a completely separate toolchain makes no sense to me... Containers will be in this role, but fragmentation will only make that process slower and more painful. I for one salute our future k8s kubelet overlords.
- TheDong 8y ago
  > Do we really need a non-OCi image format and tool chain here? This doesn't use a new competing image format, but rather uses disk images or tarballs, which have existed longer than OCI has. The toolchain also predates OCI since the underlying technology is really basically nspawn (and various other systemd.service options), which have been a part of systemd since before OCI stabilized. This is simply a new coat of paint on something that has been there for a while. > Containers will be in this role, but fragmentation will only make that process slower and more painful. Sure, so we shouldn't have let docker do anything and instead used LXC, since LXC already did containers, already had its own format and toolchain, etc. That's the exact same argument you're making now. OCI is also not really a good standard. It has exactly one usable implementation (and only in Go, not even C where it's easy to link against and use it from other languages), and since systemd has already defined various things for services (such as seccomp filters etc) which overlap with what OCI does, it makes almost no sense for systemd to begin using it.
  - cyphar 8y ago
    I actually think more people should have adopted LXC. The engineering behind LXC is beyond remarkable and it's a shame that it got such a bad wrap in the early days of Docker. I've worked with some of the LXC folks and they're all incredibly sharp and have a much wider breadth of work than just LXC. With regards to image formats, I actually think that we do need to improve the image format significantly and not just stick to tried and tested stuff. Tarballs are (quite frankly) simply the most awful format to use for container images (disk images are a close second). It's a shame that everyone uses them. I am working on a blog post to better explain what I mean by this. And yeah, runc being written in Go is probably one of the most frustrating things about working on it. I often say that one of the worst things you could decide to write in Go is a container runtime -- it's a little bit odd that people keep doing exactly that. runc actually isn't entirely written in Go -- it has to have a fairly substantial amount of C because the Go runtime cannot play well with the delicate dance required to set up a container properly. All of that being said, containers really did need to be standardised otherwise the container wars wouldn't have ended as nicely as they did. But that's not to say that the OCI doesn't have issues. It definitely does, but I'm hoping they can be fixed over time. [ I am one of the maintainers of the "exactly one usable" OCI implementation, and have been working on OCI stuff since its inception. ]
luckycharms810 8y ago
It’s astonishing to me that people would argue that docker is some how a simpler solution. Some would happily maintain a container registry, constantly worry about the order of operations in your docker file, create a build pipeline in a format incompatible with non docker use cases, spend time understanding how dockers bridge interacts with multiple Network controllers over learning how your operating system works.
- deleted 8y ago
  [deleted]
- viraptor 8y ago
  There's docker and there's docker. Docker as a local container builder/runner is simple. Docker as a deployment / networking / scheduling service is not. Not everyone is using Docker for more than "run this script in an environment I want".
- gsich 8y ago
  >It’s astonishing to me that people would argue that docker is some how a simpler solution. Some say docker is the new "curl|bash" ;)
  - TheDong 8y ago
    Really now? Here's a docker container with a shell you can type anything in to: https://contained.af/ https://contained.af/ If you manage to tell me what's in /etc/os-release on the machine (not inside the container), I'll eat my lunch.
    - jrs95 8y ago
      So...if we don’t, will you ever eat lunch again?
    - progval 8y ago
      The issue with curl|bash is not just the security (you can run curl|bash as an unprivileged user). It's also the fact that it generally doesn't do anything to integrate well with your system, and just pull all its dependencies in a folder and never update it afterward.
geertj 8y ago
Does the portable service run in a PID namespace? Couldn't find the response in the article.
- CameronNemo 8y ago
  I don't know. But if it does the service would need to reap zombies or systemd would need to spawn yet another instance of itself inside the container (like Rocket does).

152 points by type0 8y ago

Yet another tool for containerization technology :-) While this is interesting, the containers that I'd appreciate, are: multiple versions of programming languages & packages to be installed by each user without affecting the system globally. Every language has their own separate answer (virtualenv/venv, rbenv, node version manager etc). Just today I struggled with "brew install python3" which complained with "Error: python 2.7.14_2 is already installed" and the only option offered was to upgrade. Whereas what I wanted was parallel installations of python 2.7 and python3. :-/

viraptor 8y ago
If you want to preserve python 2, you `brew install python@2`. You can have both that way. (Updated homebrew required)
vbezhenar 8y ago
There shouldn't be need for containers to use different versions of programming toolkits. I'm Java developer and in our world it's extremely easy: JDK is just a directory with bunch of files. I can have (and I actually have) around 10 versions of JDKs, from Java 1.1 to Java 10. There's no need to install anything, just extract the archive. To use a particular Java usually JAVA_HOME environment variable is enough. That works for every operating system from Linux to Windows and macOS identically. That's why I don't ever use distro Java, it's a mess, just download Oracle JDK and start work.
- foota 8y ago
  Java's missing a decent packaging solution though.
  - shub 8y ago
    I've never had a problem using wars or fat jars for server deployments. Client distribution is another matter.
  - gitgud 8y ago
    Gradle works pretty well in my opinion
  - jamietanna 8y ago
    I've not had any issues with Maven before
clhodapp 8y ago
That's exactly what Nix does! Nix is wonderful tech that unfortunately seems doomed to low adoption due to poor usability. My belief is that this stems the fact that maintaining the package tree is obscenely costly of contributor time, leaving almost no time or energy to make progress on "niceties" like improving the core CLI experience and the continuous integration system.
- rspeer 8y ago
  I once started trying to learn about Nix. All the up-to-date documentation I could find, however, was about NixOS. I would like to use Nix just to manage my local development environment on Ubuntu, not to replace my entire operating system. Is that something the developers are still interested in? Is there documentation?
  - jumblemuddle 8y ago
    Have a look here: https://nixos.org/nix/manual/ https://nixos.org/nix/manual/ There's a dropdown in the top-left corner to switch between nix-related projects. (NixOS, NixOps, etc.)
    - andreabedini 8y ago
      I found nix-pills more useful than the manual: https://nixos.org/nixos/nix-pills/index.html https://nixos.org/nixos/nix-pills/index.html
- manveru 8y ago
  The Nix team (in particular grahamc with https://github.com/NixOS/ofborg https://github.com/NixOS/ofborg) is working hard on making CI better . But it's true that the on-boarding experience and CLI could be much improved. It's quite frustrating to me, because after I've been using Nix & NixOS for about 3 years now, there's no way I'd go back to the insanity out there. I never had the amount of stability, control, and predictability on any other platform, from configuring my OS, editors, WM, shell, random ricing, or building VMs, Servers, Docker containers, production deploys, development environments, there's really not much Nix can't handle. But getting familiar with Nix took some serious effort I'm sure not many people can afford. I begin to understand how frustrated people in the Lisp and Smalltalk communities must feel while people slowly reinvent everything they've enjoyed for decades (and Nix is still in its teens). Teaching people about the value of having just a single package manager, no matter what OS or language they're using, is mostly futile because there's so much value attributed to being "mainstream" that now we have hundreds of mainstream package managers with varying degrees of sophistication, security, predictability, and ability to _not_ try and take over your system. I use it to build and develop Go, Ruby, JS, Crystal, Elm, Mint, Haskell, Perl, Bash, VimL, Elisp, Guile, and whatever else comes my way. All I have to do to get a working and isolated dev env is go into that directory and let nix-shell do the rest. On NixOS you also get nixos-container with all its benefits, so I can test whole networks or just spin up some DBs, all behaving exactly the same they'll be once deployed by simply reusing that configuration. Of course it also manages Systemd, so I guess we'll just write yet another function to slap a checksum on those containers and control them as well, it's just a shame that humanity loses so much time reinventing the wheel every few months. On the other hand, there's still a lot of opportunities for start-ups in that space, like https://nixcloud.io https://nixcloud.io, https://www.tweag.io https://www.tweag.io (they just hired Eelco Dolstra so he can work full-time on the Nix core), https://www.packet.net https://www.packet.net (they sponsor almost all our new CI infrastructure), or https://vpsfree.org/ https://vpsfree.org/ (first one to offer their own NixOS based distro specifically made for VPSs), which are all doing well.
paxys 8y ago
This is pretty much the reason why Docker is so popular.
TheAceOfHearts 8y ago
I use asdf [0] for this. It lets you install and run different versions of any language. I keep a sensible global default if I'm just experimenting with stuff, but for any serious project I'd create a .tool-versions file in its root to specify the exact language version I'm developing and deploying against. [0] https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf
- mcdan 8y ago
  Agreed I love asdf I even use it on my Jenkins machines cause it’s a nice consistent way to enforce exactly what tool will be used where.
CMCDragonkai 8y ago
Just use Nix!
101km 8y ago
You actually want to "brew install pyenv" And then "pyenv install 3.7" or whatever version you're after. Multiple versions of programming languages don't require docker, containers, or Nix, that is utter insanity.
- matthewmacleod 8y ago
  Not quite utter insanity - isn’t it quite weird that every programming language has its own tool for installing multiple different versions of itself? nvm, pyenv, rbenv, rustup, etc. Maybe it’s nice to think of a common solution to this!
pmontra 8y ago
Try a language manager. One I know to work on MacOS is asdf https://github.com/asdf-vm/asdf https://github.com/asdf-vm/asdf Then, there is something wrong with either brew or MacOS or both because we can install both Python 2 and 3 in parallel on any Linux with the system package manager.
xorcist 8y ago
> multiple versions of programming languages & packages to be installed by each user without affecting the system What language runtime do you have in mind? I haven't seen one that doesn't support this. A normal user should not be allowed to affect the system, so installing language runtimes under your home directory is normal and expected. Most languages use an environment variable to point to the packages being used, so it should be possible to have multiple versions installed under your home. This is how the language developers work. They need to have many development versions installed for testing. Containers are useful for a lot of things, but a much more complicated set up than just installing a couple of language runtimes. You should have no problems doing that the normal way.

cagenut 8y ago

This is cool. I'm betting hard on k8s at $dayjob, but I often think of how much less I could be happy with. A simple daemon that used some gossip/broadcast protocol to work out a list of systemd service states would be juuust enough.

navaati 8y ago
You mean exactly like CoreOS Fleet ?
- irq-1 8y ago
  > fleet is no longer actively developed or maintained by CoreOS. CoreOS instead recommends Kubernetes for cluster orchestration. https://coreos.com/fleet/docs/latest/launching-containers-fleet.html https://coreos.com/fleet/docs/latest/launching-containers-fl...
  - nickik 8y ago
    While they no want to do cluster orchestration, as a distributed systemd it can still be useful. The core functionality is there and well tested. No longer actively maintained is a problem, but I think it still has a quite a few users.
ttwwmm 8y ago
Have you checked out https://www.habitat.sh/ https://www.habitat.sh/? It has a gossip-based discovery/coordination layer. I've been meaning to give it a try --- I too have Docker fatigue.

deleted 8y ago

[deleted]

yarrel 8y ago

The software that is just a better init system continues to not be.

donbright 8y ago
"Because the Portable Service concepts introduces zero new metadata and just builds on existing security and resource bundling features of systemd it's implemented in a set of distinct tools, relatively disconnected from the rest of systemd. " clearly, this is a messy situation. lots of different pieces doing different things in different ways. in order to harmonize this, and simplify it, I believe we need to have systemdd, a daemon for systemd, so that all of it's various pieces can be in one, neat, centralized location.
- chris_wot 8y ago
  What a ridiculous comment. Keeping these tools relatively decoupled is a good thing.
  - dogecoinbase 8y ago
    I can't speak for the GP, but I believe their comment is parody. Keeping tools decoupled is, of course, very good -- systemd is the antithesis of that norm.
    - chris_wot 8y ago
      Ah, if it was sarcasm then apologies - it's not something that one can easily pick up through text.
      - donbright 8y ago
        That is no problem. We had our comments scattered out into separate pieces, each doing their own thing in a non-standard comment idiom. In order to simplify and organize things better, I have created commentd which will combine them all into one comment. To save valuable disk space I have also compressed them and put them in binary format. begin 644 - M0F%S:6PZ($QI<W1E;BP@9&]N)W0@;65N=&EO;B!T:&4@=V%R(2!)(&UE;G1I M;VYE9"!I="!O;F-E+"!B=70@22!T:&EN:R!)(&=O="!A=V%Y('=I=&@@:70@ )86QR:6=H="X* ` end

CuriousSkeptic 8y ago

I just spent a few days trying to navigate all this. Perhaps some one here could enlighten me. If I just want to download some semi-trusted source code. And Configure; make; run; it in an environment where it can’t access anything unless I explicitly white list it (in particular, it should not be allowed to access the internet or any network resources, but I should be able to send http requests to it) Which variant of all these container stuff would make things simplest to set up? Docker seems way overkill (and seems to have the wrong defaults for this need anyways) but handcrafting things with iptables or ip netns seems a bit to low level. (And, well, not “contained”... I would prefer something declarative with automatic setup/teardown) Ubuntu snaps looks like an interesting middle ground. But it also looks completely dead. Any tips? Edit: should probably rtfa before asking. it actually looks like a good fit ;)

sjellis 8y ago
If you want the portability of Docker/OCI images without running a service etc., you might take a look at Buildah: https://www.projectatomic.io/blog/2017/06/introducing-buildah/ https://www.projectatomic.io/blog/2017/06/introducing-builda... It's a command-line tool for building and running container images.
112233 8y ago
Not a container, but IF you manage to get Qubes running, it's really easy to create temp VM off fedora or debian that cannot access your other stuffs. You can even swap distro from under your homedir later.
mongol 8y ago
I think the systemd containerization features are worthwhile to explore. Both portable services and nspawn. They will be around on most Linux systems "out of the box" which is a big benefit. It will be like bash, maybe not the best shell, but the one that is around and that you can count on.
leni536 8y ago
Firejail could be a good fit.

sandGorgon 8y ago

Interestingly, systemd maintainers have consistently refused to make available a Docker compatible version of systemd to run as the init 1. Everyone brings their own here (e.g. http://phusion.github.io/baseimage-docker/ http://phusion.github.io/baseimage-docker/) like supervisord, runit,etc. What would have been ideal is for a docker-compatible systemd to run inside the container. IMHO the maintainers are bent on creating a competing standard (like this one) and don't want to build anything that brings the advantages of systemd to the Docker ecosystem.

TheDong 8y ago
> Interestingly, systemd maintainers have consistently refused to make available a Docker compatible version of systemd to run as the init 1. Please provide evidence of that. I will provide counter-evidence, proving that is not the case unless you have very compelling evidence. 1. https://lists.freedesktop.org/archives/systemd-devel/2014-May/019005.html https://lists.freedesktop.org/archives/systemd-devel/2014-Ma... Poettering writes: > To say this explicitly: we are really interested in making sure that systemd runs out-of-the-box in containers, and docker is just one implementation of that. 2. A PR to make systemd-logind work better in a docker container is accepted with no issue: https://github.com/systemd/systemd/pull/4154 https://github.com/systemd/systemd/pull/4154 3. Running systemd as pid1 in docker (privileged or not) works, so clearly the maintainers didn't do a good job of preventing it from happening. (just google, there's plenty of info out there on how to do this) I'm sure there are many other examples too, but this is with a quick glance around. I'd be interested in your reference before you spread such strange FUD.
- sandGorgon 8y ago
  there you go - https://bugs.freedesktop.org/show_bug.cgi?id=88986 https://bugs.freedesktop.org/show_bug.cgi?id=88986 this is a bug I filed. Lennart's reply: We had support for this in systemd in the initial versions, but since nobody was using and testing this, and the semantics were different from the system version in subtle ways we removed support for it. Also note that systemd requires kernel 3.7 as minimal version right now. You cannot run it on older kernels, and hence really old operating systems anyway. Sorry, but this is nothing we want or can support! I admit that it is 2 years old - and if it has changed, then that would be nice. But there is an explicit reply there.. and I'm not spreading FUD. There's nothing more that I would love than to replace all the supervisord stuff in Docker with systemd.

king_nothing 8y ago

I’ll take s8 and some sort of log-structured FS for local buffering of structured messages logged to stderr, TYVM. No pid files, no lock files, no log files to rotate awkwardly and no unstructured log files. PS: log events shouldn’t be perceived of as lines of text but structured data messages, so that they don’t need special parsing.

InTheArena 8y ago

Oh yeah. Systemd and containers. Two technologies certain militant technical users love to complaint about :-) Do we really need a non-OCi image format and tool chain here?! I get immutability benefits, district benefits, and isolation. having a completely separate toolchain makes no sense to me... Containers will be in this role, but fragmentation will only make that process slower and more painful. I for one salute our future k8s kubelet overlords.

TheDong 8y ago
> Do we really need a non-OCi image format and tool chain here? This doesn't use a new competing image format, but rather uses disk images or tarballs, which have existed longer than OCI has. The toolchain also predates OCI since the underlying technology is really basically nspawn (and various other systemd.service options), which have been a part of systemd since before OCI stabilized. This is simply a new coat of paint on something that has been there for a while. > Containers will be in this role, but fragmentation will only make that process slower and more painful. Sure, so we shouldn't have let docker do anything and instead used LXC, since LXC already did containers, already had its own format and toolchain, etc. That's the exact same argument you're making now. OCI is also not really a good standard. It has exactly one usable implementation (and only in Go, not even C where it's easy to link against and use it from other languages), and since systemd has already defined various things for services (such as seccomp filters etc) which overlap with what OCI does, it makes almost no sense for systemd to begin using it.
- cyphar 8y ago
  I actually think more people should have adopted LXC. The engineering behind LXC is beyond remarkable and it's a shame that it got such a bad wrap in the early days of Docker. I've worked with some of the LXC folks and they're all incredibly sharp and have a much wider breadth of work than just LXC. With regards to image formats, I actually think that we do need to improve the image format significantly and not just stick to tried and tested stuff. Tarballs are (quite frankly) simply the most awful format to use for container images (disk images are a close second). It's a shame that everyone uses them. I am working on a blog post to better explain what I mean by this. And yeah, runc being written in Go is probably one of the most frustrating things about working on it. I often say that one of the worst things you could decide to write in Go is a container runtime -- it's a little bit odd that people keep doing exactly that. runc actually isn't entirely written in Go -- it has to have a fairly substantial amount of C because the Go runtime cannot play well with the delicate dance required to set up a container properly. All of that being said, containers really did need to be standardised otherwise the container wars wouldn't have ended as nicely as they did. But that's not to say that the OCI doesn't have issues. It definitely does, but I'm hoping they can be fixed over time. [ I am one of the maintainers of the "exactly one usable" OCI implementation, and have been working on OCI stuff since its inception. ]

luckycharms810 8y ago

It’s astonishing to me that people would argue that docker is some how a simpler solution. Some would happily maintain a container registry, constantly worry about the order of operations in your docker file, create a build pipeline in a format incompatible with non docker use cases, spend time understanding how dockers bridge interacts with multiple Network controllers over learning how your operating system works.

deleted 8y ago
[deleted]
viraptor 8y ago
There's docker and there's docker. Docker as a local container builder/runner is simple. Docker as a deployment / networking / scheduling service is not. Not everyone is using Docker for more than "run this script in an environment I want".
gsich 8y ago
>It’s astonishing to me that people would argue that docker is some how a simpler solution. Some say docker is the new "curl|bash" ;)
- TheDong 8y ago
  Really now? Here's a docker container with a shell you can type anything in to: https://contained.af/ https://contained.af/ If you manage to tell me what's in /etc/os-release on the machine (not inside the container), I'll eat my lunch.
  - jrs95 8y ago
    So...if we don’t, will you ever eat lunch again?
  - progval 8y ago
    The issue with curl|bash is not just the security (you can run curl|bash as an unprivileged user). It's also the fact that it generally doesn't do anything to integrate well with your system, and just pull all its dependencies in a folder and never update it afterward.

geertj 8y ago

Does the portable service run in a PID namespace? Couldn't find the response in the article.

CameronNemo 8y ago
I don't know. But if it does the service would need to reap zombies or systemd would need to spawn yet another instance of itself inside the container (like Rocket does).