7 ms·

Exactly our situation. I'd love to be able to build and deploy docker images, but the "docker is root" thing makes this impossible in our shop.

curiousgal 9y ago
Have you looked into Singularity[0]? It supports Docker containers. 0.https://singularity.lbl.gov/index.html https://singularity.lbl.gov/index.html
- chris_st 9y ago
  Thanks! I'll check that out.
- ihnorton 9y ago
  IIUC building Singularity containers still requires root.
  - stonogo 9y ago
    You build them on your machine, or on the hub (straight from github), and then you don't need root to run them.
    - cyphar 9y ago
      Singularity in general requires root, it's just in the form of setuid helpers. Now, you can force it to not require root but from memory there are a lot of caveats with using it that way. Unprivileged LXC is much more full-featured. And obviously rootless runc works great as well (though I'm a bit biased of course).
      - stonogo 9y ago
        Singularity is capable of using setuid helpers, but by default it uses user namespaces (the USERNS kernel feature) and does not need the setuid helpers. Lots of things are 'more full-featured' and none of them work well in an HPC context, where individual user jobs may need to be staged carefully.
        cyphar 9y ago
        > Lots of things are 'more full-featured' and none of them work well in an HPC context, where individual user jobs may need to be staged carefully. After long discussions with the Singularity folks I've come to the conclusion that the only special features that Singularity has are: * Their distribution format is a binary containing a loopback block device, allowing you to have "a single thing to distribute" without concern for having to extract archives (because in theory archive extraction can be lossy or buggy). The downside is it requires (real) root to mount or a setuid helper, because mounting loopback is privileged because it opens the door to very worrying root escalation exploits. When running without setuid helpers I'm not sure how they have worked around this problem, but it probably involves extraction of some sort (invalidating the point of having a more complicated loopback device over just a tar archive). * It has integration with common tools that people use in HPC. I can't quite remember the names of those tools at the moment, but some folks I know wrote some simple wrappers around runc to make it work with those tools -- so it's just a matter of implementing wrappers and not anything more fundamental than that. Aside from those two things, there's nothing particularly different about Singularity from any other low-level container runtime (aside from the way they market themselves). A lot of people quote that Singularity lets you use your host user inside the container, but this is just a fairly simple trick where the /etc/passwd of your host is bind-mounted into the container and then everything "looks" like you're still on your host. People think this is a security feature, it isn't (in fact you could potentially argue it's slightly less secure than just running as an unprivileged user without providing a view into the host's configuration). If you really wanted this feature, you could implement it with runc's or LXC's hooks in a few minutes.
- physicsguy 9y ago
  It's great if it works, but there are many cases where you can't just take a Docker container and apply it to work in Singularity. I even found this Blog Post this morning that went into some of it. https://www.sylabs.io/2018/04/singularity-compatibility-with-docker-containers/ https://www.sylabs.io/2018/04/singularity-compatibility-with...
cyphar 9y ago
I'm working on [1] and the plan is to make this work much easier for people, so they don't need to hit the "docker is root" problem anymore -- this was a pain for me a few years ago when I was a university student trying to run some code on a shared computing cluster. There's still lots of stuff left to do (like unprivileged bridge networking with TAP and similarly fun things like that). [1]: https://rootlesscontaine.rs/ https://rootlesscontaine.rs/

curiousgal 9y ago

Have you looked into Singularity[0]? It supports Docker containers. 0.https://singularity.lbl.gov/index.html https://singularity.lbl.gov/index.html

chris_st 9y ago
Thanks! I'll check that out.
ihnorton 9y ago
IIUC building Singularity containers still requires root.
- stonogo 9y ago
  You build them on your machine, or on the hub (straight from github), and then you don't need root to run them.
  - cyphar 9y ago
    Singularity in general requires root, it's just in the form of setuid helpers. Now, you can force it to not require root but from memory there are a lot of caveats with using it that way. Unprivileged LXC is much more full-featured. And obviously rootless runc works great as well (though I'm a bit biased of course).
    - stonogo 9y ago
      Singularity is capable of using setuid helpers, but by default it uses user namespaces (the USERNS kernel feature) and does not need the setuid helpers. Lots of things are 'more full-featured' and none of them work well in an HPC context, where individual user jobs may need to be staged carefully.
      - cyphar 9y ago
        > Lots of things are 'more full-featured' and none of them work well in an HPC context, where individual user jobs may need to be staged carefully. After long discussions with the Singularity folks I've come to the conclusion that the only special features that Singularity has are: * Their distribution format is a binary containing a loopback block device, allowing you to have "a single thing to distribute" without concern for having to extract archives (because in theory archive extraction can be lossy or buggy). The downside is it requires (real) root to mount or a setuid helper, because mounting loopback is privileged because it opens the door to very worrying root escalation exploits. When running without setuid helpers I'm not sure how they have worked around this problem, but it probably involves extraction of some sort (invalidating the point of having a more complicated loopback device over just a tar archive). * It has integration with common tools that people use in HPC. I can't quite remember the names of those tools at the moment, but some folks I know wrote some simple wrappers around runc to make it work with those tools -- so it's just a matter of implementing wrappers and not anything more fundamental than that. Aside from those two things, there's nothing particularly different about Singularity from any other low-level container runtime (aside from the way they market themselves). A lot of people quote that Singularity lets you use your host user inside the container, but this is just a fairly simple trick where the /etc/passwd of your host is bind-mounted into the container and then everything "looks" like you're still on your host. People think this is a security feature, it isn't (in fact you could potentially argue it's slightly less secure than just running as an unprivileged user without providing a view into the host's configuration). If you really wanted this feature, you could implement it with runc's or LXC's hooks in a few minutes.
physicsguy 9y ago
It's great if it works, but there are many cases where you can't just take a Docker container and apply it to work in Singularity. I even found this Blog Post this morning that went into some of it. https://www.sylabs.io/2018/04/singularity-compatibility-with-docker-containers/ https://www.sylabs.io/2018/04/singularity-compatibility-with...

cyphar 9y ago

I'm working on [1] and the plan is to make this work much easier for people, so they don't need to hit the "docker is root" problem anymore -- this was a pain for me a few years ago when I was a university student trying to run some code on a shared computing cluster. There's still lots of stuff left to do (like unprivileged bridge networking with TAP and similarly fun things like that). [1]: https://rootlesscontaine.rs/ https://rootlesscontaine.rs/