4 ms·

I started the rootless containers project and have had long email threads with the Singularity folks. > Which is why we use Singularity[0]. Apologies if I’m mi

by cyphar 8y ago

I started the rootless containers project and have had long email threads with the Singularity folks. > Which is why we use Singularity[0]. Apologies if I’m missing something, but to me this problem they’re trying to solve, has already been solved. Singularity ‘requires’ root or elevated privileges to create and modify containers, Well, Singularity came out at the same time as runc's rootless containers support was developed -- so at the time it wasn't solved at all. It required (and still requires for several operations and features) root privileges in order to work (such as suid helpers in addition to explicit root requirements). It's not acceptable for a usecase where you cannot run anything as root (not even installation scripts). That was the use-case, and rootless runc (and now thanks to Akihiro and Guiseppe, rootless Docker and Kubernetes) in theory can now work without the need for any suid helpers -- though these days there is an increasing usage of newuidmap and newgidmap (which isn't mandated for the rootless runc implementation). At the time, LXC's unprivileged containers was the closest thing and it had a few (mostly optional) suid binaries -- I wanted absolutely none. > but running them is done in the user’s namespace with that user’s permission set. And privileges can’t be elevated during the containers runtime as there are kernel level catches that prevent sudo and su from working inside the container. This last part really isn't revolutionary at all. All it really takes is a syscall and three files to write to. The hard part is getting everything else to work without root privileges.

mroche 8y ago
Are you a dev on binctr? Is so, the following statements are based off an assumption of yes. > Well, Singularity came out at the same time as runc's rootless containers support was developed -- so at the time it wasn't solved at all. It required (and still requires for several operations and features) root privileges in order to work (such as suid helpers in addition to explicit root requirements). Alright, but I wasn’t asking about a “back then” scenario, I was mostly trying to figure out what this offers _today_. From a Singularity end-user’s perspective, there’s only one function that requires elevated permissions, and that’s building. > It's not acceptable for a usecase where you cannot run anything as root (not even installation scripts). ... At the time, LXC's unprivileged containers was the closest thing and it had a few (mostly optional) suid binaries -- I wanted absolutely none. Now I understand what this offers and it’s overarching goal is. In my specific situation (which is the angle I’m looking at this from), that’s not really a big deal due to the way we handle global and local software installation. I understand this is not a universal situation. > This last part really isn't revolutionary at all. I don’t think I made it sound like it was revolutionary, I was just sharing what a commonly used tool in HPC does for a security measure. Based off of the binctr README: > Create fully static, including rootfs embedded, binaries that pop you directly into a container. _Can be run by an unprivileged user._ My main confusion stems from that. In comparison to Singularity, it doesn’t seem to offer anything new or noteworthy that would make me take a second look at it. The first phrase just makes me think I’m saving myself a few keystrokes as I wouldn’t have to type `singularity shell <container>` to shell into my container of choice. It just looks like yet another container solution. I still find these things cool (and their above my head in dev terms), but it feels like programming languages, a new one popping up every so often. I’m primarily looking to see what advantages (or differences in approach) this offers over Singularity, Shifter, or Charliecloud, etc for my context. At least in terms of security and efficiency.
- cpuguy83 8y ago
  binctr is like 3 years old.
  - mroche 8y ago
    Singularity started up in late 2015 started seeing widespread adoption in HPC centers by late 2016. A product for the industry by the industry. In no amount of container research I’ve done have I ever seen binctr mentioned in discussion or as a link from a search engine. Today is the first time I’ve come across it. Age doesn’t mean much for modern search results and isn’t the only factor for adoption. And it certainly doesn’t explain its featuresets or security.
    - cpuguy83 8y ago
      Ok...? No one said anything like this. Just saying binctr is not some new project just hitting the scene.
      - mroche 8y ago
        > Just saying binctr is not some new project just hitting the scene. Gotcha. You’re original statement left a lot of room for interpretation, I wasn’t sure what you were suggesting. I didn’t mean to imply binctr was a new project, I meant it relative to me as I’ve never come across it before in comparison to the well known established solutions that exist today. The more “household” project names. I probably should have made that more clear.
        pcj-github 8y ago
        I'd add that a ratio of ~2000 stars to ~50 forks suggests that the blog author is very popular person and that almost no-one is using binctr for serious work. Nonetheless, I starred it as a code reference for interacting with the docker codebase.
- cyphar 8y ago
  > Are you a dev on binctr? No, but I'm a maintainer of runc and implemented all of the core features that binctr uses (or rather, that Jessie hacked together for a proof of concept). I've also implemented rootless support in quite a few other tools to the point where now an unprivileged user can download, extract, and run a rootless container. In addition, I'm working with some other folks who joined later to get Kubernetes (and Docker) to be completely rootless. > From a Singularity end-user’s perspective, there’s only one function that requires elevated permissions, and that’s building. (Most forms of) execution still requires setuid binaries, which means that you have to have root permissions in order to install Singularity. You can use rootless containers as a completely unprivilged user (meaning that if you have unprivileged shell access to a random box you can use containers). Singularity cannot do this. > I was just sharing what a commonly used tool in HPC does for a security measure. My point was that basically any container runtime can do what you described. > I’m primarily looking to see what advantages (or differences in approach) this offers over Singularity, Shifter, or Charliecloud, etc for my context. At least in terms of security and efficiency. Many of those tools require setuid binaries that are developed as part of their container runtime -- which means that all of the security is contingent in no vulnerabilities in their setuid binaries (something that is hard to do even for the developers of programs like sudo). Rootless containers are a project that requires absolutely no privileged codepaths for any container operation (though these days you can optionally use standard setuid binaries -- by default they are not used). This is something that none of the projects you listed (as far as I'm aware) can do.

by cyphar 8y ago

mroche 8y ago

Are you a dev on binctr? Is so, the following statements are based off an assumption of yes. > Well, Singularity came out at the same time as runc's rootless containers support was developed -- so at the time it wasn't solved at all. It required (and still requires for several operations and features) root privileges in order to work (such as suid helpers in addition to explicit root requirements). Alright, but I wasn’t asking about a “back then” scenario, I was mostly trying to figure out what this offers _today_. From a Singularity end-user’s perspective, there’s only one function that requires elevated permissions, and that’s building. > It's not acceptable for a usecase where you cannot run anything as root (not even installation scripts). ... At the time, LXC's unprivileged containers was the closest thing and it had a few (mostly optional) suid binaries -- I wanted absolutely none. Now I understand what this offers and it’s overarching goal is. In my specific situation (which is the angle I’m looking at this from), that’s not really a big deal due to the way we handle global and local software installation. I understand this is not a universal situation. > This last part really isn't revolutionary at all. I don’t think I made it sound like it was revolutionary, I was just sharing what a commonly used tool in HPC does for a security measure. Based off of the binctr README: > Create fully static, including rootfs embedded, binaries that pop you directly into a container. _Can be run by an unprivileged user._ My main confusion stems from that. In comparison to Singularity, it doesn’t seem to offer anything new or noteworthy that would make me take a second look at it. The first phrase just makes me think I’m saving myself a few keystrokes as I wouldn’t have to type `singularity shell <container>` to shell into my container of choice. It just looks like yet another container solution. I still find these things cool (and their above my head in dev terms), but it feels like programming languages, a new one popping up every so often. I’m primarily looking to see what advantages (or differences in approach) this offers over Singularity, Shifter, or Charliecloud, etc for my context. At least in terms of security and efficiency.

cpuguy83 8y ago
binctr is like 3 years old.
- mroche 8y ago
  Singularity started up in late 2015 started seeing widespread adoption in HPC centers by late 2016. A product for the industry by the industry. In no amount of container research I’ve done have I ever seen binctr mentioned in discussion or as a link from a search engine. Today is the first time I’ve come across it. Age doesn’t mean much for modern search results and isn’t the only factor for adoption. And it certainly doesn’t explain its featuresets or security.
  - cpuguy83 8y ago
    Ok...? No one said anything like this. Just saying binctr is not some new project just hitting the scene.
    - mroche 8y ago
      > Just saying binctr is not some new project just hitting the scene. Gotcha. You’re original statement left a lot of room for interpretation, I wasn’t sure what you were suggesting. I didn’t mean to imply binctr was a new project, I meant it relative to me as I’ve never come across it before in comparison to the well known established solutions that exist today. The more “household” project names. I probably should have made that more clear.
      - pcj-github 8y ago
        I'd add that a ratio of ~2000 stars to ~50 forks suggests that the blog author is very popular person and that almost no-one is using binctr for serious work. Nonetheless, I starred it as a code reference for interacting with the docker codebase.
cyphar 8y ago
> Are you a dev on binctr? No, but I'm a maintainer of runc and implemented all of the core features that binctr uses (or rather, that Jessie hacked together for a proof of concept). I've also implemented rootless support in quite a few other tools to the point where now an unprivileged user can download, extract, and run a rootless container. In addition, I'm working with some other folks who joined later to get Kubernetes (and Docker) to be completely rootless. > From a Singularity end-user’s perspective, there’s only one function that requires elevated permissions, and that’s building. (Most forms of) execution still requires setuid binaries, which means that you have to have root permissions in order to install Singularity. You can use rootless containers as a completely unprivilged user (meaning that if you have unprivileged shell access to a random box you can use containers). Singularity cannot do this. > I was just sharing what a commonly used tool in HPC does for a security measure. My point was that basically any container runtime can do what you described. > I’m primarily looking to see what advantages (or differences in approach) this offers over Singularity, Shifter, or Charliecloud, etc for my context. At least in terms of security and efficiency. Many of those tools require setuid binaries that are developed as part of their container runtime -- which means that all of the security is contingent in no vulnerabilities in their setuid binaries (something that is hard to do even for the developers of programs like sudo). Rootless containers are a project that requires absolutely no privileged codepaths for any container operation (though these days you can optionally use standard setuid binaries -- by default they are not used). This is something that none of the projects you listed (as far as I'm aware) can do.