6 ms·

It should be said that Singularity does actually rely on setuid scripts to actually work with images.

mroche 8y ago
Is there anyway to do what it does without them? Genuine question, there’s still a lot about Linux system/user space that I’m not up to snuff on yet (part of a team here, and I don’t handle users, mostly tools). I’m trying to compare/contrast the goals of binctr to existing solutions that appear to do the same or similar thing(s) already, so I’m open to learning something new today! The only new thing I’m seeing from this solution is the ability to create containers as an unprivileged user, which for most scenarios I tend to view as a non-issue. Or am I misunderstanding that?
- symkat 8y ago
  > The only new thing I’m seeing from this solution is the ability to create containers as an unprivileged user, which for most scenarios I tend to view as a non-issue. Or am I misunderstanding that? I believe you're mislabeling it as a non-issue, it seems to be the core problem they're trying to solve: From the blog[0]: > Yes, we all know that containers run unprivileged processes; but creating and running the containers themselves requires root privileges at some point. > What is the awesome sauce we all gain from this? > Well judging by the original GitHub issue about unprivileged runc containers, the largest group of commenters is from the scientific community who are restricted to not run certain programs as root. [0]: https://blog.jessfraz.com/post/getting-towards-real-sandbox-containers/ https://blog.jessfraz.com/post/getting-towards-real-sandbox-...
  - mroche 8y ago
    I did read the blog post, but it all seemed about running containers as unprivileged users. Being able to create is what I’m saying is a non-issue, because it only saves a single convience step of not having to upload a container built elsewhere to a cluster. And tools like Singularity have online tools (Sylabs Library, Singularity Hub) which offer remote builders for users that don’t have privileges anywhere (which is hard to find) and don’t want to build containers locally. You can then pull and run the produced container from the Singularity executable without needing root access. An alternative is a cluster could set up a single machine that gives users sudo access only on the Singularity executable and prevent the ability to bind mount, therefore creating a relatively secure build environment on-premise. However, security matters more than conviencence for HPC clusters, so we’re more inclined to tell users “do it yourself” than give them any loophole in the system. Singularity already solves the unprivileged running of containers. Unless binctr is doing something that provides a different kind of unprivileged access? I’m not trying to be thick headed, but I’m just not connecting the dots on what’s so special about this.
    - justinsaccount 8y ago
      > I’m just not connecting the dots on what’s so special about this. It's not. binctr is not special. Singularity is not special either. They are both tools that do similar things.
- agartner 8y ago
  singularity does partially support running without suid: https://github.com/sylabs/singularity/issues/1258 https://github.com/sylabs/singularity/issues/1258 charliecloud is designed specifically not to need suid: https://github.com/hpc/charliecloud https://github.com/hpc/charliecloud docker will hopefully support full unprivileged use soon: https://github.com/moby/moby/pull/38050 https://github.com/moby/moby/pull/38050
  - cyphar 8y ago
    The Docker work is a direct derivative of the rootless containers work I started more than 2 years ago (and others have been working on before and since), which is what this blog post refers to. Singularity didn't exist at the time in any meaningful way, and suid binaries (even a small number) are completely unacceptable for the usecases I had.
  - mroche 8y ago
    Thanks for those, I’ll read through them when I get the chance!
- cyphar 8y ago
  > I’m trying to compare/contrast the goals of binctr to existing solutions that appear to do the same or similar thing(s) already, so I’m open to learning something new today! binctr is a proof of concept that was written almost 3 years ago. These days, rootless containers are being integrated into many tools. Right now, under [1] we have some tooling which allows you to run Docker, cri-o and Kubernetes in rootless mode. Cloud Foundry's default configuration now uses rootless containers. So really, it's not a choice between binctr and something else (binctr was a proof-of-concept) -- in fact please don't run binctr in production. But runc and quite a lot of other tooling has rootless support now. Singularity doesn't do rootless containers, it still requires suid binaries in several cases. But I'm biased, having had some pretty bad arguments with the lead developers in the past (mainly for misrepresenting runc). [1]: https://github.com/rootless-containers https://github.com/rootless-containers
  - mroche 8y ago
    Thanks! This helps me understand a bit better. So away from binctr, which should just be considered a reference, a rootless system isn’t just about being able to run i.e. execute a container, but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. For us, one of Singularity’s primary advantages is the host filesystem integration capabilities. Does a rootless container system have any potential drawbacks when dealing with host filesystem integrations and user permissions? Or should this be considered up to the implementer’s discretion and a separate issue? This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? I often end up getting caught up in tooling and missing the bigger picture/goal, so apologies for the long back and forth :)
    - cyphar 8y ago
      > a rootless system isn’t just about being able to run i.e. execute a container This is part of it, but I want to repeat again that Singularity (and most projects that you've mentioned) only appear to not require privileges -- they use setuid binaries. That isn't the same as not requiring privileges (in my opinion -- because requiring setuid binaries means you need root to install something on the host). Rootless containers can be used without any root access at any point -- even during container runtime installation. > but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. You have "root" inside the container, but this is a user namespace so it's actually an unprivileged user on the host (and you don't need to have privileges to create a user namespace). Package managers get a bit complicated, but we have techniques to get around their lovely issues (or you can use newuidmap/newgidmap that map a range of UIDs/GIDs and then it all works without issue -- these are standard setiud programs you get with distributions and most distributions will give users mappings by default). Docker is a bit complicated (to run as an unprivileged user or inside a container), but with some patches (which are mostly upstream from memory) you can run Docker inside a user namespace with a bit of finagling thanks to the rootless work done by myself and Akihiro. > one of Singularity’s primary advantages is the host filesystem integration capabilities. One of my problems with Singularity is that they keep creating new words to define old concepts -- can you explain what you mean by "host filesystem integration"? Are you talking about bind-mounts? If yes, then yes you can use them. > This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? Yes, this is precisely the long-term usecase we want to have. Right now you can run Kubernetes unprivileged (in a mode we call "usernetes"[1]) and the idea is that you will be able to login as any unprivileged user on a cluster machine and set up Kubernetes (or just run containers directly if you don't like Kubernetes). There are some complications with networking, but we're working on it. [1]: https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes

Is there anyway to do what it does without them? Genuine question, there’s still a lot about Linux system/user space that I’m not up to snuff on yet (part of a team here, and I don’t handle users, mostly tools). I’m trying to compare/contrast the goals of binctr to existing solutions that appear to do the same or similar thing(s) already, so I’m open to learning something new today! The only new thing I’m seeing from this solution is the ability to create containers as an unprivileged user, which for most scenarios I tend to view as a non-issue. Or am I misunderstanding that?

symkat 8y ago
> The only new thing I’m seeing from this solution is the ability to create containers as an unprivileged user, which for most scenarios I tend to view as a non-issue. Or am I misunderstanding that? I believe you're mislabeling it as a non-issue, it seems to be the core problem they're trying to solve: From the blog[0]: > Yes, we all know that containers run unprivileged processes; but creating and running the containers themselves requires root privileges at some point. > What is the awesome sauce we all gain from this? > Well judging by the original GitHub issue about unprivileged runc containers, the largest group of commenters is from the scientific community who are restricted to not run certain programs as root. [0]: https://blog.jessfraz.com/post/getting-towards-real-sandbox-containers/ https://blog.jessfraz.com/post/getting-towards-real-sandbox-...
- mroche 8y ago
  I did read the blog post, but it all seemed about running containers as unprivileged users. Being able to create is what I’m saying is a non-issue, because it only saves a single convience step of not having to upload a container built elsewhere to a cluster. And tools like Singularity have online tools (Sylabs Library, Singularity Hub) which offer remote builders for users that don’t have privileges anywhere (which is hard to find) and don’t want to build containers locally. You can then pull and run the produced container from the Singularity executable without needing root access. An alternative is a cluster could set up a single machine that gives users sudo access only on the Singularity executable and prevent the ability to bind mount, therefore creating a relatively secure build environment on-premise. However, security matters more than conviencence for HPC clusters, so we’re more inclined to tell users “do it yourself” than give them any loophole in the system. Singularity already solves the unprivileged running of containers. Unless binctr is doing something that provides a different kind of unprivileged access? I’m not trying to be thick headed, but I’m just not connecting the dots on what’s so special about this.
  - justinsaccount 8y ago
    > I’m just not connecting the dots on what’s so special about this. It's not. binctr is not special. Singularity is not special either. They are both tools that do similar things.
agartner 8y ago
singularity does partially support running without suid: https://github.com/sylabs/singularity/issues/1258 https://github.com/sylabs/singularity/issues/1258 charliecloud is designed specifically not to need suid: https://github.com/hpc/charliecloud https://github.com/hpc/charliecloud docker will hopefully support full unprivileged use soon: https://github.com/moby/moby/pull/38050 https://github.com/moby/moby/pull/38050
- cyphar 8y ago
  The Docker work is a direct derivative of the rootless containers work I started more than 2 years ago (and others have been working on before and since), which is what this blog post refers to. Singularity didn't exist at the time in any meaningful way, and suid binaries (even a small number) are completely unacceptable for the usecases I had.
- mroche 8y ago
  Thanks for those, I’ll read through them when I get the chance!
cyphar 8y ago
> I’m trying to compare/contrast the goals of binctr to existing solutions that appear to do the same or similar thing(s) already, so I’m open to learning something new today! binctr is a proof of concept that was written almost 3 years ago. These days, rootless containers are being integrated into many tools. Right now, under [1] we have some tooling which allows you to run Docker, cri-o and Kubernetes in rootless mode. Cloud Foundry's default configuration now uses rootless containers. So really, it's not a choice between binctr and something else (binctr was a proof-of-concept) -- in fact please don't run binctr in production. But runc and quite a lot of other tooling has rootless support now. Singularity doesn't do rootless containers, it still requires suid binaries in several cases. But I'm biased, having had some pretty bad arguments with the lead developers in the past (mainly for misrepresenting runc). [1]: https://github.com/rootless-containers https://github.com/rootless-containers
- mroche 8y ago
  Thanks! This helps me understand a bit better. So away from binctr, which should just be considered a reference, a rootless system isn’t just about being able to run i.e. execute a container, but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. For us, one of Singularity’s primary advantages is the host filesystem integration capabilities. Does a rootless container system have any potential drawbacks when dealing with host filesystem integrations and user permissions? Or should this be considered up to the implementer’s discretion and a separate issue? This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? I often end up getting caught up in tooling and missing the bigger picture/goal, so apologies for the long back and forth :)
  - cyphar 8y ago
    > a rootless system isn’t just about being able to run i.e. execute a container This is part of it, but I want to repeat again that Singularity (and most projects that you've mentioned) only appear to not require privileges -- they use setuid binaries. That isn't the same as not requiring privileges (in my opinion -- because requiring setuid binaries means you need root to install something on the host). Rootless containers can be used without any root access at any point -- even during container runtime installation. > but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. You have "root" inside the container, but this is a user namespace so it's actually an unprivileged user on the host (and you don't need to have privileges to create a user namespace). Package managers get a bit complicated, but we have techniques to get around their lovely issues (or you can use newuidmap/newgidmap that map a range of UIDs/GIDs and then it all works without issue -- these are standard setiud programs you get with distributions and most distributions will give users mappings by default). Docker is a bit complicated (to run as an unprivileged user or inside a container), but with some patches (which are mostly upstream from memory) you can run Docker inside a user namespace with a bit of finagling thanks to the rootless work done by myself and Akihiro. > one of Singularity’s primary advantages is the host filesystem integration capabilities. One of my problems with Singularity is that they keep creating new words to define old concepts -- can you explain what you mean by "host filesystem integration"? Are you talking about bind-mounts? If yes, then yes you can use them. > This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? Yes, this is precisely the long-term usecase we want to have. Right now you can run Kubernetes unprivileged (in a mode we call "usernetes"[1]) and the idea is that you will be able to login as any unprivileged user on a cluster machine and set up Kubernetes (or just run containers directly if you don't like Kubernetes). There are some complications with networking, but we're working on it. [1]: https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes