3 ms·

Thanks! This helps me understand a bit better. So away from binctr, which should just be considered a reference, a rootless system isn’t just about being able t

by mroche 8y ago

Thanks! This helps me understand a bit better. So away from binctr, which should just be considered a reference, a rootless system isn’t just about being able to run i.e. execute a container, but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. For us, one of Singularity’s primary advantages is the host filesystem integration capabilities. Does a rootless container system have any potential drawbacks when dealing with host filesystem integrations and user permissions? Or should this be considered up to the implementer’s discretion and a separate issue? This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? I often end up getting caught up in tooling and missing the bigger picture/goal, so apologies for the long back and forth :)

cyphar 8y ago
> a rootless system isn’t just about being able to run i.e. execute a container This is part of it, but I want to repeat again that Singularity (and most projects that you've mentioned) only appear to not require privileges -- they use setuid binaries. That isn't the same as not requiring privileges (in my opinion -- because requiring setuid binaries means you need root to install something on the host). Rootless containers can be used without any root access at any point -- even during container runtime installation. > but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. You have "root" inside the container, but this is a user namespace so it's actually an unprivileged user on the host (and you don't need to have privileges to create a user namespace). Package managers get a bit complicated, but we have techniques to get around their lovely issues (or you can use newuidmap/newgidmap that map a range of UIDs/GIDs and then it all works without issue -- these are standard setiud programs you get with distributions and most distributions will give users mappings by default). Docker is a bit complicated (to run as an unprivileged user or inside a container), but with some patches (which are mostly upstream from memory) you can run Docker inside a user namespace with a bit of finagling thanks to the rootless work done by myself and Akihiro. > one of Singularity’s primary advantages is the host filesystem integration capabilities. One of my problems with Singularity is that they keep creating new words to define old concepts -- can you explain what you mean by "host filesystem integration"? Are you talking about bind-mounts? If yes, then yes you can use them. > This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? Yes, this is precisely the long-term usecase we want to have. Right now you can run Kubernetes unprivileged (in a mode we call "usernetes"[1]) and the idea is that you will be able to login as any unprivileged user on a cluster machine and set up Kubernetes (or just run containers directly if you don't like Kubernetes). There are some complications with networking, but we're working on it. [1]: https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes
- mroche 8y ago
  > Are you talking about bind-mounts? If yes, then yes you can use them. Yes, I’m referring to the bind mount capabilities. By default they are enabled, but Singularity’s config file allow us to disable user defined mounts and set global mounts that will always be used. > Yes, this is precisely the long-term usecase we want to have. I’m confused. The situation I described was a negative where a user run container bind mounts directories they shouldn’t have access to and modifying it. Like mount host /etc to container /mnt then start editing the host systems configuration files. I may be misinterpreting your statement but I don’t see how that event should be a long term goal...
  - cyphar 8y ago
    > Yes, I’m referring to the bind mount capabilities. Okay. Every existing container runtime supports bind-mounts. Having a global configuration for them is mostly a UX thing (you could make a wrapper in half an hour that does that). > The situation I described was a negative where a user run container bind mounts directories they shouldn’t have access to and modifying it. Oh I misunderstood your question. No, that's not possible. "Root" inside a user namespace cannot access files that the underlying user would not be able to access -- this is guaranteed by the kernel (which is why I misunderstood your question -- it's a kernel security policy which affects every process). I thought you were asking whether it's possible to run a cluster as an unprivileged user. An unprivileged user can bind-mount whatever they want, but actual file access is blocked when you would expect.
    - mroche 8y ago
      As someone who doesn't know a lot of the nitty details of the kernel or containerizing, I appreciate you for taking the time explaining some of this. So just to make it completely clear in my head (sorry!), when utilizing a user namespace rootless (non-setuid) container system: * The user has complete read/write/execute to everything in the 'base' container. Anything bind-mounted in assumes the given privileges from the host system.[0] The user's context is essentially mapped to root inside the container, but not a 'true' root in the context of the host system. * The user can create/modify/run a container without needing root privileges to communicate with a daemon or any other tooling. * A user can install a rootless system to their owned directories on a host system and operate its services/tools without needing elevated privilege access. [0] In the situation where a user maps a root owned directory to its corresponding directory in the container (/usr:/usr) the container root privilege access is no longer applicable. Other permissions (rwxrwxrwx user group) also apply inside the container from bind-mounts. If the above is all true, then I think I've got it. All in all, I do think the idea of a rootless system is pretty cool. Obviously you need to be running a 3.8 or later kernel (unless backported for namespaces) and enabled in the kernel, but its cool tech. Thank you for taking the time to help me understand it better. This probably would have gone a lot faster if done in verbal conversation, so I appreciate you sticking it through :) If you don't mind me asking, what came up in your conversations with the Sylabs team? Was it a case of "we'd like to wait and see as this is new[1], not thoroughly vetted tech", not developed in-house, or they just didn't want the tool to work that way? Those are just speculations I can come up with off the top of my head. If you don't want to talk about it, that's fine, too. [1] Relatively speaking at the time. It's clearly not brand-spanking new.

by mroche 8y ago

cyphar 8y ago

> a rootless system isn’t just about being able to run i.e. execute a container This is part of it, but I want to repeat again that Singularity (and most projects that you've mentioned) only appear to not require privileges -- they use setuid binaries. That isn't the same as not requiring privileges (in my opinion -- because requiring setuid binaries means you need root to install something on the host). Rootless containers can be used without any root access at any point -- even during container runtime installation. > but also go into and modify, like using the containers package manager and fiddling about. Without needing root access to do so (due to the way these tools like yum are built to work). But also be able to use systems like Docker that typically require sudo/root for nearly everything to not need it. You have "root" inside the container, but this is a user namespace so it's actually an unprivileged user on the host (and you don't need to have privileges to create a user namespace). Package managers get a bit complicated, but we have techniques to get around their lovely issues (or you can use newuidmap/newgidmap that map a range of UIDs/GIDs and then it all works without issue -- these are standard setiud programs you get with distributions and most distributions will give users mappings by default). Docker is a bit complicated (to run as an unprivileged user or inside a container), but with some patches (which are mostly upstream from memory) you can run Docker inside a user namespace with a bit of finagling thanks to the rootless work done by myself and Akihiro. > one of Singularity’s primary advantages is the host filesystem integration capabilities. One of my problems with Singularity is that they keep creating new words to define old concepts -- can you explain what you mean by "host filesystem integration"? Are you talking about bind-mounts? If yes, then yes you can use them. > This may be thinking too much (but probably due to lack of deeper understanding) but an event I see playing out is a user on the cluster installing a rootless container system, building their own containers, runtime mounting host directories they shouldn't be that typically require root access and then modifying them. Is this a possible situation? Yes, this is precisely the long-term usecase we want to have. Right now you can run Kubernetes unprivileged (in a mode we call "usernetes"[1]) and the idea is that you will be able to login as any unprivileged user on a cluster machine and set up Kubernetes (or just run containers directly if you don't like Kubernetes). There are some complications with networking, but we're working on it. [1]: https://github.com/rootless-containers/usernetes https://github.com/rootless-containers/usernetes

mroche 8y ago
> Are you talking about bind-mounts? If yes, then yes you can use them. Yes, I’m referring to the bind mount capabilities. By default they are enabled, but Singularity’s config file allow us to disable user defined mounts and set global mounts that will always be used. > Yes, this is precisely the long-term usecase we want to have. I’m confused. The situation I described was a negative where a user run container bind mounts directories they shouldn’t have access to and modifying it. Like mount host /etc to container /mnt then start editing the host systems configuration files. I may be misinterpreting your statement but I don’t see how that event should be a long term goal...
- cyphar 8y ago
  > Yes, I’m referring to the bind mount capabilities. Okay. Every existing container runtime supports bind-mounts. Having a global configuration for them is mostly a UX thing (you could make a wrapper in half an hour that does that). > The situation I described was a negative where a user run container bind mounts directories they shouldn’t have access to and modifying it. Oh I misunderstood your question. No, that's not possible. "Root" inside a user namespace cannot access files that the underlying user would not be able to access -- this is guaranteed by the kernel (which is why I misunderstood your question -- it's a kernel security policy which affects every process). I thought you were asking whether it's possible to run a cluster as an unprivileged user. An unprivileged user can bind-mount whatever they want, but actual file access is blocked when you would expect.
  - mroche 8y ago
    As someone who doesn't know a lot of the nitty details of the kernel or containerizing, I appreciate you for taking the time explaining some of this. So just to make it completely clear in my head (sorry!), when utilizing a user namespace rootless (non-setuid) container system: * The user has complete read/write/execute to everything in the 'base' container. Anything bind-mounted in assumes the given privileges from the host system.[0] The user's context is essentially mapped to root inside the container, but not a 'true' root in the context of the host system. * The user can create/modify/run a container without needing root privileges to communicate with a daemon or any other tooling. * A user can install a rootless system to their owned directories on a host system and operate its services/tools without needing elevated privilege access. [0] In the situation where a user maps a root owned directory to its corresponding directory in the container (/usr:/usr) the container root privilege access is no longer applicable. Other permissions (rwxrwxrwx user group) also apply inside the container from bind-mounts. If the above is all true, then I think I've got it. All in all, I do think the idea of a rootless system is pretty cool. Obviously you need to be running a 3.8 or later kernel (unless backported for namespaces) and enabled in the kernel, but its cool tech. Thank you for taking the time to help me understand it better. This probably would have gone a lot faster if done in verbal conversation, so I appreciate you sticking it through :) If you don't mind me asking, what came up in your conversations with the Sylabs team? Was it a case of "we'd like to wait and see as this is new[1], not thoroughly vetted tech", not developed in-house, or they just didn't want the tool to work that way? Those are just speculations I can come up with off the top of my head. If you don't want to talk about it, that's fine, too. [1] Relatively speaking at the time. It's clearly not brand-spanking new.