4 ms·

The big annoying one in HPC is NFS/GPFS, but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the cont

by prpl 6y ago

The big annoying one in HPC is NFS/GPFS, but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container.

mroche 6y ago
I'm a little confused by this statement, since at my current employer and previous employer we used NFS mounted home dirs and I would and currently do need to explicitly change my ~/.config/containers/storage.conf to point to a local mount point, such as /var/tmp/containers/$USER in order to perform anything rootless container related. In my case that's with Podman on RHEL 7 and RHEL 8.
- maztaim 6y ago
  That's an acceptable workaround. Just remember the containers aren't available on NFS and thus other systems.
  - mroche 6y ago
    I’m aware, I’m trying to determine what prpl meant by > but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container
    - maztaim 6y ago
      I'll let Dan Walsh explain it https://www.redhat.com/sysadmin/rootless-podman-nfs https://www.redhat.com/sysadmin/rootless-podman-nfs In short, you don't remap when running as root in the container, thus no uid/gid remapping.
      - mroche 6y ago
        I had a derp-out moment yesterday, I totally misinterpreted “as root in the container”. The topic was about rootless containers, so I was completely mentally blacklisting the idea of starting a container from the root user account.
- prpl 6y ago
  That’s fine until you need to actually write to NFS/GPFS.
  - mroche 6y ago
    Words might not be clicking with me today, I’m trying to specifically decipher this part: > but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container The context of the statement isn’t clicking with me. You start with “it’s a problem with these filesystems” which I’m on board with but then follow with “but it’s fine if you do this” and I’m not seeing the correct correlation.
    - prpl 6y ago
      With user namespaces you can get different uids inside the container. They are usually mapped to something above 65535, in blocks of 65535, according to /etc/subuid. When you run as rootless on podman with uid=1000, you end up at 66535, or 1000 above your start range in /etc/subuid, for example. This is what NFS/GPFS doesn’t like. Locally, in other file systems on the node, it’s fine. When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine. The issue with this is that lots of available containers have moved away from using things as root. For example, you couldn’t start the vanilla redis container with a data directory on NFS for snapshots because redis runs as user 999.
      - mroche 6y ago
        > When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine. Oh oh oh, you were just saying run the container as root, like literally. That’s what threw me off, because I was in the mindset of rootless. Wasn’t allowing myself to think outside of that environment. Sorry for the confusion on that. Long day yesterday.
gnufx 6y ago
Changing uids is something you don't want for HPC use anyhow. Charliecloud is a simple alternative approach specifically for HPC. Actually, a lot of the time you can get away without namespacing anyhow, just by settings paths to point into an unpacked image. That does depend on what you want to run being sufficiently configurable, though.

by prpl 6y ago

The big annoying one in HPC is NFS/GPFS, but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container.

mroche 6y ago

I'm a little confused by this statement, since at my current employer and previous employer we used NFS mounted home dirs and I would and currently do need to explicitly change my ~/.config/containers/storage.conf to point to a local mount point, such as /var/tmp/containers/$USER in order to perform anything rootless container related. In my case that's with Podman on RHEL 7 and RHEL 8.

maztaim 6y ago
That's an acceptable workaround. Just remember the containers aren't available on NFS and thus other systems.
- mroche 6y ago
  I’m aware, I’m trying to determine what prpl meant by > but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container
  - maztaim 6y ago
    I'll let Dan Walsh explain it https://www.redhat.com/sysadmin/rootless-podman-nfs https://www.redhat.com/sysadmin/rootless-podman-nfs In short, you don't remap when running as root in the container, thus no uid/gid remapping.
    - mroche 6y ago
      I had a derp-out moment yesterday, I totally misinterpreted “as root in the container”. The topic was about rootless containers, so I was completely mentally blacklisting the idea of starting a container from the root user account.
prpl 6y ago
That’s fine until you need to actually write to NFS/GPFS.
- mroche 6y ago
  Words might not be clicking with me today, I’m trying to specifically decipher this part: > but that will actually work just fine if you stay as root inside the container, which maps to your uid outside the container The context of the statement isn’t clicking with me. You start with “it’s a problem with these filesystems” which I’m on board with but then follow with “but it’s fine if you do this” and I’m not seeing the correct correlation.
  - prpl 6y ago
    With user namespaces you can get different uids inside the container. They are usually mapped to something above 65535, in blocks of 65535, according to /etc/subuid. When you run as rootless on podman with uid=1000, you end up at 66535, or 1000 above your start range in /etc/subuid, for example. This is what NFS/GPFS doesn’t like. Locally, in other file systems on the node, it’s fine. When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine. The issue with this is that lots of available containers have moved away from using things as root. For example, you couldn’t start the vanilla redis container with a data directory on NFS for snapshots because redis runs as user 999.
    - mroche 6y ago
      > When you run as uid=0 in same the container, you are actually mapped to your uid, since the range starts to be applicable for uid=1. If you can stay as root, you can write to NFS/GPFS just fine. Oh oh oh, you were just saying run the container as root, like literally. That’s what threw me off, because I was in the mindset of rootless. Wasn’t allowing myself to think outside of that environment. Sorry for the confusion on that. Long day yesterday.

gnufx 6y ago

Changing uids is something you don't want for HPC use anyhow. Charliecloud is a simple alternative approach specifically for HPC. Actually, a lot of the time you can get away without namespacing anyhow, just by settings paths to point into an unpacked image. That does depend on what you want to run being sufficiently configurable, though.