4 ms·

is it? most HPC (if GPU clusters count) are probably in industry and managed by containers

birdiesanders 3y ago
Containers rely on many privilege separation systems to do what they do, they are in fact a rather extreme case of multi-user systems, but they tend to present as “single” user environs to the container’s processes.
- airocker 3y ago
  Good software hides complexity. User does not have to understand user group permissions suid etc etc
- whimsicalism 3y ago
  > they are in fact a rather extreme case of multi-user systems Are they? My understanding was that by default, the `dockerd` (or whatever) is root and then all containers map to the same non-privileged user.
bayindirh 3y ago
HPC admin here. Yes. First, we use user level container systems like apptainer/singularity, and these containers run under the user itself. This is also same for non academic HPC systems. From schedulers to accounting, everything is done at user level, and we have many, many users. It won’t change anytime soon.
- pankajkumar229 3y ago
  there is no reason for users to be maintained in the kernel.
  - bayindirh 3y ago
    Can you elaborate on that?
- whimsicalism 3y ago
  I thought most containers shared the same user, ie. `dockremap` in the case of docker. I understand academia has lots of different accounts.
  - bayindirh 3y ago
    Nope, full usermode containers (e.g.: apptainer) run under the user's own context, and furthermore under a cgroup (if we're talking HPC/SLURM at least) which restricts the user's resources to what they requested in their job file. Hence all containers are isolated from each other, not only at process level, but at user + cgroup level too. Apptainer: https://apptainer.org https://apptainer.org
    - airocker 3y ago
      I think a admin would better understand the system if there was only one subsystem doing a particular type of security and not two. Two subsystems doing security would lead to more problems down the road.
      - mbreese 3y ago
        For HPC, there are two different contexts where users need to be considered - interactive use and batch job processing. Users login to a cluster, write their scripts, work with files, etc. This is your typical user account stuff. But they also submit jobs here. Second, there are the jobs users submit. These are often executed on separate nodes and the usage is managed. Here you have both user and cgroup limits in place. The cgroups make sure that the jobs on have the required resources. The user authentication makes sure that the job can read/write data as the user. This was the user can work with their data on the interactive nodes. So the two different systems have different rationales, and both are needed. It all depends on the context.
        airocker 3y ago
        If we forget how the current system is architected, we are looking at two problems: First problem is that Linux capabilities are also dealing with isolating processes so they have limited capabilities because the user based isolation is not enough. Second problem is that local identity has no relation to the cloud identity which is undesirable. If we remove user based authentication and rely on capabilities only with identity served by cloud or kubernetes, it could be a simpler way to do authenticating and authorization

Containers rely on many privilege separation systems to do what they do, they are in fact a rather extreme case of multi-user systems, but they tend to present as “single” user environs to the container’s processes.

airocker 3y ago
Good software hides complexity. User does not have to understand user group permissions suid etc etc
whimsicalism 3y ago
> they are in fact a rather extreme case of multi-user systems Are they? My understanding was that by default, the `dockerd` (or whatever) is root and then all containers map to the same non-privileged user.

bayindirh 3y ago

HPC admin here. Yes. First, we use user level container systems like apptainer/singularity, and these containers run under the user itself. This is also same for non academic HPC systems. From schedulers to accounting, everything is done at user level, and we have many, many users. It won’t change anytime soon.

pankajkumar229 3y ago
there is no reason for users to be maintained in the kernel.
- bayindirh 3y ago
  Can you elaborate on that?
whimsicalism 3y ago
I thought most containers shared the same user, ie. `dockremap` in the case of docker. I understand academia has lots of different accounts.
- bayindirh 3y ago
  Nope, full usermode containers (e.g.: apptainer) run under the user's own context, and furthermore under a cgroup (if we're talking HPC/SLURM at least) which restricts the user's resources to what they requested in their job file. Hence all containers are isolated from each other, not only at process level, but at user + cgroup level too. Apptainer: https://apptainer.org https://apptainer.org
  - airocker 3y ago
    I think a admin would better understand the system if there was only one subsystem doing a particular type of security and not two. Two subsystems doing security would lead to more problems down the road.
    - mbreese 3y ago
      For HPC, there are two different contexts where users need to be considered - interactive use and batch job processing. Users login to a cluster, write their scripts, work with files, etc. This is your typical user account stuff. But they also submit jobs here. Second, there are the jobs users submit. These are often executed on separate nodes and the usage is managed. Here you have both user and cgroup limits in place. The cgroups make sure that the jobs on have the required resources. The user authentication makes sure that the job can read/write data as the user. This was the user can work with their data on the interactive nodes. So the two different systems have different rationales, and both are needed. It all depends on the context.
      - airocker 3y ago
        If we forget how the current system is architected, we are looking at two problems: First problem is that Linux capabilities are also dealing with isolating processes so they have limited capabilities because the user based isolation is not enough. Second problem is that local identity has no relation to the cloud identity which is undesirable. If we remove user based authentication and rely on capabilities only with identity served by cloud or kubernetes, it could be a simpler way to do authenticating and authorization