2 ms·
When we ran HPC CFD models at university we would just use a head node and SLURM to schedule work on machines. It worked really well. I'm not really sure what b
by matdehaast 6y ago
When we ran HPC CFD models at university we would just use a head node and SLURM to schedule work on machines. It worked really well. I'm not really sure what benefit Kubernetes adds in this case? If anything I feel like it adds more complexity.
Would love to hear from the OpenAI folks as to why they chose to use Kubernetes for this problem.
- volta87 6y agoFrom the POV of an HPC cluster user, when using SLURM's `srun` or similar to schedule a job, this now allows you to use `srun --container=<your container>`, and it will start each node where you app run using the container, and make sure MPI, GPUs, etc, all work. If you don't know anything about containers, it probably will be a bit hard to imagine what this buys you, but don't worry, as more clusters start moving towards this model, you'll have to learn about containers at some point. From the POV of the HPC cluster, it means that the `module` system can be replaced with containers, and that can significantly lower the maintenance overhead of the cluster. In a sense, it turns HPC cluster users into HPC cluster maintainers (that have to build their own images, preparing their own environment, etc).
- bend4r 6y agoModern HPC schedulers like Slurm are able to run containerized workloads making the problem of portability less painful. For batch workload K8s is not sufficient. You need a workflow engine on top like Kubeflow or Airflow, so run you jobs. Thus. consider K8s more as a cluster operating software, but not as a fully-enabled workload management like slurm. Furthermore, K8s has so many features you do not need in HPC environments, where high throughput and load occupation is paramount.