3 ms·
The author doesn't seem to now even the basics of SLURM? "gang scheduling" according to the official docs: https://slurm.schedmd.com/gang_scheduling.html https
by GuestFAUniverse 8mo ago
The author doesn't seem to now even the basics of SLURM?
"gang scheduling" according to the official docs:
https://slurm.schedmd.com/gang_scheduling.html https://slurm.schedmd.com/gang_scheduling.html
-- maybe I've read the docs wrong the last decade using SLURM.
- covi 8mo agoThe post says Slurm supports gang scheduling, k8s doesn't (out of the box).
- alex000kim 8mo agoAuthor here. I've seen the docs you linked to: Slurm uses "gang scheduling" to mean something specific (timesliced oversubscription where jobs alternate on shared resources). I'm using the term in its broader CS sense: all-or-nothing co-scheduling of related processes across multiple processors [1]. This is the definition used across the K8s ecosystem e.g. Volcano [2], Kueue [3], and its Coscheduling plugin all define gang scheduling as "all or nothing" allocation. I still stand by the origianl claim: Slurm allocates multi-node jobs atomically, while vanilla K8s doesn't. its default scheduler places pods as resources become available, leading to partial allocations and deadlocks for distributed training. It's just a terminology clash. Thanks for the comment anyway. [1] https://en.wikipedia.org/wiki/Gang_scheduling https://en.wikipedia.org/wiki/Gang_scheduling [2] https://volcano.sh/en/docs/plugins/ https://volcano.sh/en/docs/plugins/ [3] https://www.coreweave.com/blog/kueue-a-kubernetes-native-system-for-ai-training-workloads https://www.coreweave.com/blog/kueue-a-kubernetes-native-sys...
- GuestFAUniverse 8mo agoThanks for the clarification!