4 ms·
You stopped typing; what does Kubernetes do a bad job at with relation to scheduling workloads that declare they need at least 1 GPU resource but should be limi
by mdaniel 1y ago
You stopped typing; what does Kubernetes do a bad job at with relation to scheduling workloads that declare they need at least 1 GPU resource but should be limited to no more than 4 GPU resources on a given Node? https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus/#:~:text=you%20can%20consume%20these%20gpus%20from%20your%20containers%20by%20requesting%20the%20custom%20gpu%20resource https://kubernetes.io/docs/tasks/manage-gpus/scheduling-gpus...
- mountainriver 1y agoBecause they aren't consistently available in one region, and Kubernetes can't scale globally. If you don't believe me, go try to train LLMs on Kubernetes with on demand resources, good luck!
- mdaniel 1y agoOh, ok, sorry, I didn't realize that we were conflating Kubernetes, a container orchestration control plane, with your favorite cloud provider not putting in enough GPU hardware orders per region. My fault
- mountainriver 1y agoYou should try to actually build an ML product so you see what the issue is instead of a being a smug poodle.