4 ms·
I never had one fail, but I had to go through some big hurdles to use them at work (University research lab) properly. Namely, there are 2 big issues: - Virtua
by wtcactus 4y ago
I never had one fail, but I had to go through some big hurdles to use them at work (University research lab) properly.
Namely, there are 2 big issues:
- Virtualization: Getting vGPU functionality out of them (most of the time, there's no point in having a full 1080 Ti for only 1 job, we want to have it split on more than 1 VM).
- Kubernetes workloads: I'm running a simple workload that uses the GPU (for testing for instance) and k8s wants it all for only one pod.
- ftufek 4y agoJust set the required/requested gpu number to 0 and k8s will let you use same gpu with multiple pods. You'll need to make sure to not overload gpu mem, otherwise you'll get oom errors. You can use k8s patch functionality to add custom resources, essentially use that to represent gpu mem and include the amount of gpu the pod uses in its definition, that way you don't go over the limit.