11 ms·
Efficient GPU Usage in Cloud Services: Is Pay-per-Use Feasible?
Do others feel that a significant amount of time on a rented GPU-based instance is spent not utilizing the GPU? Often, the time is consumed debugging code, which mainly involves the CPU rather than the GPU. This leads to paying for GPU time that isn’t actually being used. Are there any cloud providers that offer a pay-per-use model specifically for GPU usage, rather than paying for the entire instance? Is this an issue for anyone else?
- billconan 2y agoI think https://docs.ray.io/en/latest/index.html https://docs.ray.io/en/latest/index.html is a good solution? basically debug locally, execute remotely?
- graphllama 2y agoInteresting! However this comes with the overhead of having to modify your code to adhere to the framework right? I was looking for a solution more along the lines of paying for the GPU when a task is actually scheduled or running on it rather than paying all the time when the VM is turned on. Basically a serverless GPU sort of a solution?