3 ms·
Yes, I have wanted something like this for a while. I try to avoid using gpus where possible because of the expense, and the ephemeral nature of my use.
by lbhdc 2y ago
Yes, I have wanted something like this for a while. I try to avoid using gpus where possible because of the expense, and the ephemeral nature of my use.
- cpeterson42 2y agoInteresting to hear. What kind of workloads are you running?
- lbhdc 2y agoVarious forms of content analysis. It is mostly traditional heuristics with ml models sprinkled in. We don't run inference on every request, but it would be great to access a dynamicly provisioned gpu for a single request (kind of like its another serverless container in the system).
- cpeterson42 2y agoThat's fascinating to hear and I think it would work really well with what we do. What I am picturing is that you could run the whole workflow including traditional heuristics in a CPU instance, which would connect to a GPU on-demand. If you are interested would love for you to try this. We're running a (very unprofitable) beta with a T4 instance + a CPU-only instance for $10/month for those who are willing to help us test this with production workloads. If you'd be interested would love to chat at carl (at) thundercompute (dot) com.