4 ms·
> There are no great FaaS options for running GPU workloads, which is why we could never go fully FaaS. I keep wondering when this is going to show up. We have
by breckenedge 3y ago
> There are no great FaaS options for running GPU workloads, which is why we could never go fully FaaS.
I keep wondering when this is going to show up. We have a lot of service providers, but even more frameworks, and every vendor seems to have their own bespoke API.
- z3ugma 3y agoCheck out beam.cloud. They’re impressing me with calling GPU runtimes as a FaaS
- gfodor 3y agoI just started playing with modal.com and so far it seems good. I haven't run anything in production yet, so YMMV.
- gen220 3y agoI don’t think anybody should go “fully FaaS”, it’s like saying screwdrivers are useless, all you need is a hammer. That being said, Cloudflare is on the path to offering a great GPU FaaS system for inference. I believe it’s still in beta, but it’s the most promising option at the moment.
- breckenedge 3y agoRight, I still find it faster to manually provision a specific instance type, install PyTorch on it, and deploy a little flask app for an inference server.