3 ms·
That sounds kinda weird to me. I use fly.io exactly because the pricing works out for hobby use. I can enable my machine for a few hours, run a bunch of inferen
by Aeolun 2y ago
That sounds kinda weird to me. I use fly.io exactly because the pricing works out for hobby use. I can enable my machine for a few hours, run a bunch of inference, and turn it off again. The whole auto start/stop thing makes it seamless too.
- BoorishBears 2y agoVast has a datacenter H200 for less than what their A100 goes for.
- Aeolun 2y agoSure, but vast feels like renting a GPU from a rando.
- BoorishBears 2y agoEither you're familiar with GPU pricing and being willfully ignorant, or you're not familiar with the pricing in which case let someone who is point out: - "Datacenter" means it's comparable to Runpod's secure cloud pricing. - A spot instance of an H200 under someone's living room media console wouldn't go for A100 rates. $3.50 will also get you an H100 at a laundry list of providers people build real businesses on. Certainly all better track records than fly.io, especially on a post where they explain it's not working out for them as an offering and then promise they'll keep it shambling along.
- johntash 2y agoYou seem like you're familiar with vast. Have you used their autoscaler/serverless offering before? I haven't tried it yet, but it wasn't immediately obvious if I could have something like ollama running and scaled to zero instances when not in use.
- johntash 2y agoIf you don't mind, what size models are you running and do you know around what you were paying? fly.io was the first provider I tried any gpu offerings at, I probably should give it another shot now that I've used a few others.