4 ms·
Did you set up your own infrastructure? I’ve been looking for a Heroku-like service to deploy some of these GPU intensive models
by choxi 4y ago
Did you set up your own infrastructure? I’ve been looking for a Heroku-like service to deploy some of these GPU intensive models
- searchableguy 4y agoYou can use https://replicate.ai https://replicate.ai for heroku like experience. It's very expensive though but they charge compute per second based on requests. Otherwise, I recommend using runpod, lambda, etc for an order of magnitude of savings. Crowd source services like vast.ai aren't reliable and barely less expensive than dedicated low cost providers.
- arthurcolle 4y agoI could use an assist/pointers on running this better. Would love to chat since you seem to have some knowledge here. If you're game, feel free to contact me at my HN username at Google's email service. Cheers!
- yolo4553 4y agoAs mentioned in a sibling comment those services are relatively expensive. If you don't mind ~30 minute one time setup you can get stable diffusion up and running incl. a nice webinterface on basically any cloud provider offering GPUs. I used https://rentry.org/GUItard https://rentry.org/GUItard as a guide and adopted it slightly for my needs. As my desktop only has a RTX3060 I am renting a GPU instance on Genesis Cloud (billed by the minute and no cost while instance is stopped) using a RTX3090 with 24 GB of vram.
- arthurcolle 4y agoHow long does 50 step inference take with the rtx3090?
- yolo4553 4y ago512x512 with k_lms is around 5s
- arthurcolle 4y agoNice, that's pretty fast