4 ms·

Great project! I have a question - how can you afford to run this? I looked at the server costs on google compute and runpod.io, and the ones with GPUs powerfu

Great project! I have a question - how can you afford to run this? I looked at the server costs on google compute and runpod.io, and the ones with GPUs powerful enough to run SD are pretty expensive. I'm guessing you don't have a lot of paying users at the beginning, so I don't think you have enough revenue to cover the costs (at least when you're just getting started). Do you just spend a lot of money to fund this, and hope that once you have enough users it will pay for itself? Or did you find some affordable way to run a server? If yes, and if it's not a secret - where are you renting the server, what are its specs, and how much does it cost?

permanent 4y ago
I'd love to know this as well.
nacs 4y ago
Stable diffusion only needs 4GB of VRAM to run on the low end so you can rent low-end consumer GPUs (nvidia RTX for example) for around $0.10 an hour to do the renders.
- UncleEntity 4y ago
  I was running it for free on google colab here and there with no problems. Haven’t tried the 2.0 out yet so not sure how that’s going.
- jkbl 4y ago
  Yeah, but that's only true when you use one model for yourself. More VRAM is needed for running such a service. It currently loads 6 models per single GPU. And I think I have some VRAM left to add even more.
jkbl 4y ago
At the beginning it was just burning money, but it stars to turn into profit. The only secret to making it affordable is to autoscale based on a current image generation traffic. It runs on a mix of Tesla V100 and RTX 3090 from runpod.io vast.ai and Lambda Labs. I actually went through a couple of iterations with it, starting as completely free service, then offering one time payments for image credits and now settled on subscriptions as I see there is a demand for the product. Especially AI Editor[1] which I think offers a unique value to the users. [1] https://getimg.ai/editor https://getimg.ai/editor
- lumenwrites 4y ago
  That's amazing, thank you for sharing! If not a secret, how are you marketing it?
  - jkbl 4y ago
    I try to be everywhere, when I have free time from adding new features ;)
synapticpaint 4y ago
On AWS, a g5 instance costs $1/hr. I can generate roughly 10 images per minute (should be able to get this down with some optimization), so 600 images per hour, so the cost per image is 1/6 of a cent, before adding overhead (idle time, start up/shut down). Source: I run a similar project: https://synapticpaint.com/ https://synapticpaint.com/ I also offer dreambooth model training for around $2-$4 / model as well as inference on custom dreambooth models. Inference on custom models is where things get a little tricky because if users are using different models and you're loading up new models all the time just to generate 6 images, then that quickly becomes the majority of the work load, drastically pushing up the inference cost. I haven't solved this problem yet. If you have any great ideas, feel free to email me (email in profile)!
raymond_goo 4y ago
https://vast.ai/ https://vast.ai/ (not affiliated)

permanent 4y ago

I'd love to know this as well.

nacs 4y ago

Stable diffusion only needs 4GB of VRAM to run on the low end so you can rent low-end consumer GPUs (nvidia RTX for example) for around $0.10 an hour to do the renders.

UncleEntity 4y ago
I was running it for free on google colab here and there with no problems. Haven’t tried the 2.0 out yet so not sure how that’s going.
jkbl 4y ago
Yeah, but that's only true when you use one model for yourself. More VRAM is needed for running such a service. It currently loads 6 models per single GPU. And I think I have some VRAM left to add even more.

jkbl 4y ago

At the beginning it was just burning money, but it stars to turn into profit. The only secret to making it affordable is to autoscale based on a current image generation traffic. It runs on a mix of Tesla V100 and RTX 3090 from runpod.io vast.ai and Lambda Labs. I actually went through a couple of iterations with it, starting as completely free service, then offering one time payments for image credits and now settled on subscriptions as I see there is a demand for the product. Especially AI Editor[1] which I think offers a unique value to the users. [1] https://getimg.ai/editor https://getimg.ai/editor

lumenwrites 4y ago
That's amazing, thank you for sharing! If not a secret, how are you marketing it?
- jkbl 4y ago
  I try to be everywhere, when I have free time from adding new features ;)

synapticpaint 4y ago

On AWS, a g5 instance costs $1/hr. I can generate roughly 10 images per minute (should be able to get this down with some optimization), so 600 images per hour, so the cost per image is 1/6 of a cent, before adding overhead (idle time, start up/shut down). Source: I run a similar project: https://synapticpaint.com/ https://synapticpaint.com/ I also offer dreambooth model training for around $2-$4 / model as well as inference on custom dreambooth models. Inference on custom models is where things get a little tricky because if users are using different models and you're loading up new models all the time just to generate 6 images, then that quickly becomes the majority of the work load, drastically pushing up the inference cost. I haven't solved this problem yet. If you have any great ideas, feel free to email me (email in profile)!

raymond_goo 4y ago

https://vast.ai/ https://vast.ai/ (not affiliated)