3 ms·
I read the article: "If you have a GPU powerful enough to generate stable diffusion results in under five seconds, you can run the experience locally using our
by TuringNYC 4y ago
I read the article:
"If you have a GPU powerful enough to generate stable diffusion results in under five seconds, you can run the experience locally using our test flask server."
Curious what sort of GPU the author was using or what some of the min requirements might be?
- genewitch 4y agoRTX 3070 can generate SD results in under 5 seconds, depending. Euler A 20 samples, 512x512. it can almost do 4 images in 5 seconds with those settings. It's possible a 3060 might work, depending. in my experience the 3060 is about 50% slower than the 3070, but that may be a bad 3060 in our test rig. but a 3060 gets pretty close to 5 seconds for an image, so try it, if you have one. just tested prompt "a test pattern for television" on both cards and 3070 took 1.87s and the 3060 took 2.93s. Similar results for the prompt "an intricate cityscape, like new york" edit: i should note we're using SD 1.4, not 1.5, although i think that just has to do with the checkpoint of the model, not the algorithm, but i could be wrong. Also the model is over 14GB, so perhaps the 3070 can't do it after all. i'll test it later as soon as the local admin wakes up and downloads it onto our machine.
- seth_ 4y agoAuthor here: fwiw we are running the app on a10g GPUs, which generally can turn around a 512x512 in 3.5s with 50 inference steps. This time includes converting the image into audio which should be done on the GPU as well for real-time purposes. We did some optimization such as a traced unet, fp16 and removing autocast. There are lots of ways it could be sped up further I'm sure!