5 ms·
> one minute and 50 seconds to generate a 512 x 512-pixel image with the default setting of 50 steps 2s/iter? That is... Not great. It is slower than CPU diff
by brucethemoose2 3y ago
> one minute and 50 seconds to generate a 512 x 512-pixel image with the default setting of 50 steps
2s/iter? That is... Not great.
It is slower than CPU diffusion: https://github.com/rupeshs/fastsdcpu https://github.com/rupeshs/fastsdcpu
Stable Diffusion in particular doesn't need much VRAM anyway. I get that many people are stuck on lower end computers, but ~4GB is not a totally unreasonable requirement.
- samsartor 3y agoLooks more like 4 s/iter which is actually quite good for CPU inference in Automatic111. It is an apples-to-oranges comparison to the LCM models used by fastsdcpu (since those can get away with fewer sampling steps) or to the quantized models favored by apple (since they use totally different underlying compute capabilities). Currently automatic111 and other popular GUIs are pretty focused on users with beefy GPUs that can sample images in seconds, so they haven't yet implemented some of these really significant CPU-friendly speedups. But we'll get there!
- mosselman 3y agoAre those cpu speedups out there? Where can I find them?
- brucethemoose2 3y agoNot for A1111's SAI backend, as far as I know. VoltaML has some CPU optimized options for the torch.compile optimization.
- KeplerBoy 3y agoThat's probably exactly what you would expect from a GPU running off DDR4 instead off GDDR6X.
- bongobingo1 3y ago> Stable Diffusion in particular doesn't need much VRAM anyway. I get that many people are stuck on lower end computers, but ~4GB is not a totally unreasonable requirement Wow, did it really get that low? I played with SD during the initial days, and I know there was some strides then to get it running on 8gb cards and maybe squeaking through on 6gb with a smaller image size (like, 256x256px). Can you generate reasonably large images on 4gb now, with just a higher wait time? I was never sure why it wasn't simple to just shuffle bits in and out of main RAM but I guess the model themselves are mostly big binary blobs that are hard to split up?
- deleted 3y ago[deleted]
- jandrese 3y agoYeah, I’m not sure where that figure came from either. My experience on an 8GB card is that it is always nudging up close to the limit and you don’t want to try to turn the knobs up much past the default. SDXL is a total nonstarter as well on 8GB.
- ColonelPhantom 3y agoUsing vladmandic sd.next with v1-5 works fine* on my RX 580 4GB at around 1s/it on ROCm and 1.5s/it on DirectML. fine = requires --medvram or --lowvram in order to not run out of memory. The speeds above are with medvram, which is just about on the edge of what's possible. These options compromise speed but are still better than running on the CPU generally.
- brucethemoose2 3y agoVoltaML is also quite good on low vram cards.
- ColonelPhantom 3y agoDoes VoltaML have any support for AMD/Intel GPUs or CPU inference? e.g. I think Intel's openVINO supports ONNX, but the Github page only mentions Nvidia and CUDA.
- brucethemoose2 3y agoThe docs are somewhat out of date, and TBH I'm not sure about openVINO I know some are running it with ROCm.
- brucethemoose2 3y ago> Can you generate reasonably large images on 4gb now, with just a higher wait time? You are stuck at low res, but you can use a GAN to upscale the output.
- mrtksn 3y agoIt's also significantly slower than Apple M1, which is 2.2 iteration per second in iPad. Though, maybe it's not a fair comparison since Apple optimises heavily their library. [0] https://github.com/apple/ml-stable-diffusion https://github.com/apple/ml-stable-diffusion
- Zpalmtree 3y agoSDXL needs a lot more VRAM however, especially when training LoRAs