4 ms·
The above project somewhat supports GPUs if you pass the correct GGML compile flags to it. `GGML_CUBLAS` for example is supported when compiling. You get a dece
by WinLychee 3y ago
The above project somewhat supports GPUs if you pass the correct GGML compile flags to it. `GGML_CUBLAS` for example is supported when compiling. You get a decent speedup relative to pure C/C++.
- brucethemoose2 3y agoInteresting. It still doesn't seem to be very quick: https://github.com/leejet/stable-diffusion.cpp/issues/6 https://github.com/leejet/stable-diffusion.cpp/issues/6 But don't get me wrong, I look forward to playing with ggml SD and its development.
- WinLychee 3y agoYeah for comparison, `tinygrad` takes a little over a second per iteration on my machine. https://github.com/tinygrad/tinygrad/blob/master/examples/stable_diffusion.py https://github.com/tinygrad/tinygrad/blob/master/examples/st...
- brucethemoose2 3y agoIs that on GPU or CPU? 1 it/s would be very respectable on CPU. The fastest implementation on my 2060 laptop is AITemplate, being about 2x faster than pure optimized HF diffusers.
- WinLychee 3y agoThat was on GPU, and there are various CPU implementations (e.g. based on Tencent/ncnn) on github that have similar runtime (1-3s / iteration).