5 ms·
C++ Implementation of StableDiffusion
- drlava 3y agoFor testing purposes of nice self contained packages like this, are pre-trained datasets available?
- krackers 3y agoA bit misleading, it's relying on onnxruntime for the actual inference. That said, given the state of python packaging and the fact that onnxruntime has APIs in C++ and Java, I don't know why more people don't distribute self-contained examples using it.
- version_five 3y agoWhat are the implications of onnxruntime vs something like llama.cpp or those nn-512 models that seem closer to just C code for inference? What's different?
- deleted 3y ago[deleted]
- Jaxkr 3y agoonnxruntime is typically for GPU acceleration while being nearly unusable on CPUs. It’s also better supported (Microsoft) and supports LOTS of APIs. Llama.cpp / ggml (while they support some hardware acceleration) is more focused on commodity hardware like x86 CPUs and Apple M-series silicon.
- sroussey 3y agoHardware acceleration includes: OpenBLAS/Apple BLAS/ARM Performance Lib/ATLAS/BLIS/Intel MKL/NVHPC/ACML/SCSL/SGIMATH and more in BLAS. Also new apple metal implementation in progress in addition to apple accelerate, and is in baseline if you enable it. Also a CUDA implementation.
- jxy 3y agoAny POSIX system that has a C++11 compiler could compile llama.cpp and run it, including all the BSDs. The OS supported by onnxruntime is very limited.
- IshKebab 3y agoIt looks like onnxruntime supports all major OSes and the web.
- pjmlp 3y agoYes, many are finally getting the point that Python "libraries" are actually bindings to native libraries, and any language can have bindings to the same libraries.
- stargrazer 3y agoSeems to be a Microsoft Windows specific implementation. Was hoping for something cross platform on to Linux.
- eclectic29 3y agoAs mentioned in the README it's using onnxruntime for inferencing. Onnxruntime is a available on all platforms including Linux. See https://onnxruntime.ai/ https://onnxruntime.ai/.
- Culonavirus 3y ago> The current codebase and the resulting Nuget packages target Windows and use DirectML, however only small sections of the code utilize Windows specific APIs, and thus could be ported to other platforms with minimal effort.
- bb88 3y agoOkay... what's stopping them from doing so? The effort? If it were trivial, that statement wouldn't exist.
- jchw 3y agoWell, that's a little uncharitable, isn't it? Assuming the developers primarily use Windows, they probably just don't have much priority for it; there's probably a lot of things they could do with the time. Perhaps they don't even have a setup to test it, and even as a staunch Linux user, I can't blame someone for not wanting to mess with all of the GPGPU drivers and non-sense. Just trying to be fair here.
- bb88 3y ago> I can't blame someone for not wanting to mess with all of the GPGPU drivers and non-sense. Thanks for the downvote and for proving my point for me. That was literally exactly what I said. :) But yeah, if you say something is "simple" and you can't do it yourself, then it's most likely not.
- eclectic29 3y agoSomeone please edit the title as it's really misleading. This linked project is using onnxruntime as opposed to implementing stable diffusion from scratch in C++. Meaning this is not in the same spirit as llama.cpp.
- cloudhan 3y agoonnxruntime is written in C++. So it is not misleading =)
- kramerger 3y agoAlso, doesn't llama.cpp use ggml.cpp ? Or at least has parts of that project copy pasted into its tree?
- Zetobal 3y agoYes, so llama.cpp is also not in the same spirit as the llama.cpp version op has in his head.
- suavesito 3y agoYes, both were created by the same person. On llama.cpp README says > This project is for educational purposes and serves as the main playground for developing new features for the ggml library. (Edit, typo.)
- smrtinsert 3y agoAssuming SD models are converted to the onnx format (or whatever its called) shouldn't current sd projects offer similar performance? The dominant cost is in generation is inference, not the project runtime I thought.
- MichaelDickens 3y agoI think the purpose isn't to improve performance but to make it not depend on a bunch of Python packages, which can be difficult to install.
- javchz 3y agoThis. Conda and Venv in paper should be like docker plug and play... But in practice they can be a hit or miss. Still better than manual global library management.
- physicsguy 3y agoConda requires a commercial license too
- erhaetherth 3y agoFwiw it's not impossible to package all this junk up in Docker. I did it recently. You have to run Docker w/ a whole bunch of flags though, like `--gpus all --ipc=host --ulimit memlock=-1 --ulimit stack=67108864 -p 7860:7860`. And the GPU isn't available during build time unless you do some hacks. But you can build the CUDA junk into the docker image along with Python and all the libs. Combine that with a .bat or .sh to contain the giant `docker run` command and you have a half-OK solution. Aside from having to start docker before being able to run it.
- greentext 3y agoI no longer feel like a crazy person for implementing other ML stuff in C some years ago.
- mem0r1 3y agoIt's really nice to see that C++ implementations in the ML field are growing. Personally I do not like Python very much. C++ does not force you to do things in a specific (Pythonic) way.
- pjmlp 3y agoGrowing? You're always free to skip Python and reach out for the actual library instead of Python bindings. And since C++17, it is quite easy to write Python like code.