Yes, there are python bindings for llama.cpp and the text-generation-webui already uses them for local inference. https://github.com/oobabooga/text-generation-w

Yes, there are python bindings for llama.cpp and the text-generation-webui already uses them for local inference. https://github.com/oobabooga/text-generation-webui/wiki/llama.cpp-models https://github.com/oobabooga/text-generation-webui/wiki/llam... "pip install llamacpp" or https://github.com/thomasantony/llamacpp-python https://github.com/thomasantony/llamacpp-python