Thanks for sharing! We've been recommending something similar with these llama-cpp-python bindings https://github.com/abetlen/llama-cpp-python#web-server https:

Thanks for sharing! We've been recommending something similar with these llama-cpp-python bindings https://github.com/abetlen/llama-cpp-python#web-server https://github.com/abetlen/llama-cpp-python#web-server, but I'll have to check that out