Just compile llama.cpp's server example, and you have a local HTTP API. It also has a simple UI (disclaimer: to which I've contributed). https://github.com/gge

Just compile llama.cpp's server example, and you have a local HTTP API. It also has a simple UI (disclaimer: to which I've contributed). https://github.com/ggerganov/llama.cpp/blob/master/examples/server/README.md https://github.com/ggerganov/llama.cpp/blob/master/examples/...