3 ms·

https://github.com/abetlen/llama-cpp-python https://github.com/abetlen/llama-cpp-python has a web server mode that replicates openai's API iirc and the readme s

by nodja 3y ago

https://github.com/abetlen/llama-cpp-python https://github.com/abetlen/llama-cpp-python has a web server mode that replicates openai's API iirc and the readme shows it has docker builds already.

TheRealClay 3y ago
Thanks! As someone just getting started, I really appreciate the tip!