4 ms·
Related: say I’ve written code that uses OpenAI API, and code that handles streaming, retries, function-calls. And now I want to switch it to using a local non-
by d4rkp4ttern 3y ago
Related: say I’ve written code that uses OpenAI API, and code that handles streaming, retries, function-calls. And now I want to switch it to using a local non-API-based model such as llama2, without changing too much code.
Is there a library that offers a layer on top of local models that simulates the OpenAI API?
- Daviey 3y agohttps://github.com/abetlen/llama-cpp-python#web-server https://github.com/abetlen/llama-cpp-python#web-server
- d4rkp4ttern 3y agoThanks, this was hard to find