4 ms·

There are plenty (perhaps far too many) tools for doing basically `curl` to OpenAI. Local LLM tools are needed however; and are much better for deploying system

by chaxor 3y ago

There are plenty (perhaps far too many) tools for doing basically `curl` to OpenAI. Local LLM tools are needed however; and are much better for deploying systems on terabytes of data at fractions of the cost.

simonw 3y ago
Yeah, that's on my roadmap for "llm " (hence the name) - I want to be able to use the same tool to execute against local models as well. Everything that goes through the tool can be logged to SQLite so this should make it easier to build up comparisons of different models.
- dndn1 3y ago
  +1 for more llms and local llms
- gdubya 3y ago
  Does it work with LocalAI [1] if you change the openai.api_base value to http://localhost:8080/ http://localhost:8080/ ? 1. https://github.com/go-skynet/LocalAI https://github.com/go-skynet/LocalAI