3 ms·
I like the idea of having a standard API for interacting with LLMs over the network. Many models need to run on beefy hardware and would benefit from offloading
by Ajedi32 3y ago
I like the idea of having a standard API for interacting with LLMs over the network. Many models need to run on beefy hardware and would benefit from offloading to a remote (possibly self-hosted) server, and I think makes logical sense to separate the code for running LLMs from the UI for accessing them.
- golergka 3y agoIt would be great to have this, but the space is rapidly moving and haven't converged on a set of uniformly accepted practices yet. For example, I'm not aware of a single open source LLM that has something similar to OpenAI's function calls.