3 ms·
As far as what 'API' to use, please just coalesce around the OpenAI API for your client software. You can start up an OpenAI compatible endpoint with vLLM for e
by Tostino 2y ago
As far as what 'API' to use, please just coalesce around the OpenAI API for your client software. You can start up an OpenAI compatible endpoint with vLLM for example. Just stick with that. You can use LiteLLM as a proxy to convert your client side requests to whatever server side format is expected for e.g. Claude.