2 ms·
If I were to self host an open source model like mistral or llama , are there options similar to this as an api gateway to proxy and authenticate , create api k
by weekay 3y ago
If I were to self host an open source model like mistral or llama , are there options similar to this as an api gateway to proxy and authenticate , create api keys , monitor spends by api etc .,? How are people running open source LLM”s in production ? Thanks
- donovan-so 3y agoWe do support self-hosted models, as long as they're exposed as an API. Would this endpoint work for you? https://github.com/bricks-cloud/BricksLLM?tab=readme-ov-file#custom-provider-proxy https://github.com/bricks-cloud/BricksLLM?tab=readme-ov-file...