2 ms·
why is Azure a no-go? - it's the only other place to get an OpenAI service - you can have dedicated capacity for it if you're big enough and it's worth it - you
by zekone 3y ago
why is Azure a no-go?
- it's the only other place to get an OpenAI service
- you can have dedicated capacity for it if you're big enough and it's worth it
- you can literally turn off public endpoint for it and make it inaccessible, and expose only through a proxy like this
- you can load balance across multiple openai deployments for regional efficiency and rate limit issues
(not trying to push for Azure in general, just trying to understand why even in the OpenAI use case, Azure is a no-go)
- iamjackg 3y agoI'm sorry, I should have been clearer -- I meant that I wish this would be a locally self-hostable observability solution for OpenAI-compatible APIs in general, not something that can only be deployed to Azure and is tied specifically to Azure's OpenAI service. For example, Helicone[0] and llm.report[1] work as generic proxies for the OpenAI API and you can deploy them anywhere. 0: https://www.helicone.ai/ https://www.helicone.ai/ 1: https://llm.report/ https://llm.report/