3 ms·
We get rate limited when using Azure's OpenAI API. As a gov contractor working with AI, I have limited means for getting access to frontier LLMs. So routing too
by Onawa 2y ago
We get rate limited when using Azure's OpenAI API. As a gov contractor working with AI, I have limited means for getting access to frontier LLMs. So routing tools that can fail over to another model can be useful.
- fkyoureadthedoc 2y agoSame. Initially we just load balanced between various regions, ultimately bought some PTUs.