3 ms·
I don't think that's a use case since you don't get rate limited when using the API.
by Kiro 2y ago
I don't think that's a use case since you don't get rate limited when using the API.
- Onawa 2y agoWe get rate limited when using Azure's OpenAI API. As a gov contractor working with AI, I have limited means for getting access to frontier LLMs. So routing tools that can fail over to another model can be useful.
- fkyoureadthedoc 2y agoSame. Initially we just load balanced between various regions, ultimately bought some PTUs.
- kordlessagain 2y agoAnthropic Build Tier 4: 4,000 RPM, 400,000 TPM, 50,000,000 TPD for Claude 3.5 Sonnet