3 ms·
So the model route layer should be something that you would have to internally build on the inference POV - like opting for an OpenRouter or creating your own A
by hackerbone 4mo ago
So the model route layer should be something that you would have to internally build on the inference POV - like opting for an OpenRouter or creating your own AI Gateway.
I've created an ai gateway myself with fallback models - this manages TPM/RPM rate limits and works seamlessly!