Yes, we implemented a separate service internally that interfaces with an LLM and so the callers can be agnostic as to what provider or model is being used. Hav

Yes, we implemented a separate service internally that interfaces with an LLM and so the callers can be agnostic as to what provider or model is being used. Haven't needed to load balance between models though.