3 ms·
I would recommend looking at OpenRouter, if anyone is interested in implementing fallbacks across model providers. I've been using it in several projects, and t
by calebkaiser 2y ago
I would recommend looking at OpenRouter, if anyone is interested in implementing fallbacks across model providers. I've been using it in several projects, and the ability to swap across models without changing any implementation code/without managing multiple API keys has been incredibly nice:
https://openrouter.ai/docs/quickstart https://openrouter.ai/docs/quickstart
- nxrabl 2y agoHave you measured how much latency it adds to each query? Naively I'd expect adding in an extra network hop to be a pretty big hit.
- oaththrowaway 2y agoIt's really pretty reasonable. I don't notice it at all. Nothing as bad as trying to relay 1min.ai or something
- calebkaiser 2y agoAnecdotally, there's been no obvious performance hit, but this is something I should test more thoroughly. I'm planning on running a benchmark across a couple of proxies this week--I'll post the results to HN, if anyone is curious