3 ms·
> This behavior makes it so you don't benefit much from the caching I don't believe this is correct? AFAIK once it routes you to a provider for a given convers
by fc417fc802 1mo ago
> This behavior makes it so you don't benefit much from the caching
I don't believe this is correct? AFAIK once it routes you to a provider for a given conversation that choice is sticky unless you hit technical difficulties. (It's more complicated than that, they recently added named routing strategies that you can append to the model name.)
IMO the relevant metric is cache TTL which isn't typically published AFAIK.
- RussianCow 1mo agoThis is very much NOT my experience in practice, even though it's how I would expect it to work. OpenRouter will happily bounce you between several providers (none of which have downtime) even within the same session. Requesting specific providers is the only way I've been able to hit a cache rate above 90%.
- glub 1mo agoSame experience here. It would choose 2 providers and then bounce between the two every 5 requests or so. I don't know why there's no "Pick the cheapest provider above nTPS on first request and stick until cache bust" setting.