3 ms·

We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see th

We have made MASSIVE improvements here, and network-wide caching rates have been improving relentlessly. We do publish the cache rates for each endpoint; see the "performance" area of our model pages. E.g. https://openrouter.ai/deepseek/deepseek-v4-flash-0731#performance https://openrouter.ai/deepseek/deepseek-v4-flash-0731#perfor... Open to feedback on how to make this better.

stavros 21d ago
Can I block endpoints whose cache rates are below a certain percentage?
ElectricalUnion 21d ago
Can I block providers (for model, not in general) that have sightly cheaper input/output tokens but have more that 10x the average cache cost? Can I block providers (for a model, not in general) that set up cache write cost when the mode is free cache writes?
randomblock1 21d ago
You know how there's a router mode to use the cheapest provider? That only takes into account uncached rates, last I checked. Make another one that takes into account effective rates (the ones that include cache).
jmward01 21d ago
I think the future -may- be a hosted routing client. Something similar to an auth client app. Why? A few reasons: - Customers being able to decide their own routing with true logic is a huge feature. Open router provides the seamless switching/api, route switching decisions are available in a client. - Similarly, providing hooks at this level gives a chance for stats/other things that are hard to plug into prod code elsewhere - a true middle man hosting for other things like MCP may also turn into a real win once it is implemented. Just a random thought though. My point about quality/cost being clobbered by bad providers remains. The fact that cache and quality is badly handled makes me doubt that training data choices are being respected. You need a more public trust/certification process for providers with real teeth when they cheat. I'm going to wait a bit to see how things evolve and check back later.
kroaton 20d ago
You need to validate what providers are actually serving. Add benchmarks, properly showcase what quantization they are serving on the model and KV cache, etc. Until that happens, your service is doomed to be shitty.

stavros 21d ago

Can I block endpoints whose cache rates are below a certain percentage?

ElectricalUnion 21d ago

Can I block providers (for model, not in general) that have sightly cheaper input/output tokens but have more that 10x the average cache cost? Can I block providers (for a model, not in general) that set up cache write cost when the mode is free cache writes?

randomblock1 21d ago

You know how there's a router mode to use the cheapest provider? That only takes into account uncached rates, last I checked. Make another one that takes into account effective rates (the ones that include cache).

jmward01 21d ago

I think the future -may- be a hosted routing client. Something similar to an auth client app. Why? A few reasons: - Customers being able to decide their own routing with true logic is a huge feature. Open router provides the seamless switching/api, route switching decisions are available in a client. - Similarly, providing hooks at this level gives a chance for stats/other things that are hard to plug into prod code elsewhere - a true middle man hosting for other things like MCP may also turn into a real win once it is implemented. Just a random thought though. My point about quality/cost being clobbered by bad providers remains. The fact that cache and quality is badly handled makes me doubt that training data choices are being respected. You need a more public trust/certification process for providers with real teeth when they cheat. I'm going to wait a bit to see how things evolve and check back later.

kroaton 20d ago

You need to validate what providers are actually serving. Add benchmarks, properly showcase what quantization they are serving on the model and KV cache, etc. Until that happens, your service is doomed to be shitty.