10 ms·
When the prefix matches a request sent to the same Providor. The thing is the TTL is different for each provider, some cache for 5 minutes some cache for 1hr. I
by dakolli 2mo ago
When the prefix matches a request sent to the same Providor. The thing is the TTL is different for each provider, some cache for 5 minutes some cache for 1hr. Its ideal to only use one provider per agent session / and per model with the best cache hit % if you care about costs.
- andai 1mo agoBut before 5m the hit rate is 100%, and after it's 0%? Why is there a probability? Is there some stochastic process that takes place during those 5 minutes that determines whether or not you get the discount?
- RussianCow 1mo agoPresumably the number that OpenRouter shows is averaged across all requests.