Caching itself is introducing a discontinuity, because whether a request does or does not hit the cache will have vastly different performance profiles (and if

Caching itself is introducing a discontinuity, because whether a request does or does not hit the cache will have vastly different performance profiles (and if not, then the cache may be a bit useless). I think the only way to approach this problem is statistically, but average is a bad metric. I think you’d care about some high percentile instead.