4 ms·
>it assumes the exact same context window across requests That is not true, caching works across multiple requests, that's why it's so good. You can do 5 differ
by Tiberium 2y ago
>it assumes the exact same context window across requests
That is not true, caching works across multiple requests, that's why it's so good. You can do 5 different concurrent requests and they'll all get cached and cache read if the cache is still warm for them.