3 ms·
Unless stuff changed since I last checked, context caching just reduces cost / latency. It does not change what tokens are emitted.
by hedora 6mo ago
Unless stuff changed since I last checked, context caching just reduces cost / latency. It does not change what tokens are emitted.