3 ms·
They reduced the cache TTL to one hour so if you leave your prompt sitting idle for an hour at 700,000 tokens the next time you hit enter send it it will be com
by conception 6mo ago
They reduced the cache TTL to one hour so if you leave your prompt sitting idle for an hour at 700,000 tokens the next time you hit enter send it it will be completely uncached and eat a ton of tokens. Something to look at.