3 ms·
I find this 1M context bollocks. It's basically crap past 100k.
by varispeed 6mo ago
I find this 1M context bollocks. It's basically crap past 100k.
- _blk 6mo agoI like not running into the mandatory compaction but I do try to actively keep it under too. From an Anthropic standpoint with the new(ish) 5min cache timeout, it's a great way to get people to burn tokens on reinitializing the cache without having them occupy TPU time.. Esp. the larger the context gets.