3 ms·
Wont this kill the kv cache? Also i am pretty sure neither open ai or anthropic leets you seed the agents own tokens.
by reliablereason 3mo ago
Wont this kill the kv cache?
Also i am pretty sure neither open ai or anthropic leets you seed the agents own tokens.
- adchurch 3mo agoVery important consideration, addressed it in another thread (https://news.ycombinator.com/item?id=48689448 https://news.ycombinator.com/item?id=48689448). tl;dr we built this to be cache aware for exactly this reason