3 ms·
Improving Throughput by Optimising KV Cache Efficiency for Agentic Workloads
- taintech 17d agoI really like this article, agent harness knows its own state: which tool call is running, whether a sub-agent is pending, when it is finished. The API throws that away because each request looks stateless. Interesting idea.