3 ms·
Wouldn’t that invalidate kv cache and you wouldn’t benefit from much cheaper cache read?
by pzo 2mo ago
Wouldn’t that invalidate kv cache and you wouldn’t benefit from much cheaper cache read?
- Dayshine 2mo agoI'm frequently idle for five minutes while I review, at which point the cache has expired
- stpedgwdgfhgdd 2mo ago[dead]