Seems like fundamentally a cool idea but: KV is not context. if the KV cache gets evicted, you'd have to rerun the translation. Seems like it could still help

Seems like fundamentally a cool idea but: KV is not context. if the KV cache gets evicted, you'd have to rerun the translation. Seems like it could still help but also feels like one of those things where it becomes vastly more complex and difficult to debug.