Seems like there'd be an opportunity for open source tooling here. Context visualized, summarizers, explorers, A/B testers, etc. Also LLM pre-caching of context

Seems like there'd be an opportunity for open source tooling here. Context visualized, summarizers, explorers, A/B testers, etc. Also LLM pre-caching of context summaries since IIUC any context change requires full N^2 recalculation of everything so adds a ton of latency and cost. And some optimizers since the previous N is actually (N-M) where M is the the first 0..M context tokens that were unchanged by your update. Though generally you probably want to summarize more of the beginning of the context, so M is likely small in most cases. Anyway, seems like most of these algorithms are fairly ad hoc things built into all the various agents themselves these days, and not something that exist in their own right. Seems like an opportunity to make this it's own ecosystem, where context tools can be swapped and used independently of the agents that use them, similar to the LLMs themselves.