3 ms·

We're in the context engineering stone age. You the engineer shouldn't be trying to curate context, you should be building context optimization/curation engines

We're in the context engineering stone age. You the engineer shouldn't be trying to curate context, you should be building context optimization/curation engines. You shouldn't be passing agents context like messages, they should share a single knowledge store with the parent, and the context optimizer should just optimally pack their context for the task description.

nickreese 1y ago
Is there a framework for this?
- jrvarela56 1y ago
  Ive used CrewAI to compose agents, it’s easy to mix and match and it does seem to change context based on roles https://docs.crewai.com/en/guides/agents/crafting-effective-agents https://docs.crewai.com/en/guides/agents/crafting-effective-...
- CuriouslyC 1y ago
  I have one that's currently still cooking, I have good experimental validation for it but I need to go back and tune the latency and improve the install story. It should help any model quite a bit but you have to hack other agents to integrate it into their api call machinery, I have a custom agent I've built that makes it easy to inject though.
- knlam 1y ago
  The best one is google ADK, I must say they are quite thoughful of all the use cases
  - CuriouslyC 1y ago
    ADK is a nice framework but it's still stuck in the agent as atomic chatbot that goes out and does stuff model. The reality is you want your agents to all be running within an orchestrator service because it's much more efficient in pretty much every way. The way you separate concerns here is to have the agent emit intents, and have those intents go to a queue to be acted upon safely and securely by executors. This system is more secure, performant and observable than the ADK setup.
sippeangelo 1y ago
"should just"?
- CuriouslyC 1y ago
  It's really not hard. It's just all the IR/optimization machinery we already have applied to a shared context tree with locality bias.
hansvm 1y ago
You're not wrong. This is just a storage/retrieval problem. But ... the current systems have limits. If you want commercial success in <3yrs, are any of those ideas remotely viable?
- CuriouslyC 1y ago
  Oh yeah, and if you tried to do one now it'd be a bad idea because I'm almost done :) The agentic revolution is very different from the chatbot/model revolution because agents aren't a model problem, they're a tools/systems/process problem. Honestly the models we have now are very close to good enough for autonomous engineering, but people aren't giving them the right tools, the right processes, we aren't orchestrating them correctly, most people have no idea how to benchmark them to tune them, etc. It's a new discipline and it's very much in its infancy.
- adastra22 1y ago
  Dude, ChatGPT isn’t even 3 years old. That’s an eternity.
kordlessagain 1y ago
Great insight! *We're hand-crafting context like medieval scribes when we should be building context compilers.*
pglevy 1y ago
Not an engineer but I think this is where my mind was going after reading the post. Seems like what will be useful is continuously generated "decision documentation." So the system has access to what has come before in a dynamic way. (Like some mix of RAG with knowledge graph + MCP?) Maybe even pre-outlining "decisions to be made," so if an agent is checking in, it could see there is something that needs to be figured out but hasn't been done yet.
- CuriouslyC 1y ago
  I actually have a "LLM as a judge" loop on all my codebases. I have an architecture panel that debates improvements given an optimization metric and convergence criteria and I feed their findings into a deterministic spec generator (cue /w validation) that can emit unit/e2e tests, scaffold terraform. It's pretty magical. This cue spec gets decomposed into individual tasks by an orchestrator that does research per ticket and bundles it.
  - adastra22 1y ago
    I think we are all building the same thing. If only there was an open source framework for aggregating all our work together.

nickreese 1y ago

Is there a framework for this?

jrvarela56 1y ago
Ive used CrewAI to compose agents, it’s easy to mix and match and it does seem to change context based on roles https://docs.crewai.com/en/guides/agents/crafting-effective-agents https://docs.crewai.com/en/guides/agents/crafting-effective-...
CuriouslyC 1y ago
I have one that's currently still cooking, I have good experimental validation for it but I need to go back and tune the latency and improve the install story. It should help any model quite a bit but you have to hack other agents to integrate it into their api call machinery, I have a custom agent I've built that makes it easy to inject though.
knlam 1y ago
The best one is google ADK, I must say they are quite thoughful of all the use cases
- CuriouslyC 1y ago
  ADK is a nice framework but it's still stuck in the agent as atomic chatbot that goes out and does stuff model. The reality is you want your agents to all be running within an orchestrator service because it's much more efficient in pretty much every way. The way you separate concerns here is to have the agent emit intents, and have those intents go to a queue to be acted upon safely and securely by executors. This system is more secure, performant and observable than the ADK setup.

sippeangelo 1y ago

"should just"?

CuriouslyC 1y ago
It's really not hard. It's just all the IR/optimization machinery we already have applied to a shared context tree with locality bias.

hansvm 1y ago

You're not wrong. This is just a storage/retrieval problem. But ... the current systems have limits. If you want commercial success in <3yrs, are any of those ideas remotely viable?

CuriouslyC 1y ago
Oh yeah, and if you tried to do one now it'd be a bad idea because I'm almost done :) The agentic revolution is very different from the chatbot/model revolution because agents aren't a model problem, they're a tools/systems/process problem. Honestly the models we have now are very close to good enough for autonomous engineering, but people aren't giving them the right tools, the right processes, we aren't orchestrating them correctly, most people have no idea how to benchmark them to tune them, etc. It's a new discipline and it's very much in its infancy.
adastra22 1y ago
Dude, ChatGPT isn’t even 3 years old. That’s an eternity.

kordlessagain 1y ago

Great insight! *We're hand-crafting context like medieval scribes when we should be building context compilers.*

pglevy 1y ago

Not an engineer but I think this is where my mind was going after reading the post. Seems like what will be useful is continuously generated "decision documentation." So the system has access to what has come before in a dynamic way. (Like some mix of RAG with knowledge graph + MCP?) Maybe even pre-outlining "decisions to be made," so if an agent is checking in, it could see there is something that needs to be figured out but hasn't been done yet.

CuriouslyC 1y ago
I actually have a "LLM as a judge" loop on all my codebases. I have an architecture panel that debates improvements given an optimization metric and convergence criteria and I feed their findings into a deterministic spec generator (cue /w validation) that can emit unit/e2e tests, scaffold terraform. It's pretty magical. This cue spec gets decomposed into individual tasks by an orchestrator that does research per ticket and bundles it.
- adastra22 1y ago
  I think we are all building the same thing. If only there was an open source framework for aggregating all our work together.