3 ms·
My bet is that the solution to continuous learning is with external storage. There is a lot of talk about context engineering - but I have not seen anyone takin
by zby 7mo ago
My bet is that the solution to continuous learning is with external storage. There is a lot of talk about context engineering - but I have not seen anyone taking context as the main bottleneck and building a system around that.
This would show that even context engineering is kind of wrong term - because context does not enter the llm in some mysterious way - it goes through prompt and the whole model of passing chat history back and forth is not the most efficient way of using the prompt limitation.
- Centigonal 7mo agoWhat do you mean when you say "external storage?"
- mhl47 7mo ago"External Storage" whatever that is can not be the same as continous learning as it does not have the strong connections/capture the interdepencies of knowledge. That said I think we will see more efforts also on the business side to have models that can help you build a knowledge base in some kind of standardized way that the model is trained to read. Or synthesize some sort on instructions how to navigate your knowledge base. Currently e.g. Copilot tries to navigate a hot mess of a MS knowledge graph that is very different for each company. And due to its amnesia it has to repeat the discovery in every session. No wonder that does not work. We have to either standardize or store somewhere (model, instructions) how to find information efficiently.
- zby 7mo agoThe key to make Copilot useful is to take the limited context problem seriously enough. There are many dimensions to it: https://zby.github.io/commonplace/notes/context-efficiency-is-the-central-design-concern-in-agent-systems/ https://zby.github.io/commonplace/notes/context-efficiency-i... and it should be the starting point for designing the systems that extensively use llms.
- andai 7mo agoWell, did you solve the problem of deciding what to remember? (And I suppose, how to retrieve it? i.e. LLM can retrieve info, but if it doesn't know about something because it hasn't been retrieved yet...)
- zby 7mo agoThere are many techniques - I analyse them at my knowledge base. I wrote 'my bet is that solving ...' - if I have solved it I would not have to hedge like this.