3 ms·
> I believe we should be "pushing" context to the models rather than giving them directions on how to "pull" the context themselves. Models have very very very
by kiitos 1y ago
> I believe we should be "pushing" context to the models rather than giving them directions on how to "pull" the context themselves.
Models have very very very limited capacity for context, it's one of their primary bottlenecks, therefore it's important to optimize (minimize) that information as much as possible, allowing the model to pull what it decides it needs makes that constraint much easier to satisfy.
- zoogeny 1y ago>what it decides it needs This is what I am suggesting: relying on the model to decide what it needs is maybe not the best use of the available context. It might be better for us to give it the information we are certain it will need.
- kiitos 1y agoI'm talking about situations where the user wants to ask questions about data that's many orders of magnitude larger than what can be included in any LLM request context. I just provide access to the data (via MCP server, RAG server, whatever) and my question/prompt, it's the LLM's responsibility to do everything else.