6 ms·
I wrote a bit about this the other day: https://simonwillison.net/2025/Jun/27/context-engineering/ https://simonwillison.net/2025/Jun/27/context-engineering/ D
by simonw 1y ago
I wrote a bit about this the other day: https://simonwillison.net/2025/Jun/27/context-engineering/ https://simonwillison.net/2025/Jun/27/context-engineering/
Drew Breunig has been doing some fantastic writing on this subject - coincidentally at the same time as the "context engineering" buzzword appeared but actually unrelated to that meme.
How Long Contexts Fail - https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-how-to-fix-them.html https://www.dbreunig.com/2025/06/22/how-contexts-fail-and-ho... - talks about the various ways in which longer contexts can start causing problems (also known as "context rot")
How to Fix Your Context - https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.... - gives names to a bunch of techniques for working around these problems including Tool Loadout, Context Quarantine, Context Pruning, Context Summarization, and Context Offloading.
- the_mitsuhiko 1y agoDrew Breunig's posts are a must read on this. This is not only important for writing your own agents, it is also critical when using agentic coding right now. These limitations/behaviors will be with us for a while.
- outofpaper 1y agoThey might be good reads on the topic but Drew makes some significant etymological mistakes. For example loadout doesn't come from gaming but military terminology. It's essentially the same as kit or gear.
- ZYbCRq22HbJ2y7 1y ago> They might be good reads on the topic but Drew makes some significant etymological mistakes. For example loadout doesn't come from gaming but military terminology. It's essentially the same as kit or gear. Doesn't seem that significant? Not to say those blog posts say anything much anyway that any "prompt engineer" (someone who uses LLMs frequently) doesn't already know, but maybe it is useful to some at such an early stage of these things.
- simonw 1y agoDrew isn't using that term in a military context, he's using it in a gaming context. He defines what he means very clearly: > The term “loadout” is a gaming term that refers to the specific combination of abilities, weapons, and equipment you select before a level, match, or round. In the military you don't select your abilities before entering a level.
- GuinansEyebrows 1y agoi think that software engineers using this terminology might be envisioning themselves as generals, not infantry :)
- xarope 1y agothe military definitely do use the term loadout. It can be based on mission parameters e.g. if armored vehicles are expected, your loadout might include more MANPATS. It can also refer to the way each soldier might customize their gear, e.g. cutaway knife in boot or on vest, NODs if extended night operations expected (I know, I know, gamers would like to think you'd bring everything, but in real life no warfighter would want to carry extra weight unnecessarily!), or even the placement of gear on their MOLLE vests (all that velcro has a reason).
- simonw 1y agoNobody is disputing that. We are saying that the statement "The term 'loudout' is a gaming term" can be true at the same time.
- DiggyJohnson 1y agoThis seems like a rather unimportant type of mistake, especially because the definition is still accurate, it’s just the etymology isn’t complete.
- scubbo 1y agoIt _is_ a gaming term - it is also a military term (from which the gaming term arose).
- coldtea 1y ago>Drew makes some significant etymological mistakes. For example loadout doesn't come from gaming but military terminology Does he pretend to give the etymology and ultimately origin of the term, or just where he or other AI-discussions found it? Because if it's the latter, he is entitled to call it a "gaming" term, because that's what it is to him and those in the discussion. He didn't find it in some military manual or learned it at boot camp! But I would mostly challenge this mistake, if we admit it as such, is "significant" in any way. The origin of loadout is totally irrelevant to the point he makes and the subject he discusses. It's just a useful term he adopted, it's history is not really relevant.
- luckydata 1y agothis is textbook pointless pedantry. I'm just commenting to find it again in the future.
- pbhjpbhj 1y agoClick on the 'time' part of the comment header, then you can 'favorite' the comment. That way you can avoid adding such comments in the future.
- storus 1y agoThose issues are considered artifacts of the current crop of LLMs in academic circles; there is already research allowing LLMs to use millions of different tools at the same time, and stable long contexts, likely reducing the amount of agents to one for most use cases outside interfacing different providers. Anyone basing their future agentic systems on current LLMs would likely face LangChain fate - built for GPT-3, made obsolete by GPT-3.5.
- simonw 1y agoCan you link to the research on millions of different terms and stable long contexts? I haven't come across that yet.
- storus 1y agoYou can look at AnyTool, 2024 (16,000 tools) and start looking at newer research from there. https://arxiv.org/abs/2402.04253 https://arxiv.org/abs/2402.04253 For long contexts start with activation beacons and RoPE scaling.
- simonw 1y agoI would classify AnyTool as a context engineering trick. It's using GPT-4 function calls (what we would call tool calls today) to find the best tools for the current job based on a 3-level hierarchy search. Drew calls that one "Tool Loadout" https://www.dbreunig.com/2025/06/26/how-to-fix-your-context.html#tool-loadout https://www.dbreunig.com/2025/06/26/how-to-fix-your-context....
- timr 1y agoSo great. We have not one, but two different ways of saying "use text search to find tools". This field, I swear...it's the PPAP [1] of engineering. [1] https://www.youtube.com/watch?v=NfuiB52K7X8 https://www.youtube.com/watch?v=NfuiB52K7X8 I have a toool...I have a seeeeearch...unh! Now I have a Tool Loadout!" *dances around in leopard print pyjamas*
- 1y ago
- risyachka 1y ago“A month-long skill” after which it won’t be a thing anymore, like so many other.
- simonw 1y agoMost of the LLM prompting skills I figured out ~three years ago are still useful to me today. Even the ones that I've dropped are useful because I know that things that used to be helpful aren't helpful any more, which helps me build an intuition for how the models have improved over time.
- refulgentis 1y agoI agree with you, but would echo OP's concern, in a way that makes me feel like a party pooper, but, is open about what I see us all expressing squeamish-ness about. It is somewhat bothersome to have another buzz phrase. I don't why we are doing this, other than there was a Xeet from the Shopify CEO, QT'd approvingly by Karpathy, then its written up at length, and tied to another set of blog posts. To wit, it went from "buzzphrase" to "skill that'll probably be useful in 3 years still" over the course of this thread. Has it even been a week since the original tweet? There doesn't seem to be a strong foundation here, but due to the reach potential of the names involved, and their insistence on this being a thing while also indicating they're sheepish it is a thing, it will now be a thing. Smacks of a self-aware version of Jared Friedman's tweet re: watching the invention of "Founder Mode" was like a startup version of the Potsdam Conference. (which sorted out Earth post-WWII. and he was not kidding. I could not even remember the phrase for the life of me. Lasted maybe 3 months?)
- simonw 1y agoThe way I see it we're trying to rebrand because the term "prompt engineering" got redefined to mean "typing prompts full of stupid hacks about things like tipping and dead grandmas into a chatbot".
- Dylan16807 1y ago
- old_man_cato 1y ago[flagged]
- d0gsg0w00f 1y agoThis hits too close to home.
- _carbyau_ 1y ago[flagged]
- NomDePlum 1y ago[flagged]
- TrainedMonkey 1y agoHire a context engineer to define the task of drawing an owl as drawing two owls.
- zdw 1y ago[flagged]
- crsv 1y agoAnd then the AI doesn’t handle the front end caching properly for the 100th time in a row so you edit the owl and nothing changes after you press save.
- jknoepfler 1y agoOh, and don't forget to retain the artist to correct the ever-increasingly weird and expensive mistakes made by the context when you need to draw newer, fancier pelicans. Maybe we can just train product to draw?
- JoeOfTexas 1y agoSo who will develop the first Logic Core that automates the context engineer.
- igravious 1y agoThe first rule of automation: that which can be automated will be automated. Observation: this isn't anything that can't be automated /
- TZubiri 1y agoRediscovering encapsulation
- dosnem 1y agoProviding context makes sense to me, but do you have any examples of providing context and then getting the AI to produce something complex? I am quite a proponent of AI but even I find myself failing to produce significant results on complex problems, even when I have clone + memory bank, etc. it ends up being a time sink of trying to get the ai to do something only to have me eventually take over and do it myself.
- simonw 1y agoQuite a few times, I've been able to give it enough context to write me an entire working piece of software in a single shot. I use that for plugins pretty often, eg this: llm -m openai/o3 \ -f https://raw.githubusercontent.com/simonw/llm-hacker-news/refs/heads/main/llm_hacker_news.py \ -f https://raw.githubusercontent.com/simonw/tools/refs/heads/main/github-issue-to-markdown.html \ -s 'Write a new fragments plugin in Python that registers issue:org/repo/123 which fetches that issue number from the specified github repo and uses the same markdown logic as the HTML page to turn that into a fragment' Which produced this: https://gist.github.com/simonw/249e16edffe6350f7265012bee9e3305#response https://gist.github.com/simonw/249e16edffe6350f7265012bee9e3...
- AnotherGoodName 1y agoI had a series of “Using Manim create an animation for formula X rearranging into formula Y with a graph of values of the function” Beautiful one shot results and i now have really nice animations of some complex maths to help others understand. (I’ll put it up on youtube soon). I don't know the manim library at all so saved me about a week of work learning and implementing
- Daub 1y agoFor visual art I feel that the existing approaches in context engineering are very much lacking. An Ai understands well enough such simple things as content (bird, dog, owl etc), color (blue green etc) and has a fair understanding of foreground/background. However, the really important stuff is not addressed. For example: in form, things like negative shape and overlap. In color contrast things like Ratio contrast and dynamic range contrast. Or how manipulating neighboring regional contrast produces tone wrap. I could go on. One reason for this state of affairs is that artists and designers lack the consistent terminology to describe what they are doing (though this does not stop them from operating at a high level). Indeed, many of the terms I have used here we (my colleagues and I) had to invent ourselves. I would love to work with an AI guru to address this developing problem.
- skydhash 1y ago> artists and designers lack the consistent terminology to describe what they are doing I don't think they do. It may not be completely consistent, but open any art book and you find the same thing being explained again and again. Just for drawing humans, you will find emphasis on the skeleton and muscle volume for forms and poses, planes (especially the head) for values and shadows, some abstract things like stability and line weight, and some more concrete things like foreshortening. Several books and course have gone over those concepts. They are not difficult to explain, they are just difficult to master. That's because you have to apply judgement for every single line or brush stroke deciding which factors matter most and if you even want to do the stroke. Then there's the whole hand eye coordination. So unless you can solve judgement (which styles derive from), there's not a lot of hope there. ADDENDUM And when you do a study of another's work, it's not copying the data, extracting colors, or comparing labels,... It's just studying judgement. You know the complete formula from which a more basic version is being used for the work, and you only want to know the parameters. Whereas machine training is mostly going for the wrong formula with completely different variables.
- Daub 1y agoI concur that there is, on some matters, a general agreement in art books. However, certainly it does not help that there is so much inconsistency of terminology. For example: the way that hue and color are so frequently used interchangeably, likewise lightness, brightness, tone and value. What bothers me more is that so much truly important material is not being addressed as explicitly as it should be. For example: the exaggeration of contrast on which so much art relies exists in two dimensions: increase of difference and decrease of difference. This application of contrast/affinity is a general principle that runs through the entirety of art. Indeed, I demonstrate it to my students by showing its application in Korean TV dramas. The only explicit mention I can find of this in art literature is in the work of Ruskin, nearly 200 years ago! Even worse is that so much very important material is not being addressed at all. For example, a common device that painters employ is to configure the neighboring regional contrast of a form can be light against dark on one edge and dark against light on the opposing edge. In figurative paintings and in classic portrait photography this device is almost ubiquitous, yet as far as I am able to determine no one has named it or even written about it. We were obliged to name it ourselves (tone wrap). > They are not difficult to explain, they are just difficult to master. Completely agree that they can be difficult to master. However, a thing cannot be satisfactorily explained unless there is consistent (or even existent) terminology for that thing. > So unless you can solve judgement (which styles derive from) Nicely put.
- arbitrary_name 1y agoFrom the first link:Read large enough context to ensure you get what you need. How does this actually work, and how can one better define this to further improve the prompt? This statement feels like the 'draw the rest of the fucking owl' referred to elsewhere in the thread
- simonw 1y agoI'm not sure how you ended up on that page... my comment above links to https://simonwillison.net/2025/Jun/27/context-engineering/ https://simonwillison.net/2025/Jun/27/context-engineering/ The "Read large enough context to ensure you get what you need" quote is from a different post entirely, this one: https://simonwillison.net/2025/Jun/30/vscode-copilot-chat/ https://simonwillison.net/2025/Jun/30/vscode-copilot-chat/ That's part of the system prompts used by the GitHub Copilot Chat extension for VS Code - from this line: https://github.com/microsoft/vscode-copilot-chat/blob/40d039d8e08c2d17435a2e65846120c394d0727b/src/extension/prompts/node/agent/agentInstructions.tsx#L60 https://github.com/microsoft/vscode-copilot-chat/blob/40d039... The full line is: When using the {ToolName.ReadFile} tool, prefer reading a large section over calling the {ToolName.ReadFile} tool many times in sequence. You can also think of all the pieces you may be interested in and read them in parallel. Read large enough context to ensure you get what you need. That's a hint to the tool-calling LLM that it should attempt to guess which area of the file is most likely to include the code that it needs to review. It makes more sense if you look at the definition of the ReadFile tool: https://github.com/microsoft/vscode-copilot-chat/blob/40d039d8e08c2d17435a2e65846120c394d0727b/src/extension/tools/node/readFileTool.tsx#L30-L52 https://github.com/microsoft/vscode-copilot-chat/blob/40d039... description: 'Read the contents of a file. Line numbers are 1-indexed. This tool will truncate its output at 2000 lines and may be called repeatedly with offset and limit parameters to read larger files in chunks.' The tool takes three arguments: filePath, offset and limit.
- daxfohl 1y agoI'm surprised there isn't already an ecosystem of libraries that just do this. When building agents you either have to roll your own or copy an algorithm out of some article. I'd expect this to be a lot more plug and play, and as swappable as LLMs themselves by EOY, along with a bunch of tooling to help with observability, A/B testing, cost and latency analysis (since changing context kills the LLM cache), etc.
- daxfohl 1y agoOr maybe it's that each of these things is pretty simple in itself. Clipping context is one line of code, summarizing could be a couple lines to have an LLM summarize it for you, etc. So not substantial enough for a formal library. Whereas the combinations of these techniques is very application dependent, so not reusable enough to warrant separating as an independent library. Or maybe it just hasn't matured yet and we'll see more of it in the future. We'll see.
- daxfohl 1y agoThough in a way, this feels similar to things like garbage collection, disk defragmentation, or even query planning. Yes, you could build libraries that do these sorts of things for you, but in all likelihood the LLM providers will embed custom-built versions of them that have been battle tested and trained thoroughly to interop well with the corresponding LLM. So whole there could still be an ecosystem, it would likely be a fairly niche thing for very specific use cases or home-grown LLMs. Maybe something like the equivalent of AWS Firecracker for whatever the equivalent of AWS Lambda is in the future LLM world.