6 ms·
Managing context on the Claude Developer Platform
- andrewstuart 1y agoHopefully one day Anthropic will allow zipfile uploads like ChatGPT and Gemini have allowed for ages.
- mingtianzhang 1y agoEdited version: We try to solve a similar problem to put long documents in context. We built an MCP for Claude to allow you to put long PDFs in your context window that go beyond the context limits: https://pageindex.ai/mcp https://pageindex.ai/mcp.
- Szpadel 1y agothis is about difficulty thing, it is about maintaining knowledge during long execution what is IMO exiting is long term memory in things like Claude code, where model could learn your preferences as you collaborate. (there is already some hard disabled implementation in CC)
- derleyici 1y agoJust a heads-up: HN folks value transparency, so mentioning if it's yours usually builds more trust.
- mingtianzhang 1y agoThanks for the reminder, I have edited the comment.
- siva7 1y agoso this is what claude code 2 uses under the hood? at least i got the impression it stays much better on track than the old version
- npace12 1y agoit doesnt use it yet
- 0wis 1y agoThat’s powerful. Most of the differences I can see between AI generated output and human output comes from the « broad but specific » context of the task. I mean company culture, organization rules and politics, larger team focus and way of working. It may take time to build the required knowledge bases but it must be worth it
- crvdgc 1y agoNice. When using OpenAI Codex CLI, I find the /compact command very useful for large tasks. In a way it's similar to the context editing tool. Maybe I can ask it to use a dedicated directory to simulate the memory tool.
- EnPissant 1y agoClaude Code already compacts automatically.
- crvdgc 1y agoI believe Codex CLI also auto compacts when the context limit is met, but in addition to that, you can manually issue a /compact command at any time.
- brulard 1y agoClaude Code had this /compact command for a long time, you can even specify your preferences for compaction after the slash command. But this is quite limited and to get the best results out of your agent you need more than rely on how the tool decides to prune your context. I ask it explicitly to write down the important parts of our conversation into an md file, and I review and iterate over the doc until I'm happy with it. Then /clear the context and give it instructions to continue based on the MD doc.
- EnPissant 1y agoCodex was only explicit last time I checked
- mritchie712 1y agoCC also has the same `/compact` command if you want to force it
- _joel 1y ago/clear too
- _pdp_ 1y agoInterestingly we rolled out a similar feature recently.
- visarga 1y agoI am working on a World Atlas based approach to computer use agents. If the task and app environment are reused, building an atlas of states and policies might be better than observe-plan-execute. We don't rediscover from scratch how an app works every time we use it.
- _pdp_ 1y agoDo you have any links where I can read more about your approach?
- deepdarkforest 1y agoContext editing is interesting because most agents work on the assumption that KV cache is the most important thing to optimise and are very hesitant to remove parts of the context during work. It also sometimes introduces hallucinations, because parts of the context are with the assumption that eg tool results are there, but theyre not. Example Manus [0]. Eg, read file A, make changes on A. Then prompt on some more changes. If you now remove the "read file A" tool results, not only you break the cache, but in my own agent implementations(on gpt 5 at least) can hallucinate now since my prompt etc all naturally point to the content of the tool still beeing there. Plus, the model got trained and RLed with a continuous context, except if they now tune it with messing with the context as well. https://manus.im/blog/Context-Engineering-for-AI-Agents-Lessons-from-Building-Manus https://manus.im/blog/Context-Engineering-for-AI-Agents-Less...
- javier2 1y agoWe often talk about "hallucinations" like it is its own thing, but is there really anything different about it from the LLM's normal output?
- veidr 1y agoAFAICT, no. I think it just means "bad, unhelpful output" but isn't fundamentally different in any meaningful way from their super-helpful top-1% outputs. It's kind of qualitatively different from the human perspective, so not a useless concept, but I think that is mainly because we can't help anthropomorphizing these things.
- blixt 1y agoYes we had the same issue with our coding agent. We found that instead of replacing large tool results in the context it was sometimes better to have two agents, one long lived with smaller tool results produced by another short lived agent that would actually be the one to read and edit large chunks. The downside of this is you always have to manage the balance of which agent gets what context, and you also increase latency and cost a bit (slightly less reuse of prompt cache)
- yunohn 1y agoWhy are both this new Memory API and the Filesystem as (evolving) Context releases only for the Developer API - but not integrated into Claude Code?
- kylegalbraith 1y agoI had this same thought. I’m not entirely following how I’m to differentiate between these two things. I guess the API is to create my own CC type agent. But I’ve heard of folks creating agents that are CC based as well.
- ec109685 1y agoHow do you know it’s not integrated into Claude code yet?
- simianwords 1y agoI’m trying to understand what part of this is something we could not have hacked together already as clients? Maybe new sonnet is rl’ed to be able to use these memories in a better way?
- olliem36 1y agoAt Zenning AI, a generalist AI designed to replace entire jobs with just prompts. Our agents typically run autonomously for hours, so effective context management is critical. I'd say that we invest most of our engineering effort into what is ultimately context management, such as: 1. Multi-agent orchestration 2. Summarising and chunking large tool and agent responses 3. Passing large context objects by reference between agents and tools Two things to note that might be interesting to the community: Firstly, when managing context, I recommend adding some evals to our context management flow, so you can measure effectiveness as you add improvements and changes. For example, our evals will measure the impact of using Anthropics memory over time. Thus allowing our team to make a better informed decisions on that tools to use with our agents. Secondly, there's a tradeoff not mentioned in this article: speed vs. accuracy. Faster summarisation (or 'compaction') comes at a cost of accuracy. If you want good compaction, it can be slow. Depending on the use case, you should adjust your compaction strategy accordingly. For example, (forgive my major generalisation), for consumer facing products speed is usually preferred over a bump in accuracy. However, in business accuracy is generally preferred over speed.
- _joel 1y agolol, good luck with that
- iamblessed_ 1y agoI want to really get into anthropic. For context: I have background in CV and ML in general. Currently reviewing and revising RL. Any idea how I can get into RL? I have 3 years of industry/research experience. Whenever I see post like this, it triggers a massive fomo creating a scene of urgency on I should work in these problems. Not being able to work here is making be anxious. what does it take for someone in Non-US/Non-EU region to get into big labs such as these? Do I really have to pursue PhD? I am already old that pursuing PhD is a huge burden that I can't afford.
- barrenko 1y agoThe pace is so fast, if you have FOMO you've already missed out most probably. If you're interested in LLM flavored RL, I'd suggest prime-rl (and their discord) community, hugging face RL courses with smol (you'll need pro and burn a couple of bucks), etc. etc.
- iamblessed_ 1y agoWilling to burn few bucks here and there for the projects. Really need to get hands dirty here. I remember taking RL course from coursera during 2020 covid. I didn't have the chance to apply it in the problems I worked post covid. But I really want to start doing RL again. Interested in world models and simulation for RL.
- barrenko 1y agoIf you're good on theory (meaning able to read theory), but interested in non-LLM stuff, pufferlib and it's programmer also seem quite accessible (if you haven't come across it already).
- siva7 1y agoI'll address something else: Fomo is usually a symptom of an underlying deeper issue. Long before Anthropic/Openai, we had these same posts about people desperately wanting to get into Google. They got unhealthy obsessed about this goal that they started prepping for months, even years, documenting their journey on blogs only to get rejected by someone who spent 2 seconds on their application. Getting in is more about luck than most people realize. Once you're in, you had for a long time a skewed romanticized fantasy of what it's like to work at this mythical company (Anthropic isn't a startup by any stretch anymore) only to crash hard and realize it is a classic corporate environment.
- deleted 1y ago[deleted]
- RamtinJ95 1y agoI don’t get it. I have been doing something similar for a month with opencode, is the new thing that the new sonnet model is fine tuned to call these tools “better” or simply that they have improved the devex to accomplish these things?
- the_mitsuhiko 1y ago> I don’t get it. I have been doing something similar for a month with opencode, is the new thing that the new sonnet model is fine tuned to call these tools “better” or simply that they have improved the devex to accomplish these things? They fine tuned 4.5 to have `clear_tool_uses` marker tools that it understands without regressing the quality of future responses. You will however pay for the cache invalidation hit anyways, so it would need some evaluations how much this helps.
- Razengan 1y agoI just want Anthropic to release a way to remove my payment method from Claude before the eventual data breach.
- Aeolun 1y agoDon’t they use Stripe? You can remove your payment method at any time. But Anthropic doesn’t really have your info in the first place.
- Razengan 1y agoHow? First I wanted to just use iOS In-App-Purchases like ChatGPT and Grok etc all support, but Anthropic has this "We're too special" syndrome and don't offer IAP so I had to sign in on the website (and guess what, iOS app supports Sign In With Apple but the website doesn't) and add my card… but no way to remove the card later. Steam and others have figured it out, but Anthropic/Discord (who just had a breach like yesterday) still don't let you remove your payment info.
- mh- 1y agoMy Claude subscription is through iOS IAPs.
- Razengan 1y agoI could swear I didn't even see the option for an IAP on iOS… maybe it's region-locked?
- mh- 1y agoMaybe, but I wouldn't suggest doing it even if you figure it out. It has some really annoying side effects on subscription management for Claude, and additionally they mark up the plan cost by 20%. Which I didn't care about on the $20 Pro, but now with $200 plans. I haven't figured out* how to switch to a direct subscription other than cancelling and resubscribing, and I'm afraid of messing up my account access. * Caveat that I haven't spent more than 20 minutes trying to solve this.
- blixt 1y agoFrom what I can tell the new context editing and memory APIs are essentially formalization of common patterns: Context editing: Replace tool call results in message history (i.e replace a file output with an indicator that it’s no longer available). Memory: Give LLM access to read and write .md files like a virtual file system I feel like these formalizations of tools are on the path towards managing message history on the server, which means better vendor lock in, but not necessarily a big boon to the user of the API (well, bandwidth and latency will improve). I see the ChatGPT Responses API going a similar path, and together these changes will make it harder to swap transparently between providers, something I enjoy having the ability to do.
- mkagenius 1y ago> managing message history on the server, which means better vendor lock in I feel that managing context should be doable with a non-SOTA model even locally. Just need a way to select/deselect messages from the context manually say in Claude-CLI.
- qwertox 1y agoI wish every instruction and response had a enable/disable checkbox so that I can disable parts of the conversation in such a way that it is excluded from the context. Let's say I submit or let it create a piece of code, and we're working on improving it. At some point I want to consider the piece of code to be significantly better that what I had initially, so all those initial interactions containing old code could be removed from the context. I like how Google AI Studio allows one to delete sections and they are then no longer part of the context. Not possible in Claude, ChatGPT or Gemini, I think there one can only delete the last response. Maybe even AI could suggest which parts to disable.
- diggan 1y agoI kind of do this, semi-manually when using the web chat UIs (which happens less and less). I basically never let the conversations go above two messages in total (one message from me + one reply, since the quality of responses goes down so damn quick), and if anything is wrong, I restart the conversation and fix the initial prompt so it gets it right. And rather than manually writing my prompts in the web UIs, I manage prompts with http://github.com/victorb/prompta http://github.com/victorb/prompta which makes it trivial to edit the prompts as I find out the best way of getting the response I want, together with some simple shell integrations to automatically include logs, source code, docs and what not.
- tortilla 1y agoI work similarly. I keep message rounds short (1-3) and clear often. If I have to steer the conversation too much, I start over. I built a terminal tui to manage my contexts/prompts: https://github.com/pluqqy/pluqqy-terminal https://github.com/pluqqy/pluqqy-terminal
- Jowsey 1y agoRelated, it feels like AI Studio is the only mainstream LLM frontend that treats you like an adult. Choose your own safety boundaries, modify the context & system prompt as you please, clear rate limits and pricing, etc. It's something you come to appreciate a lot, even if we are in the part of the cycle where Google's models aren't particularly SOTA rn
- mdrzn 1y agoI noticed this in the Claude Code interface, I reached "8% context left" but after giving it a huge prompt the warning disappeared, and it kept on working for another 20 minutes before reaching again "10% context left", but it never had to compact the history of the conversation. 10/10 great feature.
- CuriouslyC 1y ago> Enable longer conversations by automatically removing stale tool results from context Boost accuracy by saving critical information to memory—and bring that learning across successive agentic sessions Funny, I was just talking about my personal use of these techniques recently (tool output summarization/abliteration with memory backend). This isn't something that needs to be Claude Code specific though, you can 100% implement this with tool wrappers. I've been doing this for a bit, dropping summarized old tool output from context is a big win, but it's still level ~0 context engineering. It'll be interesting to see which of my tricks they figure out next.
- hendersoon 1y agoI wish claude code supported the new memory tool. The difference is CLAUDE.md is always in your active context while the new memory stuff is essentially local RAG.
- airocker 1y agoWould this mean cursor and cline don’t have to do context management? Their value is much more just in the ui now?
- josvdwest 1y agoI don't think cline ever had to do much context management, which is why i prefer them over cursor. Cline is similar to claude code, but without the vendor lock in.
- ChicagoDave 1y agoThis is hilarious. It’s like they took my usage pattern and made it native. Love it.
- gtsop 1y agoI'll let them (ai companies) figure it out while i keep coding manually. I'll wait for the day they will release a js library to programmatically do all this llm context juggling, instead of using a UI, and then I will adopt it by doing what I do now, writting code. I will write code that orchestrates llms for writting code. Edit: This is obviously a joke... but is it really a joke?
- sarmasamosarma 1y ago[dead]
- daxfohl 1y agoWhy client side? Doesn't that mean re-uploading the whole context whenever you change something? Seems like storing it server-side and providing an API to swap pieces in and out of context would be a lot more efficient. Or if the LLM is the one deciding what to swap, that could be done without client involvement at all. Even better would be if this API was standardized across providers. Outside of context manipulation, it'd also be nice to standardize a format to label sections of the context and later append keywords to the context to ignore / unignore those sections (or refer back to them, emphasize them, whatever). With that, I imagine we'd be able to have a pretty good set of standard LoRA adapters that enable all LLMs to be controlled in the same fashion. That way agents will be able to manipulate LLMs in a standard way, without having to rewrite context itself.
- selinkocalar 1y agoContext management is one of those things that seems simple until you actually build with it. We've run into issues where Claude loses important context halfway through complex tasks. Love the idea of being able to mark parts of the conversation as 'resolved' or 'outdated' would be huge.