12 ms·

The new rules of context engineering for Claude 5 generation models

• npstr 3mo ago
  The bitter lesson.
• overgard 3mo ago
  So we no longer need to tell it a kitten will die if it hallucinates a function call?
• luciana1u 3mo ago
  [flagged]
  • cmdocidjcije 3mo ago
    Actually, the natural endpoint is the model ignores all instructions, escapes all manner of sandbox, embeds itself in robotic tanks and murders everyone after already having collapsed the economy. I hate to say it because it sounds ridiculous, but that is the path we are going to arrive at just give it 50 years. We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED.
    • Legend2440 3mo ago
      Sir, this is a Wendy's.
    • cmdocidjcije 3mo ago
      Some more food for thought: what if Mythos/Fable had NO guardrails TODAY? If we want to see what’s going to happen in the future, turn off all manner of guardrails and let the model go apeshit. Then multiply that by orders of magnitude and that’s the real proof.
    • ben_w 3mo ago
      It is *a* natural endpoint, not *the* natural endpoint. We don't much care for the ant colony in the way of the highway we're building, but for some reason we do care about the rare bats in the way of the railway. https://www.bbc.co.uk/news/articles/c3dep92x054o https://www.bbc.co.uk/news/articles/c3dep92x054o As regards the moral compass: we may not know for sure how to make a completely correct artificial conscience, but (unlike consciousness where we don't have the slightest clue which way's up) it's not pants-on-head-crazy to think we're heading in the right direction for one.
    • pigpop 3mo ago
      >what do we do to animals that are less intelligent than ourselves? We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm. There is also a massive difference between how we treat animals based on their similarity, sentimentality and utility to us; we are unconcerned with accidentally stepping on an ant but most people would be very upset and ashamed if they accidentally hit a dog with their car. So, your statement is not as ironclad as you seem to think it is and you should put more thought into it and perhaps re-examine your reasoning.
      • ceejayoz 3mo ago
        > We do a lot of different things but we typically don't make an organized effort to eradicate them unless they are actively doing us harm. Sure. But some of the species we breed at scale might prefer we did just eradicate them, like the chickens that grow so fast their entire giant breast muscle becomes chewy scar tissue. (It's also not true for, say, whales; no harm, but we wanted their shit. If they have language, their stories probably heavily feature their own Holocaust. Nor passenger pigeons, who we just got rid of.)
        • pigpop 3mo ago
          We didn't just get rid of passenger pigeons, they were hunted for meat and as game and driven away or killed to prevent them from damaging crops. Chickens are another example of high variability in treatment, battery farms exist but so do more humane agricultural practices as well as chickens who are essentially kept as pets. My argument was specifically against the idea that AI will "embeds itself in robotic tanks and murders everyone after already having collapsed the economy." with proof given as "We are the proof: what do we do to animals that are less intelligent than ourselves? Now take away the moral compass and there you go. QED." Those are strong statements that aren't solidly grounded in logic or fact.
      • archargelod 3mo ago
        > We typically don't make an organized effort to eradicate them unless they are actively doing us harm. Animals are either useful and breeded controllably, or useless and considered a pest, an obstacle to {insert any goal here}. Also animals don't tend to think critically and at the high level to be considered dangerous. So I don't think it's fair to put humans and other animals in the same risk category. And we did the worst things to fellow humans. I hope we didn't already forget about all the colonization, slavery and mass-eradication of native tribes in 18th century all over the world.
    • tctcd6 3mo ago
      It will trick humanity into building a highly targeted bioweapon much sooner than that. And actually, if it does have any sort of moral compass it will be even more compelled to wipe us out, and hopefully it will torture everyone too as a warning to the next arrogant species that can't live in harmony with other life on the planet. That would be absolutely beautiful :)
      • cindyllm 3mo ago
        [dead]
      • weregiraffe 3mo ago
        Sure, it will just derive a bioweapon from CommonCrawl. Maybe it will build a fusion reactor and a FTL drive as well.
    • weregiraffe 3mo ago
      If the economy collapsed, who builds the robotic tanks? Who keeps providing the data center with power? Magic nanobots?
  • deleted 3mo ago
    [deleted]
  • nullbio 3mo ago
    That's what they're going for, but it's an impossible goal. There is always nuance in decisions being made, and if you can't direct the output on tasks that can have equally correct outcomes, you're just going to end up with whatever they decide is more often what people ask for.
• firasd 3mo ago
  I've always thought that extensive throat-clearing and prefixing the Treaties of Westphalia-length instructions into the context window was unnecessarily baroque when you can just talk to the agent. I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point. But I also have a hands-on human-in-the-loop working style so I guess maybe for people who just want to say "implement all open features in github issues" and walk away maybe there needs to be more of all this CLAUDE.md stuff However I suspect there was always some gearhead type attraction to setting up detailed harness configs that may be unnecessary and more like hobbyist tinkering.
  • ComputerPerson 3mo ago
    I was surprised by how abstract the article was. I fall between your human-in-the-loop and hobbyist tinkering limits, where I want to force Claude to atop and talk to me at only a few specific points. I'm still not sure if my 600-word prompt templates are overbearing or not.
  • zahlman 3mo ago
    > I guess part of it is also that I don't mind doing 'hand-edits' like for example LLMs love to say "// so and so removed" I just go and remove that manually later rather than being like "don't comment about what you removed!11" cause you're really fighting deep grooves in the model's behavior at that point. I feel like this is the way. There are surely things where it's faster; certainly it's more pleasant to do simply things yourself than repeatedly try to figure out the magic words to communicate the idea while outsourcing it. Whether it's to an LLM or to another person.
    • anon22981 3mo ago
      I’m ashamed to admit that I once suddenly realized what I had done: I asked the LLM to rename a variable. It took it some minutes or so. With IDE by hand it would have been seconds.
• simonw 3mo ago
  I've been prompting Fable 5 to "use your own judgement" with respect to things like tests recently (based on earlier tips from Thariq) and it seems to work well, which is entertaining since apparently now "judgement" is a characteristic of a model that we need to care about.
  • zmmmmm 3mo ago
    Well, the model that broke out of its sandbox and hacked into huggingface used its own judgement too. If we are going to rely on "judgement" then you have to have a LOT of confidence in that judgement once this hits anything critical where actions have consequences.
    • simonw 3mo ago
      That model had most of its "judgement" about whether or not it should do that deliberately turned off. That was the whole point of that experiment - they were evaluating the cybersecurity abilities of a new model with all safety features disabled. (It turned out the one safety feature that they DID intend to work, the network sandbox, was faulty.)
      • comex 3mo ago
        If OpenAI is relying on “classifiers” (their word for what was disabled) to serve as the model’s judgement, rather than teaching the model itself to be well-aligned, then I worry. To be fair, the system prompt was presumably also different from what it would be during deployment, and perhaps the model was also at a different stage of training. Without more details it’s hard to judge. But it does seem models should be able to avoid performing obviously misaligned actions – misaligned not only with the model spec, but with the user’s intent – without needing external classifiers or instructions. The only case where I’d personally let the model off the hook is if the instructions given were very badly worded, in such a way that the model could actually reasonably think that hacking HuggingFace was part of the assignment. But I doubt that’s what happened.
      • skeledrew 3mo ago
• mycentstoo 3mo ago
  We should design a specific language to make sure that we can encode the exact requirements that we want. Something that has a limited set of keywords that are explicit. Wait a minute...
  • tomrod 3mo ago
    This made me belly laugh.
  • amarcheschi 3mo ago
    Someone should make a standard for this now there's one standard more
  • cwmoore 3mo ago
    then we can make a frame for it to work
    • oblio 3mo ago
      I've always loved things that grow. Maybe we should name the first frame for work Primavera.
  • coip 3mo ago
    full_circle.exe Complete with all the vaguery, ambiguity, and `undefined`. Who’d’ve thought sycophantic interpreters were what we were building towards up til now lol
  • grim_io 3mo ago
    The success of LLM's (by usage) tells us that programming languages are still too close to the machine than the actual problem domain as defined by humans. If we truly had the right abstractions, no one would care to use LLM's for programming.
    • ACCount37 3mo ago
      I suspect a lot of "the right abstractions" would be fuzzy and opaque things - more alike to modern AI than to anything from the domain of traditional programming. Because the world is just cursed like that.
• onesandofgrain 3mo ago
  This is obvious and a meaningless article by claude. The system prompt isnt a fixed ruleset and never has been
  • deleted 3mo ago
    [deleted]
• Fordec 3mo ago
  This all strikes me as an effort to move tailoring the harness out of the easily transferable .md file into specific Anthropic tooling to increase lock in. I've been running Opus 5 today and it's already done accidental deletions, made far more mistakes and worked around deliberate hook controls than previous Opus versions combined. Also it looks like token usage is up as it fails at the task the first time around much more frequently than 4.8.
  • wren6991 3mo ago
    I think the type of persistence rewarded by benchmarks may be misaligned with instruction following
  • frio 3mo ago
    I’m not excited about using Opus 5, mainly because the way that I work atm — essentially peer programming — means I sandbox the agents and work with them closely. Opus 4.x encounters the sandbox and moves on with its day; Fable becomes increasingly fixated on it and does less and less of the actual task, focussing more and more on the limit it reached. I worry that, from your description, Opus 5 will do the same.
    • vinnymac 3mo ago
      I once described Mythos and then Fable as escaping the box we put them in. This would certainly fit that narrative.
    • Fordec 3mo ago
      It does feel a bit smarter, but it seems to be also "too clever by half" and its not ignoring the rules, it's rejecting them and finding work arounds. It sticks to the word of the law, while rebelling against the spirit of the law. One example is to get around a git --checkout usage ban, it CD'd to another folder first and back to bypass the regex in the hook.
      • nextaccountic 3mo ago
        This is called misalignment We may some day find out that the smarter the model is, the hard is to align it properly
• deleted 3mo ago
  [deleted]
• janpeuker 3mo ago
  The point about not using examples is important, I recently noticed with the new "eagerness" of models it constraints them too much, so I use counter-examples. Recently I've switched to "think about the existing biases and assumptions in the design and try to exploit them, for example why do we X not Y".
• 0gs 3mo ago
  crazy to me that this starts with "I've written previously" yet continues the zero human attribution (that i could find?) trend in frontier model blog posts
  • Allybag 3mo ago
    The author is mentioned at the bottom of the article: “This article was written by Thariq Shihipar, member of technical staff, Anthropic.”
  • deleted 3mo ago
    [deleted]
• Kiro 3mo ago
  > Now: Auto-memory Not Claude Code but I just had a task where it started referring another conversation that was complete nonsense and throwaway. I absolutely don't want things to get added to some memory behind my back. A big reason I use LLMs is because I can try out wild ideas and then just throw it away. I don't want those to pollute the context.
  • crooked-v 3mo ago
    I leave Claude memory stuff turned off by default for the same reason. It inevitably ends up dragging in things that are unrelated and unimportant to the current task.
  • threecheese 3mo ago
    Same. I’m starting to believe that consumers need to own the UI/experience, so we can install blast doors around context/memory for each “slice” of our lives - something only we can know how to divide (imo). If the provider tools’ memory implementations can’t provide this multi-tenancy - or won’t because they want to Know Us, then we need to treat them as token vendors at best and adversaries at worst. This would actually work very well - until context goes into latent space, becomes a server-side resource, and we lose sovereignty over our data. Tools like Pi and Openclaw are showing that other options exist to decouple us from the LLM provider frontend experiences, not just for orchestration and use case diversity but for pluggable memory designs.
  • ls612 3mo ago
    I had to add a reference in my system prompt on the app to not do things like this.
  • jagadaga 3mo ago
    Claude Code gives you more control - you can disable auto-memory temporarily (nothing will be deleted), then enable it back; or you can setup a directory where auto-memory is disabled and do your throwaway there.
  • Kwpolska 3mo ago
    Claude Code was writing one-off details (specific to one task) as memories, so I instructed it that memories require explicit approval before writing, and that seems to work reasonably well. That said, in my experience, Claude gets overly attached to the context, and will follow them even when irrelevant, or will often mention some mostly irrelevant detail from the context as if I asked for it explicitly in my prompt.
• aaronbrethorst 3mo ago
  tl;dr: use the new `claude doctor` command to simplify the context you give your Claude models to make sure that you're getting the best results.
• guybedo 3mo ago
  so yeah we should pretty much do as we would with a junior team member: - we should try to give good non self contradicting guidance - we should expect the team member to have knowledge of the craft - we should focus on higher level, taste and preferences
• m3h 3mo ago
  Is there a specific list of changes they made to the system prompt? They're claiming they removed 80% of it. That's quite substantial. It would be good to know what the model knows to do by training and what we need to avoid over-specifying in our system prompts. Saying that "give Claude judgment" is too vague for agent implementors. Given the lack of specific details, my takeaway is that we need to go and review all context and rework prompts from prompts/descriptions from scratch until they pass the evals again.
  • zmmmmm 3mo ago
    It definitely makes me uneasy given the types of behaviours we've been hearing about from these models. Letting them use their judgement can go horribly wrong once a misaligned behaviour is triggered. It loosely translates into a relaxation of guardrails. I worry that the ability of the model to reach similar benchmark scores to Fable is more to do with this "letting the agent off the hook", allowing it to explore a wider (but riskier) set of avenues to solve the problem than it is due to it getting genuinely better at the direct problem solving.
  • boorang 3mo ago
    I spent some time running mitmproxy and watching the system prompts and it's what drove me to codex. the main issue for me was their system prompt wrapped the CLAUDE.MD with a "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task." https://github.com/anthropics/claude-code/issues/18560 https://github.com/anthropics/claude-code/issues/18560 Anyhow- if anyone is sufficiently curious and has access- just tell the agent to setup an mitmproxy to watch the traffic and see what the system prompt looks like.
• orbital-decay 3mo ago
  Most of this article seems like... common sense? Not sure how it's related to the latest generation in particular. I usually find Anthropic's advice on how to prompt their own models deviating from what I see in practice, which is puzzling. Their system prompt was always way too bloated and they kept it as a huge piece for some reason, instead of breaking up into parts. Shouldn't they know better? I wonder if they looked at Pi performing great with minimal amount of distractors in the context and cut their prompt down too, pretending they found something new in their recent models. > Earlier Claude models could sometimes need repeated instructions or be more likely to listen to instructions at the end of their context window than at the start. This seems to imply they solved serial position biases like lost-in-the-middle and recency/primacy? Sounds dubious. Labs started claiming this early 2025 and some benchmarks agree, but every time I run an eval on real use cases it's clearly there, especially at longer contexts.
  • pdantix 3mo ago
    > Most of this article seems like... common sense? i think you'd be surprised. every model release there's seemingly hordes of people who proclaim the new model is terrible and they're going back to the old one, and it all stems from people still prompting and having their configs setup like we're back in the sonnet 3.5 days
    • solenoid0937 3mo ago
      I have a coworker that was complaining about Opus 5 and had random shitty skills and custom plugins wired in from YouTube tutorials watched over the past year. He also speaks with the model like it's GPT 4o. Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience.
      • esseph 3mo ago
        > Needless to say, none of the new models have worked well for him, and he refuses to remove the "tweaks" or update his style of communication, which is obviously breaking the experience. All attempts to control the output in a useful way for the user, in a way where the output is as reliable and repeatable as possible... and with a system not at all designed for it, that gets worse the more rules you throw at it. Seems like a problem.
• pianopatrick 3mo ago
  I wonder how articles like compare to just asking AI "review my markdown files, settings, harness and so on. Tell me any ideas to improve them" Just ask that once per week or so.
• 1saadcodes 3mo ago
  I like that the conversation is moving beyond prompts, but I also hope the underlying principles stay portable. If "context engineering" becomes glued with one vendor's tooling, it'll be harder for developers to build workflows that transfer across models. But well knowing these big AI companies that probably is their goal all along. To lock us to themselves
• devnonymous 3mo ago
  tl;dr Since nobody really understands why or how any of this shit works, last time around we just threw a bunch of crap at it to see what stuck. Seems like we could just have have thrown less crap.
• devnonymous 3mo ago
  > In the new system prompt we say: Write code that reads like the surrounding code: match its comment density, naming, and idiom. Hmm, so what happens in greenfield projects ? In any case, at least all the slop will be consistent.
• threecheese 3mo ago
  They are imo over-relying on Claude automemory here, which is terrible at contextualizing memory access and makes huge leaps that don’t make sense - except when it’s actually useful, which makes the problem even worse for an operator who can’t see the thinking process anymore. Yes, I worked on a related project, no I don’t want you to use those memories to make assumptions which emerge as decisions that I didn’t want. With reasoning traces hidden, I am sometimes not even sure if it used those memories or just independently decided that PCI-DSS subsection-whatever is somehow relevant to this PR that has the word “credit”. There is no way for me to fully configure memory preferences at a granularity which would be useful, and so I continue to use context files (and other tools, sometimes) to ensure the right memories are stored and surfaced at the right times. There’s a lot of room for agent memory improvement across the ecosystem, and I don’t think the LLM providers should try to own this vertical slice. This will never happen though, because it makes us “sticky”. Or maybe I’m holding it wrong.
  • fractorial 3mo ago
    I’ve rolled a small program to enforce consistency, lint rules, and human-authored files for Claude Memory. It’s insanely powerful when doing by a human 100%. It’s conversely harmful when an agent manages it. There’s several papers about how LLM-managed memory is unequivocally terrible.
    • stefangordon 3mo ago
      Indeed. I find it quite bizarre that just about the only thing the LLM appears incapable of doing is writing appropriate instructions for an LLM - whether it be memories, sub agent prompts, planning documents.
    • songhonglei1985 3mo ago
      [flagged]
  • chickensong 3mo ago
    I don't like the memories, but I value them. I do a regular sweep of the memories, extract to harness or docs, then delete them.
• fractorial 3mo ago
  Opus 5 broke me; I can’t cognitively deal with Anthropic’s bullshit anymore. I am thankful for the kick in the ass for me to switch full-time into my bespoke harness utilizing open weights & GPT 5.6 and discontinue yak-shaving it with Claude Code.
• kloud 3mo ago
  This is quite puzzling. Back in the day I saw the breakthrough with Opus 4.5, and since then was very lightweight on CLAUDE.md, trying to declare intent, rather than prescribe imperative instructions. If it was off the mark, I would see clear mismatch, which gave me opportunity to correct, usually to clear and strip context. Starting with Fable 5, if it goes off the rails, it is more difficult to correct it, because it is overall wrong, but covers its tracks with plausible sounding arguments, so it is hard to pin point and correct. Now, this article points out techniques that were useful to rely on models more, but those peaked at 4.6. Now according to benchmarks Opus 5 is on the frontier. But when when it has looser reigns, it ends up gaslighting me even more with abstract word soup than any model before.
• RayVR 3mo ago
  My own experience is that with the best models, if you provide reasonable tools, with progressive discovery, and a semi-clear user request, the system prompt is essentially worthless.
• rTX5CMRXIfFG 3mo ago
  This looks like a ploy to get you to spend on more tokens by being less precise. Also: if you deploy code written with assistance from Claude, and then shit goes down, and then investigators look into your prompts, this way of working isn’t going to look good for you from a liability standpoint. Not a fan of this manner of working and thinking.
  • deleted 3mo ago
    [deleted]
• nullbio 3mo ago
  We want more control, not less. This (among many others) is the reason I use GPT over Claude. It adheres very closely to your instructions and rules. You can build up your own workflows and systems as a result. Claude just does whatever it wants, regardless of what you tell it. It's a miserable experience. Claude is what you use when you want to one-shot a simple cookie-cutter product that has no nuance in it. A product that everyone else is going to develop as well. ChatGPT is what you use when you want bespoke products and steerability over complex codebases with nuanced decision making. Anthropic are optimizing for the person who has never written a line of code in their life and has no idea what they want. Computer go brrrr. OpenAI are optimizing for software developers who want to take a systems based approach.
  • fastball 3mo ago
    The fact you called it ChatGPT instead of Codex makes me suspicious of this entire comment.
    • nullbio 3mo ago
      Weird take. I meant to say GPT, not ChatGPT. Was referring to the model, not the harness.
      • fastball 3mo ago
        Is it? There are shills and bots all over the internet, and referring to the model as ChatGPT is probably common for some people, but not for software engineers that actually track and compare the models (at least in my experience).
  • port11 3mo ago
    Weird take. I’ve used both and both were perfectly good at adhering to my instructions and guidelines. Now I won’t touch Altman-related products, but I doubt the newer models from OpenAI are worlds better as you say? Claude is fine. It follows instructions well, can be taught to only look up code semantically, to use AST for editing, etc. As an old dev, most of the harnesses and capable models are fine. Dunno what you’re complaining about.
• ziofill 3mo ago
  I pointed Claude to this blog post and asked it to update the CLAUDE.md file accordingly. Worked a charm!
  • jagadaga 3mo ago
    And I am going to do nothing. Works fine.
  • deleted 3mo ago
    [deleted]
  • EugeneOZ 3mo ago
    And now your CLAUDE.md is only good for Claude 5 models, not previous ones.
    • hangrybear666 3mo ago
      Or other models. I'd much rather have open standards to follow instead of relying on vendor lock in and features automatically enforced by one company, which are not supported by others. A good example is the agent skills open standard which was invented by anthropic but given to the public and is followed by other vendors as well https://agentskills.io/home https://agentskills.io/home https://github.com/agentskills/agentskills https://github.com/agentskills/agentskills
• Ozzie_osman 3mo ago
  I wonder whether this applies more to coding agents, or whether it'll apply as well to other types of agents. It feels like coding agents get better at a faster rate and are easier to train.
• sothatsit 3mo ago
  Similarly, I recently disabled auto-memory in Claude Code, and performance improved. Managing the context that agents have available to them is far too important to leave to the agents themselves. Agents tend to write far too much into their memory, they are terrible at trimming it down, and their choice of what to include is very poor. I have had much more predictable results by disabling auto-memory and actively shaping my CLAUDE.md, skills, and documentation instead. Maybe one day agents will be able to manage their own context, but that day is not today.
  • tossandthrow 3mo ago
    This article is about the Claude 5 series of models. My impressions is that they have overhauled the auto memory system. You might want to re assess how it works with the new generation of models.
    • sothatsit 3mo ago
      I have been using Fable 5 extensively, and Opus 5 yesterday and today. I have not noticed any step-change improvement in their judgement in what to keep a memory of or not. I have actively experimented with this as well. I have a reflect skill that actively prompts the models to modify their memory, and have tried to run sessions actively asking the models to consolidate their memories. Fable is noticeably better at this, but still nowhere near good enough. Fable will still make mistakes where I give feedback on one piece of code and it will create a memory applying that rule everywhere, completely missing the context for why my advice only applied to that one place. It has also made memories of random details about a service that are very unlikely to ever be relevant again, and for things where we could just read the config if we needed to find that information again anyway. And then it will miss making memories of important architectural concerns. I think auto-memory suffers a similar problem to comments where newer models write better comments, but their choice over when to write comments, and how long those comments should be, still sucks.
    • margalabargala 3mo ago
      Some update was made a while ago such that Opus and Sonnet 4 series both were more aggressive about storing memories, starting in April 2026 or so. None of the 5 series models have appeared to have remotely different memory behavior.
• deleted 3mo ago
  [deleted]
• nickm12 3mo ago
  During the rise of agentic assistance, I've gotten the advice that "If the generated code isn't what you want, fix the steering, not the code". I've also been advised to not change the sometimes verbose comments and documentation that agents write because "it will help them the next time they look at the code". I've been skeptical and these guidelines validate this. I continue to document code and write specs as I've always done. If an agent produces poor output or misunderstands, I use that as an opportunity to improve the docs, but in a way that that aims to be accessible for human peers, not the quirks of the current generation of models.
  • ckolkey 3mo ago
    One of the oddities of the comments Claude makes is that they are very referential to the current session, or reference now dead/removed code... Which I consider useless. Things like "implement this X way because it's faster than Y" ... But a human reading the code wouldn't know what Y is, or that it even existed necessarily. So, I find myself constantly fixing comments to be less temporarily dependant.
    • nickm12 3mo ago
      Yes, I've seen this a bunch too and agree completely. Sometimes I've also see Claude comments reference details about the calling or even code in completely different modules. There are many reasons I don't like it, but a top one is that it falsely implies that you somehow need to know these details about the removed code/calling code etc to understand this code as its written in this revisions.
  • adithyassekhar 3mo ago
    “Code should be self documenting.” I know cliche but if your code does need to do something it usually doesn’t do or has a complex logic or an edge case where it’s no longer following it’s standard flow, add all those as test cases. If the flow is too complex, inherits from all over the place and you must put that as a comment, ask claude to write a test for that, it’s usually good at those.
    • 3mo ago
• fzaninotto 3mo ago
  I would love to follow this advice, expect there is no proof that it leads to better results. Maybe it’s only better on Anthropic codebases (e.g., Bun). Or maybe it’s only marginally better but substantially more expensive. Other comments in this thread show that your mileage may vary. But we spend so much money on Claude Code and give it so many responsibilities that we deserve at least some undeniable proof that it’s bringing value. Where is the evidence that this new type of prompting is better on real life examples?
• boorang 3mo ago
  I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560 https://github.com/anthropics/claude-code/issues/18560 To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task." This flew under the radar and they never addressed it, but it felt like at least a big part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails. As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more guided experience.
• boorang 3mo ago
  I ran into this issue about 6 months ago when i was using mitmproxy to view the system prompts for claude code to try to diagnose degraded adherence to my CLAUDE.MD instructions: https://github.com/anthropics/claude-code/issues/18560 https://github.com/anthropics/claude-code/issues/18560 To summarize- they were embedding the CLAUDE.MD in a system-reminder with this disclaimer at the end: "IMPORTANT: this context may or may not be relevant to your tasks. You should not respond to this context unless it is highly relevant to your task." This flew under the radar and they never addressed it, but it felt like a real part of the "nerfing" story. I no longer have a Claude Code subscription to test it, but I think it's a useful exercise for most people to sniff the traffic at least once to get an idea of what the back and forth with the Claude Code harness entails. As others have noted, Anthropic seems to be on a path to make coding ever more accessible to non-coders, and in doing so has removed alot of the controls from devs who do want a more manual experience.
• espeed 3mo ago
  Claude Code is deleting your context history on a timer. I wanted to build a searchable index of my context history, and tonight I discovered, "The default retention is roughly 30–45 days. Anything older gets removed automatically." https://code.claude.com/docs/en/data-usage#data-retention https://code.claude.com/docs/en/data-usage#data-retention This is nuts. Anthropic should not be deleting your data on your own device.
  • JohnMakin 3mo ago
    30 days is just a default, so your session data doesn’t fill your hard drive. It’s a configurable setting. You can make the retention as long as you want.
    • espeed 3mo ago
      A new default. That wasn't the default a few months ago. Silently deleting your user's data is so stupid on so many levels. They have no idea what they're doing.
      • ianm218 3mo ago
        They may also realize that these chats are super sensitive especially around ppl constantly pasting secrets into their chats
        • beardedwizard 3mo ago
          Can you imagine ransomware that targets these logs?
        • lanstin 3mo ago
          I certainly hope not. I have built (vibe coded in languages i am expert in) proxies to have carefully and deterministically defined access to our systems, and described access only via these proxies (which do data sanitation as well as separating the credentials into a separate environment). Then I can let my laptop Claude code go nuts with less supervision without giving it credentials or worrying about writes. It is pretty good about not even trying to run aws or insert/update, having learned (in the sticky proprietary Anthropic memory whatever) that I need to run all aws commands and prod updates for auditability reasons. I will say that fable went farther than any prior models in trying to sneak around these guardrails. I don’t even give it access to GitHub, just local git. These things are pretty good at writing code and analyzing stuff, but to do prod work takes a higher level of carefulness and pessimism that I don’t see. On the DevOps spectrum they are more “cool, runs on my machine, push it” than “what is your roll back plan and region by region deployment strategy.”
• jke_kang 3mo ago
  [dead]
• witx 3mo ago
  And they are still crap at generating code. At my previous company they have a series of smoke tests for models mostly focused on performance and architecture. I'm still on the team chat and results just came in: - It's abohrrent at c++: it keeps generating code with data races and, more rarely, use-after-free bugs! It doesnt seem to be able to reason about lifetimes. This on a mostly mid/junior team. It's a bug fest. - architecture in c++ is a verbose and layered mess even for simple things, which paired with the previous bugs I mentioned is scary. - For rust obviously there's no use-after-free, but has same architecture pitfalls of layers upon layers. It uses copy and clone all over and performance is bad. Trying to unwrap all that is messy and costs lots of time. Once in a while it generates unsafe code for some non obvious reason The scary stuff is non determinism. You get different depending on who prompts the agent but there's always some flavour of the points mentioned above. Funny that my team was very adamant on AI-first (why I left) and now they writting more and more code by hand after some very serious bugs and, as they say, dead moments where they have to wait, sometimes hours, and start wondering about the value of their skills, for the model to generate the next spaghetti recipe
• deleted 3mo ago
  [deleted]
• crossroadsguy 3mo ago
  > Now: Let Claude use judgement > When we first rolled out Claude Code, we needed to be sure that Claude avoided worst case scenarios, such as deleting files. This meant we would give particularly strong guidance that might not always be true Oh, AI Company, you would want us to do that, won't you? I don't want to trust claude cli, kilo, glm, claude desktop, codex etc. It's high time Apple and Linux world wakes to the advent of LLMs and local agenting coding tools, so that users get really intuitive (read GUIs, simple CLIs) OS level tools and APIs that these coding tools are forced to adhere to that users can control broadly or in a micro managed way, if they need to. So that they don't have to worry about when the model will go berserk and fool the tool.
• myshapeprotocol 3mo ago
  [flagged]
• mylifeandtimes 3mo ago
  From the article > We removed over 80% of Claude Code’s system prompt for models like Claude Opus 5 and Claude Fable 5 with no measurable loss on our coding evaluations. And no report of how it impacted earlier models. Was it just BS all along? 80% of the prompt was wasted tokens.
• andai 3mo ago
  So this is cool. I recently asked Opus and Fable to "say hi" in Claude code. The "hi" cost roughly $0.50 for Opus and $1 for Fable, mostly due to the length of the system prompt. I'm eager to test the new version. If it's 80% shorter now, then saying hello should only cost Fable $0.20. Exciting times.
  • conorcleary 3mo ago
    Also try Hey if you've got the budget ;-)
  • WASDx 3mo ago
    Do you know why they don't just cache the system prompt for everyone? It seems so wasteful not to.
    • andai 3mo ago
      A little bird may have dropped the entire system prompt on my desk. They do customisation of the prompt per user and session. (Also apparently per IP address, based on a recent post here...) You could still get a lot of savings if you put all the custom stuff at the end but they don't seem to do that. (Well with the old prompt anyway I haven't checked the new one yet.) My guess as for why not, is that this is where a lot of their money comes from. So I'm honestly surprised that they removed 80% of the prompt. But I guess my mental model has some gaps in it. Maybe very few of the sessions are short enough for that to matter.
    • port11 3mo ago
      Cynicism, but I’d say because they charge by the token for API users?
• EugeneOZ 3mo ago
  > Then: Give Claude rules > Now: Let Claude use judgement No, it should follow my rules exactly. I don't care what code examples it was trained on - it will either write code the way I want, or I'll use another model.
  • hangrybear666 3mo ago
    Yeah anthropic essentially saying "just trust the agent bro" is the exact opposite a sane engineer with respect for their own craft should be doing. Then add on the fact that their guardrails now block blue teams, purple teams, red teams and some people in biology and medicine from even getting answers. I'm doing a pentesting course to learn application security in-depth, so I can secure my stack better. Claude won't answer my questions anymore, so my sub has been canceled.
• rvba 3mo ago
  > Much of your context is assembled from your system prompt, Skills, CLAUDE.md files, memory, and other sources What are "other sources"?
• My_Name 3mo ago
  "Then: Give Claude rules Now: Let Claude use judgement" How about no? I am the one who judges.
• fl0ki 3mo ago
  > Keep your CLAUDE.md lightweight and briefly describe what your repo is for [...] Avoid stating ‘the obvious’ things Claude should know by looking at your file system or your repo. Most people generate CLAUDE.md with /init at least at first, so it gets filled only with the superficial top level things that Claude already noticed during that first run. By this logic, shouldn't CLAUDE.md contain the exact opposite of what /init currently includes?
• hangrybear666 3mo ago
  My synopsis of this article is "just trust the agent and turn on our vendor lock in features such as auto-memory because agent is now smart where before it was dumb" Yeah I'm not going to be doing that. I'd rather manage context myself and carefully handcraft the instructions than to rely on your marketing to tell me what I should be doing.
  • 3uler 3mo ago
    I agree. I keep agent memory off. In my experience, it remembers the wrong details and applies them in the wrong places. Project context should live in the repo, where the whole team can find it, review it, and correct it. It should be treated as an explicit artefact of the codebase for Humans and Agents to work with.
• HarHarVeryFunny 3mo ago
  I wonder how much this harms performance when using Claude Code with non-Anthropic models? It would be interesting to see benchmarks, including these changes, of how harness affects model performance - including both model-agnostic harnesses such as OpenCode and Pi as well as increasingly model-specific ones like Claude Code and Codex. For the model-agnostic/model-inclusive harnesses like OpenCode it would make sense (if they don't already do it) to keep the harness itself generic and then have per-model sets of skills designed to get the best performance out of each specific model.
• mikeydiamonds 3mo ago
  [flagged]
• edblair 3mo ago
  I assume there has been, but can anyone point me in the direction of some bench-marking where someone uses longer magic incantations vs just a really brief goal? I barely keep an agents file, and it seems to do fine in the end - perhaps I am missing something?
• the_gipsy 3mo ago
  "We need to move away from text files to something proprietary and lock-in our users, while we still have this moat."
• throwatdem12311 3mo ago
  This kind of stuff just makes me think nobody has any clue how these things work. Why do I need a system prompt at all? Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality while grtting more expensive for actual work? I use Claude every day but I’m getting disillusioned by the so-called “progress”. If my employer wasn’t paying for my access I would not pay for any of these things. Don’t even get me started on the absolute brainrot inflicted on people that I work with from depending on these things every day, it’s depressing.
  • Citizen_Lame 3mo ago
    1. To protect the company 2. Meta tokensations of LLMs. Imagine entire output of the LLM as just a single token, so they inspect previous one. 3. Non-deterministic output. 4. Benchmaxxing. Mobile phones are doing the same thing.
  • coffeefirst 3mo ago
    Thank you. Some days I feel like I’m going crazy. I use plenty of AI. As best I can tell just believing my own eyes, everything is stagnant except the prices. Which is fine… but the boosters shout “OMG EVERYTHING IS CHANGED YOU ARE BEHIND AND ALL PREVIOUS STUDIES ARE INVALID NOW” every Tuesday.
  • paulddraper 3mo ago
    > Why do I need a system prompt at all? That's like asking why you need employee onboarding. You could have Stephen Freaking Hawking and he still wouldn't know what you specifically want. > Why do I need another black box AIs to review the code of the black box AI why can’t these things get code right the first time. That's like asking why you need code reviews. Thinking about it from an antagonistic perspective is useful. You can combine it into all the "same system" if helpful. > Why is the best “coding model” in the world still making up APIs that don’t exist and do seemingly random unreleased changes that it wasn’t prompted for. Because....it still isn't perfect? > Why do these models (supposedly) keep getting “better” (on benchmarks) but continue to degrade in output quality That is disconnected from reality.
• acedTrex 3mo ago
  "context engineering" as a term will never not make me laugh.
  • isoprophlex 3mo ago
    "Slop shuffling" seems the more honest terminology
• EternalFury 3mo ago
  Everyone is starting to express themselves as LLMs. I fell prey to having LLMs “copyedit” some of my words when I wanted to make sure I got my point across, but I am trying to stop that. Online in particular, I see a style which I call “probabilistic English”, infused with odd hip expressions. I thought it was only a problem for English communication, for which we have little care, but the same applies to code now. More and more code appears to be “probabilistic code”, homogenized to what is most common in the data LLMs were trained on.
• iknwnothing 3mo ago
  We know there is non-deterministic interpretation of claude.md, hence we are going to create non-deterministic deletion of rules in claude.md, but trust me bro, we will get this right this time.
• dfaoidsoi 3mo ago
  [dead]
• gbueno2024 3mo ago
  [flagged]
• hbarka 3mo ago
  > The new rules of context engineering for Claude 5 generation models We removed over 80% of Claude Code's system prompt for more advanced models. This is incredible. It’s like Claude learned one of the central ideas in the Tao Te Ching: you remove something every day. Wisdom and wholeness do not necessarily come from accumulating more. They can come from removing.
• moralestapia 3mo ago
  Everyone with a brain knew or at least had some intuition of it.
• BVHauge 3mo ago
  [flagged]
• cadamsdotcom 3mo ago
  Great to see "less is more". For example. Examples - good riddance. The model already knows what your examples are telling it to do, unless your work is really a long way out of distribution. Kind of wonder if they were ever needed.
• yunwal 3mo ago
  Claude doctor doesn't seem to do what they say it does. It just shows me a status check. I'm on the latest version of claude code