7 ms·

Hey all, Boris from the Claude Code team here. We've been investigating these reports, and a few of the top issues we've found are: 1. Prompt cache misses whe

by bcherny 6mo ago

Hey all, Boris from the Claude Code team here. We've been investigating these reports, and a few of the top issues we've found are: 1. Prompt cache misses when using 1M token context window are expensive. Since Claude Code uses a 1 hour prompt cache window for the main agent, if you leave your computer for over an hour then continue a stale session, it's often a full cache miss. To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session), and are investigating defaulting to 400k context instead, with an option to configure your context window to up to 1M if preferred. To experiment with this now, try: CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000 claude. 2. People pulling in a large number of skills, or running many agents or background automations, which sometimes happens when using a large number of plugins. This was the case for a surprisingly large number of users, and we are actively working on (a) improving the UX to make these cases more visible to users and (b) more intelligently truncating, pruning, and scheduling non-main tasks to avoid surprise token usage. In the process, we ruled out a large number of hypotheses: adaptive thinking, other kinds of harness regressions, model and inference regressions. We are continuing to investigate and prioritize this. The most actionable thing for people running into this is to run /feedback, and optionally post the feedback ids either here or in the Github issue. That makes it possible for us to debug specific reports.

denysvitali 6mo ago
OpenAI (Codex) keeps on resetting the usage limits each time they fuck up... I have yet to see Anthropic doing the same. Sorry but this whole thing seems to be quite on purpose.
- Madmallard 6mo ago
  Money money money money
- losteric 6mo ago
  [flagged]
  - mvkel 6mo ago
    Why did it suddenly become an issue, despite prompt caching behavior being unchanged?
  - denysvitali 6mo ago
    Me and my colleagues faced, over the last ~1 month or so, the same issues. With a new version of Claude Code pretty much each day, constant changes to their usage rules (2x outside of peak hours, temporarily 2x for a few weeks, ...), hidden usage decisions (past 256k it looks like your usage consumes your limits faster) and model degradation (Opus 4.6 is now worse than Opus 4.5 as many reported), I kind of miss how it can be an user error. The only user error I see here is still trusting Anthropic to be on the good side tbh. If you need to hear it from someone else: https://www.youtube.com/watch?v=stZr6U_7S90 https://www.youtube.com/watch?v=stZr6U_7S90
    - bcherny 6mo ago
      > past 256k it looks like your usage consumes your limits faster This is false. My guess is what is happening is #1 above, where restarting a stale session causes a 256k cache miss. That said, I hear the frustration. We are actively working on improving rate limit predictability and visibility into token usage.
      - tetraodonpuffer 6mo ago
        just like everybody else I and my colleagues at work have seen major regressions in terms of available usage over the past month, seemingly unrelated to caching/resuming. On an enterprise sub doing the same work I personally went from being able to have several sessions running concurrently without hitting limits, to only having one session at a time and hitting my 5h every day twice a day in 3-4 hours tops (and due to the apparent lower intelligence I have been at the terminal watching what opus is doing like a hawk, so it's not a I went for coffee I have to hit the cache). The first day I ever hit my 5h this year was the day everybody reported it (I think it was the Monday you introduced the 2x promotion after hours? not sure, like 3 weeks ago?) To avoid 1M issues, this week I have also intentionally used the 256k context model, disabled adaptive thinking and did the same "plans in multiple short steps with /clear in-between" to minimize context usage, and yet nothing helps. It just feels ~2x to ~3x less tokens than before, and a lot less smart than in February. Nowadays every time I complete a plan I spend several sessions afterwards saying things like "we have done plan X, the changes are uncommitted, can you take a look at what we did" and every time it finds things that were missed or outright (bad) shortcuts/deviations from plan despite my settings.json having a clear "if in doubt ask the user, don't just take the easy way out". As a random data point, just today opus halfway through a session told me to make a change to code inside a pod then rollout restart it to use said change, and when called out on it it of course said that I was right and of course that wouldn't work... It is understandable that given your incredible growth you are between a rock and a hard place and have to tweak limits, compute does not grow on trees, but the consistent "you are holding it wrong" messaging is not helpful. I am wondering if realistically your only option is to move everybody to metered, with clear token usage displayed, and maybe have pro/max 5/max 20 just be a "your first $x of tokens is 50/75% off". Allow folks to tweak the thinking budget, and change the system prompt to remove things like "try the easy solution first" which anecdotally has been introduced in the past while, and allow users to verify on prompt if the prompt would cause the whole context to be sent or if cache is available.
- weird-eye-issue 6mo ago
  Can you clearly state what they messed up?
  - nodja 6mo ago
    Not parent but I can guess from watching mostly from the sidelines. They introduced a 1M context model semi-transparently without realizing the effects it would have, then refused to "make it right' to the customer which is a trait most people expect from a business when they spend money on it, specially in the US, and specially when the money spent is often in the thousands of dollars. Unless anthropic has some secret sauce, I refuse to believe that their models perform anywhere near the same on >300k context sizes than they do on 100k. People don't realize but even a small drop in success rate becomes very noticeable if you're used to have near 100%, i.e. 99% -> 95% is more noticeable than 55% -> 50%. I got my first claude sub last month (it expires in 4 days) and I've used it on some bigish projects with opencode, it went from compacting after 5-10 questions to just expanding the context window, I personally notice it deteriorating somewhere between 200-300k tokens and I either just fork a previous context or start a new one after that because at that size even compacting seems to generate subpar summaries. It currently no longer works with opencode so I can't attest to how it well it worked the past week or so. If the 1M model introduction is at fault for this mass user perception that the models are getting worse, then it's anthropics fault for introducing confusion into the ecosystem. Even if there was zero problems introduced and the 1M model was perfect, if your response when the users complain is to blame it on the user, then don't expect the user will be happy. Nobody wants to hear "you're holding it wrong", but it seems that anthropic is trying to be apple of LLMs in all the wrong ways as well.
    - logicchains 6mo ago
      Especially since Codex faced the same issue but the team decided to explicitly default to only ~200k context to avoid surprises and degradation for users.
    - atonse 6mo ago
      I still love Claude and nothing but a ton of respect for Boris and the team building such a phenomenal product. That said, I feel that things started to feel a bit off usage-wise after the introduction of 1M context. I'd personally be happy to disable it and go back to auto-compacting because that seems to have been the happy medium.
deleted 6mo ago
[deleted]
fps-hero 6mo ago
Am I so out of touch? No! It’s the children who are wrong!
- rimliu 6mo ago
  you are prompting it wrong
mvkel 6mo ago
Why did this become an issue seemingly overnight when 1M context has been available for a while, and I assume prompt caching behavior hasn't changed? EDIT: prompt caching behavior -did- change! 1hr -> 5min on March 6th. I'm not sure how starting a fresh session fixes it, as it's just rebuilding everything. Why even make this available? It feels like the rules changed and the attitude from Anth is "aw I'm sorry you didn't know that you're supposed to do that." The whole point of CC is to let it run unattended; why would you build around the behavior of watching it like a hawk to prevent the cache from expiring?
- bcherny 6mo ago
  > 1hr -> 5min on March 6th This is not accurate. The main agent typically uses a 1h cache (except for API customers, which can enable 1h but it is not on by default because it costs more). Sub-agents typically use a 5m cache.
  - throwdbaaway 6mo ago
    https://github.com/anthropics/claude-code/issues/46829#issuecomment-4231266649 https://github.com/anthropics/claude-code/issues/46829#issue... - Have you checked with your colleague? (and his AI, of course)
    - fluidcruft 6mo ago
      Doesn't what's said at the link approximately agree? The 5m bug was said to be isolated to use of overage (API billing).
  - aaronblohowiak 6mo ago
    So if I run a test suite or compile my rust program in a sub agent I’m going to get cache misses? Boo.
    - skeledrew 6mo ago
      Sub agents don't have much context and don't stay around for long, so misses in that case are trivial.
rawicki 6mo ago
For me definitely the worst regression was the system prompt telling claude to analyze file to check if it's malware at every read. That correlates with me seeing also early exhausted quotas and acknowledgments of "not a malware" at almost every step. It is a horrible error of judgement to insert a complex request for such a basic ability. It is also an error of judgement to make claude make decisions whether it wants to improve the code or not at all. It is so bad, that i stopped working on my current project and went to try other models. So far qwen is quite promising.
- bcherny 6mo ago
  I don't think that's accurate. The malware prompt has been around since Sonnet 3.7. We carefully evaled it for each new model release and found no regression to intelligence, alongside improved scores for cyber risk. That said, we have removed the prompt for Opus 4.6 since it no longer needed it.
  - rawicki 6mo ago
    I started seeing "not a malware, continuing" in almost every reply since around 2 weeks ago. Maybe you just reintroduced it with some regression? Opus 4.6
    - bcherny 6mo ago
      That's weird. Would you mind running /feedback and sharing the id here next time you see this? I'd love to debug
      - obrajesse 6mo ago
        I’ve seen this a couple of times recently. Including right after compact. I’ll /feedback it next time I see it
      - rawicki 6mo ago
        Sure, I really appreciate you looking at this. a6edd0d1-a9ed-4545-b237-cff00f5be090 / https://github.com/anthropics/claude-code/issues/47027 https://github.com/anthropics/claude-code/issues/47027 I'm happy to provide any other info that can be useful (as long as i'm not sharing any information about the code or tools we use into a public github issue).
throwaway2027 6mo ago
I don't want a nudge. I want a clear RED WARNING with "You've gone away from your computer a bit too long and chatted too much at the coffee machine. You're better off starting a new context!"
- bcherny 6mo ago
  Ack, it is currently blue but we can make it red
  - oezi 6mo ago
    I think after the TTL expires the session should be autocompacted and the user should given a choice to continue with compacted version or be hit with the full read cost of continuing with their large but expired context. At the moment users are blind what is going on.
- SpaceNoodled 6mo ago
  Why is nobody even asking why that should be an issue? No other text editor shits the bed that way. The whole point of the computer is that it patiently waits for my input.
  - GeoAtreides 6mo ago
    let me put this way: not your ram, not your cache, not waiting patiently for your input.
    - SpaceNoodled 6mo ago
      Good thing they're not charging for it, then.
    - subscribed 6mo ago
      Good thing they didn't silently, quietly change cache from 1 hour to 5 minutes, right?
- senko 6mo ago
brokencode 6mo ago
Would it be possible to increase the cache duration if misses are a frequent source of problems? Maybe using a heartbeat to detect live sessions to cache longer than sessions the user has already closed. And only do it for long sessions where a cache miss would be very expensive.
- bcherny 6mo ago
  Yes, we're trying a couple of experiments along these lines. Good intuition.
yummytummy 6mo ago
Ah, so cache usage impacts rate limits. There goes the ”other harnesses aren’t utilizing the cache as efficiently” argument.
- bcherny 6mo ago
  Claude Code is the most prompt cache-efficient harness, I think. The issue is more that the larger the context window, the higher the cost of a cache miss.
  - yummytummy 6mo ago
    That might be, but the argument was that poor cache utilization was costing Anthropic too much money in other harnesses. If cache is considered in rate limits, it doesn’t matter from a cost perspective, you’ll just hit your rate limits faster in other harnesses that don’t try to cache optimize.
    - bcherny 6mo ago
      There were two issues with some other 3p harnesses: 1. Poor cache utilization. I put up a few PRs to fix these in OpenClaw, but the problem is their users update to new versions very slowly, so the vast majority of requests continued to use cache inefficiently. 2. Spiky traffic. A number of these harnesses use un-jittered cron, straining services due to weird traffic shape. Same problem -- it's patched, but users upgrade slowly. We tried to fix these, but in the end, it's not something we can directly influence on users' behalf, and there will likely be more similar issues in the future. If people want to use these they are welcome to, but subscriptions clients need to be more efficient than that.
      - SyneRyder 6mo ago
        How much jitter would you prefer, how many seconds / minutes out? I have some morning tasks that run while I'm asleep via claude -p, and it sounds like I'm slightly contributing to your spikes (presumably hourly and on quarter hours).
j45 6mo ago
Pulling all the skills and agents in the world in, when unused are a big hit. I deleted all of mine and added back as needed and there was an improvement. Running Claude Cowork in the background will hit tokens and it might not be the most efficient use of token use. Last, but not least, turning off 1M token context by default is helpful.
docheinestages 6mo ago
Why are you all of a sudden running into so many issues like this? Could it be that all of the Anthropics employees have completely unlimited and unbounded accounts, which means you don't get a feeling of how changes will affect the customers?
- bcherny 6mo ago
  The number of people using Claude Code has grown very quickly, which means: - More configurations and environments we need to test - Given an edge/corner case, it is more likely a significant number of users run into it - As the ecosystem has grown, more people use skills and plugins, and we need to offer better tools and automation to ensure these are efficient We do actually dogfood rate limits, so I think it's some combination of the above.
  - gozucito 6mo ago
    I think the suspicion regarding skills and plugins is fair and logical. And it is absolutely the case that some use significantly more tokens. with that said, on my 5x plan, I could have multiple sessions working and the limit was far away. Around when you introduced the whole more tokens during off-peak hours and fewer tokens during working US hours, Even with a single session, using no plugins at all (I uninstalled OMC) I run into limits very often. I have not performed any rigorous tests but it feels like I have about 25% of what I used to have or less. This is all without using teams of agents, or ralph loops or anything like that. Just /plan and execute in a single session. I have restored the /clear context before executing plan to try and mitigate things. I will also try the 400k context since, in my experience, the 1M tokens have not made Opus 4.6 noticeably smarter for my small webapp use-case. Best of luck to you! ps: whenever you introduce a change, please make it optional AND ask the user about it at first. Don't just yank things suddenly (like the /clear context and apply plan option.) as I spent hours trying to figure out how I broke it before I saw your note and how to re-enable it.
  - sharts 6mo ago
    How do ya’ll test?
  - 6mo ago
avree 6mo ago
Hey Boris - why is the best way to get support making a Hacker News or X post, and hoping you reply? Why does Anthropic Enterprise Support never respond to inquiries?
- egamirorrim 6mo ago
  I mean if we're building an unrelated wishlist... Can 20x max users get auto mode already? Or can the enterprise plans get something equivalent to 20x max? Given I'm running two max accounts to get the usage I want, can we get a 25x and 40x tier? :-)
  - oefrha 6mo ago
    It’s called /extra-usage and they really want you to use it.
_fizz_buzz_ 6mo ago
I have a feature request: I build an mcp server, but now it has over 60 tools. Most sessions i really don’t need most of them. I suppose I could make this into several servers. But it would maybe be nice to give the user more power here. Like let me choose the tools that should be loaded or let me build servers that group tools together which can be loaded. Not sure if that makes sense …
hughw 6mo ago
Where can i learn about concepts like prompt cache misses? I don't have a mental model how that interacts with my context of 1M or 400k tokens... I can cargo cult follow instructions of course but help us understand if you can so we can intelligently adapt our behavior. Thanks.
- hughw 6mo ago
  And why does /clear help things? Doesn't that wipe out the history of that session? Jeez.
  - CWwdcdk7h 6mo ago
    [dead]
- bcherny 6mo ago
  The docs are a good place to start: https://platform.claude.com/docs/en/build-with-claude/prompt-caching#how-prompt-caching-works https://platform.claude.com/docs/en/build-with-claude/prompt...
  - snthpy 6mo ago
    Thanks. Just noting that those docs say the cache duration is 5 min and not 1 hour as stated in sibling comment: > By default, the cache has a 5-minute lifetime. The cache is refreshed for no additional cost each time the cached content is used. > > If you find that 5 minutes is too short, Anthropic also offers a 1-hour cache duration at additional cost.
    - yoaviram 6mo ago
      Apparently Anthropic downgraded cache TTL to 5 min without telling anyone. My biggest issue with the recent issues with Claude Code is the lack transparency, although it looks like even Boris doesn't know about one: https://news.ycombinator.com/item?id=47736476 https://news.ycombinator.com/item?id=47736476
reenorap 6mo ago
Boris, you're seeing a ton of anecdotes here and Claude has done something that has affected a bunch of their most fervent users. Jeff Bezos famously said that if the anecdotes are contradicting the metrics, then the metrics are measuring the wrong things. I suggest you take the anecdotes here seriously and figure out where/why the metrics are wrong.
- bcherny 6mo ago
  We are taking it seriously, and are continuing to investigate. We are not trusting the metrics.
  - Ucalegon 6mo ago
    [flagged]
    - amirhirsch 6mo ago
      Dude is on hacker news on a Sunday. half the GDP of the world is competing with him. What metrics would you like to see?
      - Ucalegon 6mo ago
        An enforceable SLA with the services that Anthropic offers rather than putting an employee to respond to things on Sunday.
        roamerz 6mo ago
        >> rather than putting an employee to respond to things on Sunday. Maybe just maybe they didn’t put him here, rather he just a normal guy who reads HN, who is passionate about his role, and is here on his own time.
        Ucalegon 6mo ago
        Maybe... maybe... maybe... none of this builds trust when there is something that does build trust; putting revenue on the line and opening yourself to legal liability. Otherwise everything is empty and meaningless, its just PR, and nothing more.
samuelknight 6mo ago
Have you considered poking the cache? When a user walks away during the business day but CC is sitting open, you can refresh that cache up to 10x before it costs the same as a full miss. Realistically it would be <8x in a working day.
dkersten 6mo ago
Eh you say that every time and yet it keeps happening.
999900000999 6mo ago
You've created quite a conundrum. The only people who are going to run into issues are superpower users who are running this excessively beyond any reasonable measure. Most people are going to be quite happy with your service. But at the same time, and this is just a human nature thing people are 10 times more likely we complain about an issue than to compliment something working well. I don't know how to fix this, but I strongly suspect this isn't really a technical issue. It's more of a customer support one.
ramon156 6mo ago
Boris, wasnt this the same thing ~2 weeks ago? Is it the same cache misses as before? What's the expected time till solved? Seems like its taking a while
fluidcruft 6mo ago
How can we turn of 1m context? I don't find it has ever helped.
- mwigdahl 6mo ago
  He mentioned this in his original comment: "CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000"
  - fluidcruft 6mo ago
    There's also CLAUDE_CODE_DISABLE_1M_CONTEXT and I'm really not clear on what the difference is and why to pick one over the other. But I guess one disables models that have 1m and the other keeps those models but sets the limit lower?
earino 6mo ago
Hello Boris! How do I increase the 1 hour prompt cache window for the main agent? I would love to be able to set that to, say, 4 hours. That gives me enough time to work on something, go teach a class, grab a snack, and come back and pick up where I left off.
- subscribed 6mo ago
  Another CC team member confirmed it's 5 minutes now, not 1 hour. See the links in https://news.ycombinator.com/item?id=47747209 https://news.ycombinator.com/item?id=47747209
re-thc 6mo ago
> To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session) Is this really an improvement? Shouldn't this be something you investigate before introducing 1M context? What is a long stale session? If that's not how Claude Code is intended to be used it might as well auto quit after a period of time. If not then if it's an acceptable use case users shouldn't change their behavior. > People pulling in a large number of skills, or running many agents or background automations, which sometimes happens when using a large number of plugins. If this was an issue there should have been a cap on it before the future was released and only increased once you were sure it is fine? What is "a large number"? Then how do we know what to do? It feels like "AI" has improved speed but is in fact just cutting corners.
j-pb 6mo ago
The /clear nudge isn't a solution though. Compacting or clearing just means rebuilding context until Claude is actually productive again. The cost comes either way. I get that 1M context windows cost more than the flat per-token price reflects, because attention scales with context length, but the answer to that is honest pricing or not offering it. Not annoying UX nudges. What’s actually indefensible is that Claude is already pushing users to shrink context via, I presume, system prompt. At maybe 25% fill: “This seems like a good opportunity to wrap it up and continue in a fresh context window.” “Want to continue in a fresh context window? We got a lot of work done and this next step seems to deserve a fresh start!” If there’s a cost problem, fix the pricing or the architecture. But please stop the model and UI from badgering users into smaller context windows at every opportunity. That is not a solution, it’s service degradation dressed as a tooltip.
- g4cg54g54 6mo ago
  currently "clear makes it worse" https://github.com/anthropics/claude-code/issues/47098 https://github.com/anthropics/claude-code/issues/47098 + https://github.com/anthropics/claude-code/issues/47107 https://github.com/anthropics/claude-code/issues/47107 launching with `CLAUDE_CODE_DISABLE_GIT_INSTRUCTIONS=1 claude "Hello"` till those are fixed seems to be th way
- foota 6mo ago
  The cost issues they're seeing (at least from what they've stated) are from users, not internally. Basically, it takes either $5 or $6.25 (depending on 5m or 1h ttl) to re-ingest a 1M context length conversation into cache for opus 4.6, that's obviously a very high cost, and users are unhappy with it. I think 400k as a default seems about right from my experience, but just having the ability to control it would be nice. For the record, even just making a tool call at 1M tokens costs 50 cents (which could be amortized if multiple calls are made in a round), so imo costs are just too high at long context lengths for them to be the default.
varispeed 6mo ago
Can you explain why Opus 4.6 suddenly becomes dumb as a sack of potatoes, even if context is barely filled? Can you explain why Opus 4.6 will be coming up with stupid solutions only to arrive at a good one when you mention it is trying to defraud you? I have a feeling the model is playing dumb on purpose to make user spend more money. This wasn't the case weeks ago when it actually working decently.
accounting2026 6mo ago
[flagged]
MuffinFlavored 6mo ago
I wish people would pay more attention to: * Anthropic is in some way trying to run a business (not a charity) and at least (eventually?) make money and not subsidize usage forever * "What a steal/good deal" the $100-$200/mo plans are compared to if they had to pay for raw API usage and less on "how dare you reserve the right to tweak the generous usage patterns you open-ended-ly gave us, we are owed something!"
- logicchains 6mo ago
  All those apply to OpenAI+Codex too, but they're far more generous with limits than Anthropic, and with granting fresh limits to apologize when they fuck up.
- lbreakjai 6mo ago
  As an (ex) paying customer, I'm expecting some consistency. I used to be satisfied with the value I got, until the limits changed overnight, and I'd get a ten of my previous usage. If Anthropic is allowed to alter the deal whenever, then I'd expect to be able to get my money back, pro-rata, no questions asked.
- oskarw85 6mo ago
  [flagged]
- visarga 6mo ago
  yes, $200/mo is a serious subscription, we are owed something, and I won't feel ashamed for saying that especially when you are told using the subagent for code review "claude -p" is now billed on API on top of $200 sub
danmaz74 6mo ago
Could we get an option to use Opus with a smaller context window? I noticed that results get much worse way earlier than when you reach 1M tokens, and I would love to have a setting so that I could force a compaction at eg 300k tokens.
- SyneRyder 6mo ago
  You probably just missed it in his post, but: "To experiment with this now, try: CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000 claude." Maybe try changing the 4 to a 3 and see if that works for you?
  - danmaz74 6mo ago
    Thank you, will definitely try that!
- stitched2gethr 5mo ago
  Try this: `claude --model claude-opus-4-6`
EGreg 6mo ago
Boris, is the KV cache TTL now reduced to 5 minutes from 1 hour? I think this may be the biggest concern for people building tools on the API: https://github.com/anthropics/claude-code/issues/46829 https://github.com/anthropics/claude-code/issues/46829 I would argue that KV caching is a net gain for Ant and a well-maintained cache is the biggest thing that can generate induced demand and a thriving third party ecosystem. https://safebots.ai/papers/KV.pdf https://safebots.ai/papers/KV.pdf
KronisLV 6mo ago
> defaulting to 400k context instead, with an option to configure your context window to up to 1M if preferred This seems really useful! I'm surprised that "Opus 4.6" (200K) and "Opus 4.6 1M" are the only Opus options in the desktop app, whereas in the CLI/TUI app you don't seem to even get that distinction. I bet that for a lot of folks something like 400k, 600k or 800k would work as better defaults, based on whatever task they want to work on.
ahofmann 6mo ago
Resizing the context window seems like a very good idea to me. I noticed a decline of productivity when the 1M context window was released and I'd like to bring it back to 200k, because it was totally fine for the things I was working on.
deleted 6mo ago
[deleted]
jauntywundrkind 6mo ago
There's an issue someone raised showing that prompt caches are only 5 minutes. The reply seems to be: oh huh, interesting. Maybe that's a good thing since people sometimes one-shot? That doesn't feel like the messaging I want to be reading, and the way it conflicts with the message here that cache is 1 hour is confusing. https://news.ycombinator.com/item?id=47741755 https://news.ycombinator.com/item?id=47741755 Is there any status information or not on whether cache is used? It sure looks like the person analyzing the 5m issue had to work extremely hard to get any kind of data. It feels like the iteration loop of people getting better at this stuff would go much much better if this weren't such a black box, if we had the data to see & understand: is the cache helping?
- TheTaytay 6mo ago
  Aren’t they saying that it’s 5minutes for things like subagents (that wouldn’t benefit from it?)
yumraj 6mo ago
> Since Claude Code uses a 1 hour prompt cache window for the main agent, if you leave your computer for over an hour then continue a stale session, it's often a full cache miss. To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session), and are investigating defaulting to 400k context instead I don’t understand this. I frequently have long breaks. I never want to clear or even compact because I don’t want to lose the conversations that I’ve had and the context. Clearing etc causes other issues like I have to restate everything at times and it misses things. I do try to update the memory which helps. I wish there was a better solution than a time bound cache
- cowwoc2020 6mo ago
  Makes me wish that shortly before the server-side expiration, we could save the cache on the client-side, indefinitely. But my understanding is that we're talking about ~60GB of data per session, so it sounds unrealistic to do...
  - yumraj 6mo ago
    Where are you getting 60GB from? It shouldn’t be that large. But yes, would love to save context/cache such that it can be played back/referred to if needed. /compact is a little black box that I just have to trust that is keeping the important bits.
    - davmre 6mo ago
      The KV cache consists of activation vectors for every attention head at every layer of the model for every token, so it gets quite large. ChatGPT also estimates 60-100GB for full token context of an Opus-sized model: https://chatgpt.com/share/69dc5030-268c-83e8-92c2-6cef962dc5c2 https://chatgpt.com/share/69dc5030-268c-83e8-92c2-6cef962dc5...
      - visarga 6mo ago
        There are ways to quantize or compress KV cache down.
g4cg54g54 6mo ago
from looking at the raw requests, that cant seem right? its all "cache_control": { "type": "ephemeral" } there is no "ttl" anywhere. // edit: cc_version=2.1.104.f27
throwpoaster 6mo ago
Wait what? If I get told to come back in three hours because I'm using the product too much, I get penalized when I resume? What's the right way to work on a huge project then? I've just been saying "Please continue" -- that pops the quota?
apgwoz 6mo ago
As another data point, I pay for Pro for a personal account, and use no skills, do nothing fancy, use the default settings, and am out of tokens, with one terminal, after an hour. This is typically working on a < 5,000 line code base, sometimes in C, sometimes in Go. Not doing incredibly complicated things.
throwpoaster 6mo ago
Have you tried asking Mythos for a fix?
8note 6mo ago
> Since Claude Code uses a 1 hour prompt cache window for the main agent this seems a bit awkward vs the 5 hour session windows. if i get rate limited once, I'll get rate limited immediately again on the same chat when the rate limit ends? any chance we can get some form of deffered cache so anything on a rate limited account gets put aside until the rate limit ends?
mmd45 6mo ago
shouldn't compaction be interactive with the user as to what context will continue to be the most relevant in the future??? what if the harness allowed for a turn to clarify the user's expected future direction of the conversation and did the consolidation based upon the addition info? there definitely seems to be a benefit to pruning the context and keeping the signal to noise high wrt what is still to be discussed.
cowwoc2020 6mo ago
Boris, Even if Anthropic is working in good faith to lower infrastructure costs, developers need more than 5 minutes to notice that CC completed a task, review its changes and ask it to merge. Only developers who do not review code changes can live with such a TTL... Consider making this value configurable as the ideal TTL value is different for each person. If people are willing to pay more for 30 minutes TTL than 5 minutes, they should be able to.
- deleted 6mo ago
  [deleted]
jiwidi 6mo ago
Hi Boris, Long term claude code user here. Is the first time i've had to setup a hook to codex to review claude output. Is hallucinating like never before Is missing key concepts/instructions in context like never before Is writing bad code that will "pass test" much more. Before it use to try be critic and do good code, now it will try to hack test and bypass intructions for a green pass.
cmaster11 6mo ago
Thank you for your responses, especially on a Sunday. They give us some insights and at least a couple temporary workarounds to use, while the issues are being addressed :) much appreciated
sunir 6mo ago
I suspect 1M token context is questionable value because of the secondary effect of burning quota vs getting work done. I think the model select that let me choose 1M made sense because I could decide if I was working on large documents and compacting more often was more effective.
anoazian 6mo ago
I’ve seen the /clear command prompt and I found the verbiage to be a bit unclear. I think clarifying that the cache has expired and providing an understandable metric on the impact - ie “X% of your 5-hour window” for Pro/Mad users and details on token use for API users. A pop-up that requires explicit acknowledgment might also help, although that could be more of an annoyance to enterprise users. One pattern I use frequently is using one high level design and implementation agent that I’ll use for multiple sessions and delegate implementation to lower level agents. In this case it’d be helpful to have one of two options: 1. If Claude CLI could create an auto compaction of the conversation history before cache expiration. For example, if I’m beyond X minutes or Y prompts in a conversation and I’ve been inactive for a threshold it could auto-compact close to the expiration and provide that as an option on resume. 2. If I could configure cache expiration proactively and Anthropic could use S3 or a similar slow load mechanism to offload the cache for a longer period - possibly 24-72h. I can appreciate that longer KV cache expiration would complicate capacity management and make inference traffic less fungible but I wouldn’t mind waiting seconds to minutes for it to load from a slower store to resume without quota hits.
taspeotis 6mo ago
Hi, thanks for Claude Code. I was wondering though if you'd considering adding a mode to make text green and characters come down from the top of the screen individually, like in The Matrix?
- visarga 6mo ago
  No, I want a little monkey doing tricks. /s
  - taspeotis 6mo ago
    This guy? https://en.wikipedia.org/wiki/BonziBuddy https://en.wikipedia.org/wiki/BonziBuddy
tigershark 6mo ago
Claude Code cache is not 1 hour. There is a "Closed as not planned" issue in GitHub that confirms that it has been moved to 5 minutes since March: https://github.com/anthropics/claude-code/issues/46829 https://github.com/anthropics/claude-code/issues/46829. I started seeing the massive degradation exactly on the 23rd of March, hence after a few days I unsubscribed because it was completely unusable, with a ~5h session being depleted in as little as 15-20 mins.
- subscribed 6mo ago
  Looks like the cache change to 5 minutes was so secretive that even CC team doesn't know that. Or someone just vibe coded "Hey, Claude, make them burn allowances quicker" and merged without telling anyone. Both are plausible to me.
smrtinsert 6mo ago
Number 2 makes me chuckle honestly. Too many people going down the 10x rabbit holes on youtube. Next up, a framework that 100xs your workflow. You know its good because it comes with 300 agents and 20 mcp servers and 1200 skills
foofloobar 6mo ago
[flagged]
- stingraycharles 6mo ago
  This comment seems unnecessarily hostile.
  - foofloobar 6mo ago
    They appear to take issues seriously mostly when they become posts on hacker news and when articles are published online by major news sites. Customer support is mostly a bot. I don't even know how to reach some actual humans to get support. I'm sorry if you and others are offended. They've had these issues for several weeks now. I haven't seen any real improvements during this time. I see more features and more bugs. There have been several releases made over the last few days without any changelogs. The quotas are still as opaque as they've been. This company has some extremely shady business practices.
  - prmph 6mo ago
    Why? It seems just fine to me. This is what Anthropic needs to do if they want to survive. I'm always looking out for someone to integrate an actually good harness to a good model. Once that happens, I'm jumping ship if Anthropic keeps playing these tricks. It's almost unusable for me now. A simple prompt to merge 3 sub-100-line files with simple node code, on Sonnet 4.6, uses up 20% of my 5 hour quota, on a new/fresh session.
    - foofloobar 6mo ago
      To be fair, my comment was a bit harsher before the update. The way they handle the development, communication and how they treat customers isn't fine. I've seen some angry people post and comment in manners which truly deserved the label hostile. The whole product with the infrastructure and Claude Code's code appear to be vibe coded.
      - 6mo ago
pu_pe 6mo ago
So Anthropic is trying to save money on infrastructure, we all get it. However, it's not ok to degrade the performance your users have paid for. Last week the issue was that you reduced the default "effort" level, now the prompt cache is shortened. Several users experience far more restrictive usage limits lately. There is only so much you can do through "UX improvements" or some smart routing on the backend. Your flagship product is actively getting worse, and if users need to fiddle with hidden settings and keep track of GitHub issues every week they will start voting with their money.
- baq 6mo ago
  Dear sir, please think of the shareholders, they need a fair exit.
- steveBK123 6mo ago
  Classic VC pump playbook - run it uneconomically until everyone is addicted, then 5x prices once you have enough critical mass. See 2010s "Millennial Lifestyle Subsidy"..
  - FuckButtons 6mo ago
    It seems pretty transparent that they are heavily resource constrained, (training run for Claude 5.x, higher usage / growth than anticipated). I don’t disagree that their long play is monopolistic pricing, but what we’re observing seems better explained by the fact they have a very tight compute budget they are trying to optimize over to put as much as they can into next gen experiments / training to make sure they stay competitive over the next 6-months / year.
  - lukewarm707 6mo ago
    you know once, anthropic was supposed to be a public benefit org!
- zachrip 6mo ago
  Where did they say the prompt cache is shortened?
richardjennings 6mo ago
/loop message ping every 4 minutes keeps the cache warm while the CC REPL is not active.
miroljub 6mo ago
People, just switch to MiniMax and ditch CC completely. It's not worth it any more.
je42 6mo ago
Does this 60min ttl of also apply to claude code web? I have regularly sessions open for multiple days. Is that a pattern that is not advised?
FabDee 6mo ago
One thing I didn't see anywhere here, except your mention about pulling in large number of skills, is that the token consumption is significantly higher for users with many agents, skills, and MCPs installed, and many are mere ghosts. The 5m TTL from #46829 compounds the effect: in my case, I found ~20k tokens of ghost context I hadn't intentionally opened. Each idle period after 5m wastes that as a full cache miss. Boris, would you please confirm on-record: is the current cache TTL for the main agent context 1h or 5m? Issue #46829 was closed as "not planned".
guybedo 6mo ago
you should check with people working on Claude Code, cache has been udpated to 5min ... https://github.com/anthropics/claude-code/issues/46829#issuecomment-4231266649 https://github.com/anthropics/claude-code/issues/46829#issue... So yeah, 1M window that expires every 5min .... not good
mercnz 6mo ago
it seems if context can't be held for over an hour it should warn you a countdown or such; i already enabled the tokens verbosity thing to see what token level i'm at, but i often leave things sitting rather than complete so that i'm tying things up to start something new in the morning rather than starting on a new thing. so like i just resumed a session that was near-complete, and now it's gone and reloaded all that session in? bit i hadn't detached it. i kind of thougth /summary itself had to read the whole token flow, but that the token context was held locally for some reason..

by bcherny 6mo ago

denysvitali 6mo ago

OpenAI (Codex) keeps on resetting the usage limits each time they fuck up... I have yet to see Anthropic doing the same. Sorry but this whole thing seems to be quite on purpose.

Madmallard 6mo ago
Money money money money
losteric 6mo ago
[flagged]
- mvkel 6mo ago
  Why did it suddenly become an issue, despite prompt caching behavior being unchanged?
- denysvitali 6mo ago
  Me and my colleagues faced, over the last ~1 month or so, the same issues. With a new version of Claude Code pretty much each day, constant changes to their usage rules (2x outside of peak hours, temporarily 2x for a few weeks, ...), hidden usage decisions (past 256k it looks like your usage consumes your limits faster) and model degradation (Opus 4.6 is now worse than Opus 4.5 as many reported), I kind of miss how it can be an user error. The only user error I see here is still trusting Anthropic to be on the good side tbh. If you need to hear it from someone else: https://www.youtube.com/watch?v=stZr6U_7S90 https://www.youtube.com/watch?v=stZr6U_7S90
  - bcherny 6mo ago
    > past 256k it looks like your usage consumes your limits faster This is false. My guess is what is happening is #1 above, where restarting a stale session causes a 256k cache miss. That said, I hear the frustration. We are actively working on improving rate limit predictability and visibility into token usage.
    - tetraodonpuffer 6mo ago
      just like everybody else I and my colleagues at work have seen major regressions in terms of available usage over the past month, seemingly unrelated to caching/resuming. On an enterprise sub doing the same work I personally went from being able to have several sessions running concurrently without hitting limits, to only having one session at a time and hitting my 5h every day twice a day in 3-4 hours tops (and due to the apparent lower intelligence I have been at the terminal watching what opus is doing like a hawk, so it's not a I went for coffee I have to hit the cache). The first day I ever hit my 5h this year was the day everybody reported it (I think it was the Monday you introduced the 2x promotion after hours? not sure, like 3 weeks ago?) To avoid 1M issues, this week I have also intentionally used the 256k context model, disabled adaptive thinking and did the same "plans in multiple short steps with /clear in-between" to minimize context usage, and yet nothing helps. It just feels ~2x to ~3x less tokens than before, and a lot less smart than in February. Nowadays every time I complete a plan I spend several sessions afterwards saying things like "we have done plan X, the changes are uncommitted, can you take a look at what we did" and every time it finds things that were missed or outright (bad) shortcuts/deviations from plan despite my settings.json having a clear "if in doubt ask the user, don't just take the easy way out". As a random data point, just today opus halfway through a session told me to make a change to code inside a pod then rollout restart it to use said change, and when called out on it it of course said that I was right and of course that wouldn't work... It is understandable that given your incredible growth you are between a rock and a hard place and have to tweak limits, compute does not grow on trees, but the consistent "you are holding it wrong" messaging is not helpful. I am wondering if realistically your only option is to move everybody to metered, with clear token usage displayed, and maybe have pro/max 5/max 20 just be a "your first $x of tokens is 50/75% off". Allow folks to tweak the thinking budget, and change the system prompt to remove things like "try the easy solution first" which anecdotally has been introduced in the past while, and allow users to verify on prompt if the prompt would cause the whole context to be sent or if cache is available.
weird-eye-issue 6mo ago
Can you clearly state what they messed up?
- nodja 6mo ago
  Not parent but I can guess from watching mostly from the sidelines. They introduced a 1M context model semi-transparently without realizing the effects it would have, then refused to "make it right' to the customer which is a trait most people expect from a business when they spend money on it, specially in the US, and specially when the money spent is often in the thousands of dollars. Unless anthropic has some secret sauce, I refuse to believe that their models perform anywhere near the same on >300k context sizes than they do on 100k. People don't realize but even a small drop in success rate becomes very noticeable if you're used to have near 100%, i.e. 99% -> 95% is more noticeable than 55% -> 50%. I got my first claude sub last month (it expires in 4 days) and I've used it on some bigish projects with opencode, it went from compacting after 5-10 questions to just expanding the context window, I personally notice it deteriorating somewhere between 200-300k tokens and I either just fork a previous context or start a new one after that because at that size even compacting seems to generate subpar summaries. It currently no longer works with opencode so I can't attest to how it well it worked the past week or so. If the 1M model introduction is at fault for this mass user perception that the models are getting worse, then it's anthropics fault for introducing confusion into the ecosystem. Even if there was zero problems introduced and the 1M model was perfect, if your response when the users complain is to blame it on the user, then don't expect the user will be happy. Nobody wants to hear "you're holding it wrong", but it seems that anthropic is trying to be apple of LLMs in all the wrong ways as well.
  - logicchains 6mo ago
    Especially since Codex faced the same issue but the team decided to explicitly default to only ~200k context to avoid surprises and degradation for users.
  - atonse 6mo ago
    I still love Claude and nothing but a ton of respect for Boris and the team building such a phenomenal product. That said, I feel that things started to feel a bit off usage-wise after the introduction of 1M context. I'd personally be happy to disable it and go back to auto-compacting because that seems to have been the happy medium.

deleted 6mo ago

[deleted]

fps-hero 6mo ago

Am I so out of touch? No! It’s the children who are wrong!

rimliu 6mo ago
you are prompting it wrong

mvkel 6mo ago

Why did this become an issue seemingly overnight when 1M context has been available for a while, and I assume prompt caching behavior hasn't changed? EDIT: prompt caching behavior -did- change! 1hr -> 5min on March 6th. I'm not sure how starting a fresh session fixes it, as it's just rebuilding everything. Why even make this available? It feels like the rules changed and the attitude from Anth is "aw I'm sorry you didn't know that you're supposed to do that." The whole point of CC is to let it run unattended; why would you build around the behavior of watching it like a hawk to prevent the cache from expiring?

bcherny 6mo ago
> 1hr -> 5min on March 6th This is not accurate. The main agent typically uses a 1h cache (except for API customers, which can enable 1h but it is not on by default because it costs more). Sub-agents typically use a 5m cache.
- throwdbaaway 6mo ago
  https://github.com/anthropics/claude-code/issues/46829#issuecomment-4231266649 https://github.com/anthropics/claude-code/issues/46829#issue... - Have you checked with your colleague? (and his AI, of course)
  - fluidcruft 6mo ago
    Doesn't what's said at the link approximately agree? The 5m bug was said to be isolated to use of overage (API billing).
- aaronblohowiak 6mo ago
  So if I run a test suite or compile my rust program in a sub agent I’m going to get cache misses? Boo.
  - skeledrew 6mo ago
    Sub agents don't have much context and don't stay around for long, so misses in that case are trivial.

rawicki 6mo ago

For me definitely the worst regression was the system prompt telling claude to analyze file to check if it's malware at every read. That correlates with me seeing also early exhausted quotas and acknowledgments of "not a malware" at almost every step. It is a horrible error of judgement to insert a complex request for such a basic ability. It is also an error of judgement to make claude make decisions whether it wants to improve the code or not at all. It is so bad, that i stopped working on my current project and went to try other models. So far qwen is quite promising.

bcherny 6mo ago
I don't think that's accurate. The malware prompt has been around since Sonnet 3.7. We carefully evaled it for each new model release and found no regression to intelligence, alongside improved scores for cyber risk. That said, we have removed the prompt for Opus 4.6 since it no longer needed it.
- rawicki 6mo ago
  I started seeing "not a malware, continuing" in almost every reply since around 2 weeks ago. Maybe you just reintroduced it with some regression? Opus 4.6
  - bcherny 6mo ago
    That's weird. Would you mind running /feedback and sharing the id here next time you see this? I'd love to debug
    - obrajesse 6mo ago
      I’ve seen this a couple of times recently. Including right after compact. I’ll /feedback it next time I see it
    - rawicki 6mo ago
      Sure, I really appreciate you looking at this. a6edd0d1-a9ed-4545-b237-cff00f5be090 / https://github.com/anthropics/claude-code/issues/47027 https://github.com/anthropics/claude-code/issues/47027 I'm happy to provide any other info that can be useful (as long as i'm not sharing any information about the code or tools we use into a public github issue).

throwaway2027 6mo ago

I don't want a nudge. I want a clear RED WARNING with "You've gone away from your computer a bit too long and chatted too much at the coffee machine. You're better off starting a new context!"

bcherny 6mo ago
Ack, it is currently blue but we can make it red
- oezi 6mo ago
  I think after the TTL expires the session should be autocompacted and the user should given a choice to continue with compacted version or be hit with the full read cost of continuing with their large but expired context. At the moment users are blind what is going on.
SpaceNoodled 6mo ago
Why is nobody even asking why that should be an issue? No other text editor shits the bed that way. The whole point of the computer is that it patiently waits for my input.
- GeoAtreides 6mo ago
  let me put this way: not your ram, not your cache, not waiting patiently for your input.
  - SpaceNoodled 6mo ago
    Good thing they're not charging for it, then.
  - subscribed 6mo ago
    Good thing they didn't silently, quietly change cache from 1 hour to 5 minutes, right?
senko 6mo ago

brokencode 6mo ago

Would it be possible to increase the cache duration if misses are a frequent source of problems? Maybe using a heartbeat to detect live sessions to cache longer than sessions the user has already closed. And only do it for long sessions where a cache miss would be very expensive.

bcherny 6mo ago
Yes, we're trying a couple of experiments along these lines. Good intuition.

yummytummy 6mo ago

Ah, so cache usage impacts rate limits. There goes the ”other harnesses aren’t utilizing the cache as efficiently” argument.

bcherny 6mo ago
Claude Code is the most prompt cache-efficient harness, I think. The issue is more that the larger the context window, the higher the cost of a cache miss.
- yummytummy 6mo ago
  That might be, but the argument was that poor cache utilization was costing Anthropic too much money in other harnesses. If cache is considered in rate limits, it doesn’t matter from a cost perspective, you’ll just hit your rate limits faster in other harnesses that don’t try to cache optimize.
  - bcherny 6mo ago
    There were two issues with some other 3p harnesses: 1. Poor cache utilization. I put up a few PRs to fix these in OpenClaw, but the problem is their users update to new versions very slowly, so the vast majority of requests continued to use cache inefficiently. 2. Spiky traffic. A number of these harnesses use un-jittered cron, straining services due to weird traffic shape. Same problem -- it's patched, but users upgrade slowly. We tried to fix these, but in the end, it's not something we can directly influence on users' behalf, and there will likely be more similar issues in the future. If people want to use these they are welcome to, but subscriptions clients need to be more efficient than that.
    - SyneRyder 6mo ago
      How much jitter would you prefer, how many seconds / minutes out? I have some morning tasks that run while I'm asleep via claude -p, and it sounds like I'm slightly contributing to your spikes (presumably hourly and on quarter hours).

j45 6mo ago

Pulling all the skills and agents in the world in, when unused are a big hit. I deleted all of mine and added back as needed and there was an improvement. Running Claude Cowork in the background will hit tokens and it might not be the most efficient use of token use. Last, but not least, turning off 1M token context by default is helpful.

docheinestages 6mo ago

Why are you all of a sudden running into so many issues like this? Could it be that all of the Anthropics employees have completely unlimited and unbounded accounts, which means you don't get a feeling of how changes will affect the customers?

bcherny 6mo ago
The number of people using Claude Code has grown very quickly, which means: - More configurations and environments we need to test - Given an edge/corner case, it is more likely a significant number of users run into it - As the ecosystem has grown, more people use skills and plugins, and we need to offer better tools and automation to ensure these are efficient We do actually dogfood rate limits, so I think it's some combination of the above.
- gozucito 6mo ago
  I think the suspicion regarding skills and plugins is fair and logical. And it is absolutely the case that some use significantly more tokens. with that said, on my 5x plan, I could have multiple sessions working and the limit was far away. Around when you introduced the whole more tokens during off-peak hours and fewer tokens during working US hours, Even with a single session, using no plugins at all (I uninstalled OMC) I run into limits very often. I have not performed any rigorous tests but it feels like I have about 25% of what I used to have or less. This is all without using teams of agents, or ralph loops or anything like that. Just /plan and execute in a single session. I have restored the /clear context before executing plan to try and mitigate things. I will also try the 400k context since, in my experience, the 1M tokens have not made Opus 4.6 noticeably smarter for my small webapp use-case. Best of luck to you! ps: whenever you introduce a change, please make it optional AND ask the user about it at first. Don't just yank things suddenly (like the /clear context and apply plan option.) as I spent hours trying to figure out how I broke it before I saw your note and how to re-enable it.
- sharts 6mo ago
  How do ya’ll test?
- 6mo ago

avree 6mo ago

Hey Boris - why is the best way to get support making a Hacker News or X post, and hoping you reply? Why does Anthropic Enterprise Support never respond to inquiries?

egamirorrim 6mo ago
I mean if we're building an unrelated wishlist... Can 20x max users get auto mode already? Or can the enterprise plans get something equivalent to 20x max? Given I'm running two max accounts to get the usage I want, can we get a 25x and 40x tier? :-)
- oefrha 6mo ago
  It’s called /extra-usage and they really want you to use it.

_fizz_buzz_ 6mo ago

I have a feature request: I build an mcp server, but now it has over 60 tools. Most sessions i really don’t need most of them. I suppose I could make this into several servers. But it would maybe be nice to give the user more power here. Like let me choose the tools that should be loaded or let me build servers that group tools together which can be loaded. Not sure if that makes sense …

hughw 6mo ago

Where can i learn about concepts like prompt cache misses? I don't have a mental model how that interacts with my context of 1M or 400k tokens... I can cargo cult follow instructions of course but help us understand if you can so we can intelligently adapt our behavior. Thanks.

hughw 6mo ago
And why does /clear help things? Doesn't that wipe out the history of that session? Jeez.
- CWwdcdk7h 6mo ago
  [dead]
bcherny 6mo ago
The docs are a good place to start: https://platform.claude.com/docs/en/build-with-claude/prompt-caching#how-prompt-caching-works https://platform.claude.com/docs/en/build-with-claude/prompt...
- snthpy 6mo ago
  Thanks. Just noting that those docs say the cache duration is 5 min and not 1 hour as stated in sibling comment: > By default, the cache has a 5-minute lifetime. The cache is refreshed for no additional cost each time the cached content is used. > > If you find that 5 minutes is too short, Anthropic also offers a 1-hour cache duration at additional cost.
  - yoaviram 6mo ago
    Apparently Anthropic downgraded cache TTL to 5 min without telling anyone. My biggest issue with the recent issues with Claude Code is the lack transparency, although it looks like even Boris doesn't know about one: https://news.ycombinator.com/item?id=47736476 https://news.ycombinator.com/item?id=47736476

reenorap 6mo ago

Boris, you're seeing a ton of anecdotes here and Claude has done something that has affected a bunch of their most fervent users. Jeff Bezos famously said that if the anecdotes are contradicting the metrics, then the metrics are measuring the wrong things. I suggest you take the anecdotes here seriously and figure out where/why the metrics are wrong.

bcherny 6mo ago
We are taking it seriously, and are continuing to investigate. We are not trusting the metrics.
- Ucalegon 6mo ago
  [flagged]
  - amirhirsch 6mo ago
    Dude is on hacker news on a Sunday. half the GDP of the world is competing with him. What metrics would you like to see?
    - Ucalegon 6mo ago
      An enforceable SLA with the services that Anthropic offers rather than putting an employee to respond to things on Sunday.
      - roamerz 6mo ago
        >> rather than putting an employee to respond to things on Sunday. Maybe just maybe they didn’t put him here, rather he just a normal guy who reads HN, who is passionate about his role, and is here on his own time.
        Ucalegon 6mo ago
        Maybe... maybe... maybe... none of this builds trust when there is something that does build trust; putting revenue on the line and opening yourself to legal liability. Otherwise everything is empty and meaningless, its just PR, and nothing more.

samuelknight 6mo ago

Have you considered poking the cache? When a user walks away during the business day but CC is sitting open, you can refresh that cache up to 10x before it costs the same as a full miss. Realistically it would be <8x in a working day.

dkersten 6mo ago

Eh you say that every time and yet it keeps happening.

999900000999 6mo ago

You've created quite a conundrum. The only people who are going to run into issues are superpower users who are running this excessively beyond any reasonable measure. Most people are going to be quite happy with your service. But at the same time, and this is just a human nature thing people are 10 times more likely we complain about an issue than to compliment something working well. I don't know how to fix this, but I strongly suspect this isn't really a technical issue. It's more of a customer support one.

ramon156 6mo ago

Boris, wasnt this the same thing ~2 weeks ago? Is it the same cache misses as before? What's the expected time till solved? Seems like its taking a while

fluidcruft 6mo ago

How can we turn of 1m context? I don't find it has ever helped.

mwigdahl 6mo ago
He mentioned this in his original comment: "CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000"
- fluidcruft 6mo ago
  There's also CLAUDE_CODE_DISABLE_1M_CONTEXT and I'm really not clear on what the difference is and why to pick one over the other. But I guess one disables models that have 1m and the other keeps those models but sets the limit lower?

earino 6mo ago

Hello Boris! How do I increase the 1 hour prompt cache window for the main agent? I would love to be able to set that to, say, 4 hours. That gives me enough time to work on something, go teach a class, grab a snack, and come back and pick up where I left off.

subscribed 6mo ago
Another CC team member confirmed it's 5 minutes now, not 1 hour. See the links in https://news.ycombinator.com/item?id=47747209 https://news.ycombinator.com/item?id=47747209

re-thc 6mo ago

> To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session) Is this really an improvement? Shouldn't this be something you investigate before introducing 1M context? What is a long stale session? If that's not how Claude Code is intended to be used it might as well auto quit after a period of time. If not then if it's an acceptable use case users shouldn't change their behavior. > People pulling in a large number of skills, or running many agents or background automations, which sometimes happens when using a large number of plugins. If this was an issue there should have been a cap on it before the future was released and only increased once you were sure it is fine? What is "a large number"? Then how do we know what to do? It feels like "AI" has improved speed but is in fact just cutting corners.

j-pb 6mo ago

The /clear nudge isn't a solution though. Compacting or clearing just means rebuilding context until Claude is actually productive again. The cost comes either way. I get that 1M context windows cost more than the flat per-token price reflects, because attention scales with context length, but the answer to that is honest pricing or not offering it. Not annoying UX nudges. What’s actually indefensible is that Claude is already pushing users to shrink context via, I presume, system prompt. At maybe 25% fill: “This seems like a good opportunity to wrap it up and continue in a fresh context window.” “Want to continue in a fresh context window? We got a lot of work done and this next step seems to deserve a fresh start!” If there’s a cost problem, fix the pricing or the architecture. But please stop the model and UI from badgering users into smaller context windows at every opportunity. That is not a solution, it’s service degradation dressed as a tooltip.

g4cg54g54 6mo ago
currently "clear makes it worse" https://github.com/anthropics/claude-code/issues/47098 https://github.com/anthropics/claude-code/issues/47098 + https://github.com/anthropics/claude-code/issues/47107 https://github.com/anthropics/claude-code/issues/47107 launching with `CLAUDE_CODE_DISABLE_GIT_INSTRUCTIONS=1 claude "Hello"` till those are fixed seems to be th way
foota 6mo ago
The cost issues they're seeing (at least from what they've stated) are from users, not internally. Basically, it takes either $5 or $6.25 (depending on 5m or 1h ttl) to re-ingest a 1M context length conversation into cache for opus 4.6, that's obviously a very high cost, and users are unhappy with it. I think 400k as a default seems about right from my experience, but just having the ability to control it would be nice. For the record, even just making a tool call at 1M tokens costs 50 cents (which could be amortized if multiple calls are made in a round), so imo costs are just too high at long context lengths for them to be the default.

varispeed 6mo ago

Can you explain why Opus 4.6 suddenly becomes dumb as a sack of potatoes, even if context is barely filled? Can you explain why Opus 4.6 will be coming up with stupid solutions only to arrive at a good one when you mention it is trying to defraud you? I have a feeling the model is playing dumb on purpose to make user spend more money. This wasn't the case weeks ago when it actually working decently.

accounting2026 6mo ago

[flagged]

MuffinFlavored 6mo ago

I wish people would pay more attention to: * Anthropic is in some way trying to run a business (not a charity) and at least (eventually?) make money and not subsidize usage forever * "What a steal/good deal" the $100-$200/mo plans are compared to if they had to pay for raw API usage and less on "how dare you reserve the right to tweak the generous usage patterns you open-ended-ly gave us, we are owed something!"

logicchains 6mo ago
All those apply to OpenAI+Codex too, but they're far more generous with limits than Anthropic, and with granting fresh limits to apologize when they fuck up.
lbreakjai 6mo ago
As an (ex) paying customer, I'm expecting some consistency. I used to be satisfied with the value I got, until the limits changed overnight, and I'd get a ten of my previous usage. If Anthropic is allowed to alter the deal whenever, then I'd expect to be able to get my money back, pro-rata, no questions asked.
oskarw85 6mo ago
[flagged]
visarga 6mo ago
yes, $200/mo is a serious subscription, we are owed something, and I won't feel ashamed for saying that especially when you are told using the subagent for code review "claude -p" is now billed on API on top of $200 sub

danmaz74 6mo ago

Could we get an option to use Opus with a smaller context window? I noticed that results get much worse way earlier than when you reach 1M tokens, and I would love to have a setting so that I could force a compaction at eg 300k tokens.

SyneRyder 6mo ago
You probably just missed it in his post, but: "To experiment with this now, try: CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000 claude." Maybe try changing the 4 to a 3 and see if that works for you?
- danmaz74 6mo ago
  Thank you, will definitely try that!
stitched2gethr 5mo ago
Try this: `claude --model claude-opus-4-6`

EGreg 6mo ago

Boris, is the KV cache TTL now reduced to 5 minutes from 1 hour? I think this may be the biggest concern for people building tools on the API: https://github.com/anthropics/claude-code/issues/46829 https://github.com/anthropics/claude-code/issues/46829 I would argue that KV caching is a net gain for Ant and a well-maintained cache is the biggest thing that can generate induced demand and a thriving third party ecosystem. https://safebots.ai/papers/KV.pdf https://safebots.ai/papers/KV.pdf

KronisLV 6mo ago

> defaulting to 400k context instead, with an option to configure your context window to up to 1M if preferred This seems really useful! I'm surprised that "Opus 4.6" (200K) and "Opus 4.6 1M" are the only Opus options in the desktop app, whereas in the CLI/TUI app you don't seem to even get that distinction. I bet that for a lot of folks something like 400k, 600k or 800k would work as better defaults, based on whatever task they want to work on.

ahofmann 6mo ago

Resizing the context window seems like a very good idea to me. I noticed a decline of productivity when the 1M context window was released and I'd like to bring it back to 200k, because it was totally fine for the things I was working on.

deleted 6mo ago

[deleted]

jauntywundrkind 6mo ago

There's an issue someone raised showing that prompt caches are only 5 minutes. The reply seems to be: oh huh, interesting. Maybe that's a good thing since people sometimes one-shot? That doesn't feel like the messaging I want to be reading, and the way it conflicts with the message here that cache is 1 hour is confusing. https://news.ycombinator.com/item?id=47741755 https://news.ycombinator.com/item?id=47741755 Is there any status information or not on whether cache is used? It sure looks like the person analyzing the 5m issue had to work extremely hard to get any kind of data. It feels like the iteration loop of people getting better at this stuff would go much much better if this weren't such a black box, if we had the data to see & understand: is the cache helping?

TheTaytay 6mo ago
Aren’t they saying that it’s 5minutes for things like subagents (that wouldn’t benefit from it?)

yumraj 6mo ago

> Since Claude Code uses a 1 hour prompt cache window for the main agent, if you leave your computer for over an hour then continue a stale session, it's often a full cache miss. To improve this, we have shipped a few UX improvements (eg. to nudge you to /clear before continuing a long stale session), and are investigating defaulting to 400k context instead I don’t understand this. I frequently have long breaks. I never want to clear or even compact because I don’t want to lose the conversations that I’ve had and the context. Clearing etc causes other issues like I have to restate everything at times and it misses things. I do try to update the memory which helps. I wish there was a better solution than a time bound cache

cowwoc2020 6mo ago
Makes me wish that shortly before the server-side expiration, we could save the cache on the client-side, indefinitely. But my understanding is that we're talking about ~60GB of data per session, so it sounds unrealistic to do...
- yumraj 6mo ago
  Where are you getting 60GB from? It shouldn’t be that large. But yes, would love to save context/cache such that it can be played back/referred to if needed. /compact is a little black box that I just have to trust that is keeping the important bits.
  - davmre 6mo ago
    The KV cache consists of activation vectors for every attention head at every layer of the model for every token, so it gets quite large. ChatGPT also estimates 60-100GB for full token context of an Opus-sized model: https://chatgpt.com/share/69dc5030-268c-83e8-92c2-6cef962dc5c2 https://chatgpt.com/share/69dc5030-268c-83e8-92c2-6cef962dc5...
    - visarga 6mo ago
      There are ways to quantize or compress KV cache down.

g4cg54g54 6mo ago

from looking at the raw requests, that cant seem right? its all "cache_control": { "type": "ephemeral" } there is no "ttl" anywhere. // edit: cc_version=2.1.104.f27

throwpoaster 6mo ago

Wait what? If I get told to come back in three hours because I'm using the product too much, I get penalized when I resume? What's the right way to work on a huge project then? I've just been saying "Please continue" -- that pops the quota?

apgwoz 6mo ago

As another data point, I pay for Pro for a personal account, and use no skills, do nothing fancy, use the default settings, and am out of tokens, with one terminal, after an hour. This is typically working on a < 5,000 line code base, sometimes in C, sometimes in Go. Not doing incredibly complicated things.

throwpoaster 6mo ago

Have you tried asking Mythos for a fix?

8note 6mo ago

> Since Claude Code uses a 1 hour prompt cache window for the main agent this seems a bit awkward vs the 5 hour session windows. if i get rate limited once, I'll get rate limited immediately again on the same chat when the rate limit ends? any chance we can get some form of deffered cache so anything on a rate limited account gets put aside until the rate limit ends?

mmd45 6mo ago

shouldn't compaction be interactive with the user as to what context will continue to be the most relevant in the future??? what if the harness allowed for a turn to clarify the user's expected future direction of the conversation and did the consolidation based upon the addition info? there definitely seems to be a benefit to pruning the context and keeping the signal to noise high wrt what is still to be discussed.

cowwoc2020 6mo ago

Boris, Even if Anthropic is working in good faith to lower infrastructure costs, developers need more than 5 minutes to notice that CC completed a task, review its changes and ask it to merge. Only developers who do not review code changes can live with such a TTL... Consider making this value configurable as the ideal TTL value is different for each person. If people are willing to pay more for 30 minutes TTL than 5 minutes, they should be able to.

deleted 6mo ago
[deleted]

jiwidi 6mo ago

Hi Boris, Long term claude code user here. Is the first time i've had to setup a hook to codex to review claude output. Is hallucinating like never before Is missing key concepts/instructions in context like never before Is writing bad code that will "pass test" much more. Before it use to try be critic and do good code, now it will try to hack test and bypass intructions for a green pass.

cmaster11 6mo ago

Thank you for your responses, especially on a Sunday. They give us some insights and at least a couple temporary workarounds to use, while the issues are being addressed :) much appreciated

sunir 6mo ago

I suspect 1M token context is questionable value because of the secondary effect of burning quota vs getting work done. I think the model select that let me choose 1M made sense because I could decide if I was working on large documents and compacting more often was more effective.

anoazian 6mo ago

I’ve seen the /clear command prompt and I found the verbiage to be a bit unclear. I think clarifying that the cache has expired and providing an understandable metric on the impact - ie “X% of your 5-hour window” for Pro/Mad users and details on token use for API users. A pop-up that requires explicit acknowledgment might also help, although that could be more of an annoyance to enterprise users. One pattern I use frequently is using one high level design and implementation agent that I’ll use for multiple sessions and delegate implementation to lower level agents. In this case it’d be helpful to have one of two options: 1. If Claude CLI could create an auto compaction of the conversation history before cache expiration. For example, if I’m beyond X minutes or Y prompts in a conversation and I’ve been inactive for a threshold it could auto-compact close to the expiration and provide that as an option on resume. 2. If I could configure cache expiration proactively and Anthropic could use S3 or a similar slow load mechanism to offload the cache for a longer period - possibly 24-72h. I can appreciate that longer KV cache expiration would complicate capacity management and make inference traffic less fungible but I wouldn’t mind waiting seconds to minutes for it to load from a slower store to resume without quota hits.

taspeotis 6mo ago

Hi, thanks for Claude Code. I was wondering though if you'd considering adding a mode to make text green and characters come down from the top of the screen individually, like in The Matrix?

visarga 6mo ago
No, I want a little monkey doing tricks. /s
- taspeotis 6mo ago
  This guy? https://en.wikipedia.org/wiki/BonziBuddy https://en.wikipedia.org/wiki/BonziBuddy

tigershark 6mo ago

Claude Code cache is not 1 hour. There is a "Closed as not planned" issue in GitHub that confirms that it has been moved to 5 minutes since March: https://github.com/anthropics/claude-code/issues/46829 https://github.com/anthropics/claude-code/issues/46829. I started seeing the massive degradation exactly on the 23rd of March, hence after a few days I unsubscribed because it was completely unusable, with a ~5h session being depleted in as little as 15-20 mins.

subscribed 6mo ago
Looks like the cache change to 5 minutes was so secretive that even CC team doesn't know that. Or someone just vibe coded "Hey, Claude, make them burn allowances quicker" and merged without telling anyone. Both are plausible to me.

smrtinsert 6mo ago

Number 2 makes me chuckle honestly. Too many people going down the 10x rabbit holes on youtube. Next up, a framework that 100xs your workflow. You know its good because it comes with 300 agents and 20 mcp servers and 1200 skills

foofloobar 6mo ago

[flagged]

stingraycharles 6mo ago
This comment seems unnecessarily hostile.
- foofloobar 6mo ago
  They appear to take issues seriously mostly when they become posts on hacker news and when articles are published online by major news sites. Customer support is mostly a bot. I don't even know how to reach some actual humans to get support. I'm sorry if you and others are offended. They've had these issues for several weeks now. I haven't seen any real improvements during this time. I see more features and more bugs. There have been several releases made over the last few days without any changelogs. The quotas are still as opaque as they've been. This company has some extremely shady business practices.
- prmph 6mo ago
  Why? It seems just fine to me. This is what Anthropic needs to do if they want to survive. I'm always looking out for someone to integrate an actually good harness to a good model. Once that happens, I'm jumping ship if Anthropic keeps playing these tricks. It's almost unusable for me now. A simple prompt to merge 3 sub-100-line files with simple node code, on Sonnet 4.6, uses up 20% of my 5 hour quota, on a new/fresh session.
  - foofloobar 6mo ago
    To be fair, my comment was a bit harsher before the update. The way they handle the development, communication and how they treat customers isn't fine. I've seen some angry people post and comment in manners which truly deserved the label hostile. The whole product with the infrastructure and Claude Code's code appear to be vibe coded.
    - 6mo ago

pu_pe 6mo ago

So Anthropic is trying to save money on infrastructure, we all get it. However, it's not ok to degrade the performance your users have paid for. Last week the issue was that you reduced the default "effort" level, now the prompt cache is shortened. Several users experience far more restrictive usage limits lately. There is only so much you can do through "UX improvements" or some smart routing on the backend. Your flagship product is actively getting worse, and if users need to fiddle with hidden settings and keep track of GitHub issues every week they will start voting with their money.

baq 6mo ago
Dear sir, please think of the shareholders, they need a fair exit.
steveBK123 6mo ago
Classic VC pump playbook - run it uneconomically until everyone is addicted, then 5x prices once you have enough critical mass. See 2010s "Millennial Lifestyle Subsidy"..
- FuckButtons 6mo ago
  It seems pretty transparent that they are heavily resource constrained, (training run for Claude 5.x, higher usage / growth than anticipated). I don’t disagree that their long play is monopolistic pricing, but what we’re observing seems better explained by the fact they have a very tight compute budget they are trying to optimize over to put as much as they can into next gen experiments / training to make sure they stay competitive over the next 6-months / year.
- lukewarm707 6mo ago
  you know once, anthropic was supposed to be a public benefit org!
zachrip 6mo ago
Where did they say the prompt cache is shortened?

richardjennings 6mo ago

/loop message ping every 4 minutes keeps the cache warm while the CC REPL is not active.

miroljub 6mo ago

People, just switch to MiniMax and ditch CC completely. It's not worth it any more.

je42 6mo ago

Does this 60min ttl of also apply to claude code web? I have regularly sessions open for multiple days. Is that a pattern that is not advised?

FabDee 6mo ago

One thing I didn't see anywhere here, except your mention about pulling in large number of skills, is that the token consumption is significantly higher for users with many agents, skills, and MCPs installed, and many are mere ghosts. The 5m TTL from #46829 compounds the effect: in my case, I found ~20k tokens of ghost context I hadn't intentionally opened. Each idle period after 5m wastes that as a full cache miss. Boris, would you please confirm on-record: is the current cache TTL for the main agent context 1h or 5m? Issue #46829 was closed as "not planned".

guybedo 6mo ago

you should check with people working on Claude Code, cache has been udpated to 5min ... https://github.com/anthropics/claude-code/issues/46829#issuecomment-4231266649 https://github.com/anthropics/claude-code/issues/46829#issue... So yeah, 1M window that expires every 5min .... not good

mercnz 6mo ago

it seems if context can't be held for over an hour it should warn you a countdown or such; i already enabled the tokens verbosity thing to see what token level i'm at, but i often leave things sitting rather than complete so that i'm tying things up to start something new in the morning rather than starting on a new thing. so like i just resumed a session that was near-complete, and now it's gone and reloaded all that session in? bit i hadn't detached it. i kind of thougth /summary itself had to read the whole token flow, but that the token context was held locally for some reason..