14 ms·

Claude Sonnet 4 now supports 1M tokens of context

• cadamsdotcom 1y ago
  I’m glad to see the only company chasing margins - which they get by having a great product and a meticulous brand - finding even more ways to get margin. That’s good business.
• revskill 1y ago
  The critical issue with LLM which never beats human: break what worked.
• throwaway888abc 1y ago
  holy moly! awesome
• mettamage 1y ago
  Shame it's only the API. Would've loved to see it via the web interface on claude.ai itself.
  • minimaxir 1y ago
    Can you even fit 200+k tokens worth of context in the web interface? IMO Claude's API workbench is the worst of the three major providers.
    • mettamage 1y ago
      Via text files right? Just drag and drop.
    • data-ottawa 1y ago
      When working on artifacts after a few change requests it definitely can.
    • 77pt77 1y ago
      Even if you can't, a conversation can easily get larger than that.
  • fblp 1y ago
    I assume this will mean that long chats continue to get the "prompt is too long" error?
• tankenmate 1y ago
  This is definitely good to have this as an option but at the same time having more context reduces the quality of the output because it's easier for the LLM to get "distracted". So, I wonder what will happen to the quality of code produced by tools like Claude Code if users don't properly understand the trade off being made (if they leave it in auto mode of coding right up to the auto compact).
  • jasonthorsness 1y ago
    What do you recommend doing instead? I've been using Claude Code a lot but am still pretty novice at the best practices around this.
    • TheDong 1y ago
      Have the AI produce a plan that spans multiple files (like "01 create frontend.md", "02 create backend.md", "03 test frontend and backend running together.md"), and then create a fresh context for each step if it looks like re-using the same context is leading it to confusion. Also, commit frequently, and if the AI constantly goes down the wrong path ("I can't create X so I'll stub it out with Y, we'll fix it later"), you can update the original plan with wording to tell it not to take that path ("Do not ever stub out X, we must make X work"), and then start a fresh session with an older and simpler version of the code and see if that fresh context ends up down a better path. You can also run multiple attempts in parallel if you use tooling that supports that (containers + git worktrees is one way)
      • F7F7F7 1y ago
        Inventivatbly the files become a mess of their own. Changes and learnings from one part of the plan often dont result in adaptation to impacted plans down chain. In the end you have a mish mash of half implemented plans and now you’ve lost context too. Which leads to blowing tokens on trying to figure out what’s been implemented, what’s half baked, and what was completely ignored. Any links to anyone who’s built something at scale using this method? It always sounds good on paper. I’d love to find a system that works.
• falcor84 1y ago
  Strange that they don't mention whether that's enabled or configurable in Claude Code.
  • farslan 1y ago
    Yeah same, I'm curious about this. I would guess it's by default enabled with Claude Code.
  • csunoser 1y ago
    They don't say it outright. But I think it is not in Claude Code yet. > We’re also exploring how to bring long context to other Claude products. - Anthropic That is, any other product that is not Anthropic API tier 4 or Amazon bedrock.
  • CharlesW 1y ago
    From a co-marketing POV, it's considered best practice to not discuss home-grown offerings in the same or similar category as products from the partners you're featuring. It's likely they'll announce this week, albeit possibly just within the "what's new" notes that you see when Claude Code is updated.
    • reasonableklout 1y ago
      They just sent an email that the feature is in beta in CC.
• faangguyindia 1y ago
  In my testing the gap between claude and gemini pro 2.5 is close. My company is in asia pacific and we can't get access to claude via vertex for some stupid reason. but i tested it via other providers, the gap used to be huge but now not.
  • Tostino 1y ago
    For me the gap is pretty large (in Gemini Pro 2.5's favor). For reference, the code I am working on is a Spring Boot / (Vaadin) Hilla multi-module project with helm charts for deployment and a separate Python based module for ancillary tasks that were appropriate for it. I've not been able to get any good use out of Sonnet in months now, whereas Gemini Pro 2.5 has (still) been able to grok the project well enough to help out.
    • jona777than 1y ago
      I initially found Gemini Pro 2.5 to work well for coding. Over time, I found Claude to be more consistently productive. Gemini Pro 2.5 became my go-to for use cases benefitting from larger context windows. Claude seemed to be the safer daily driver (if I needed to get something done.) All that being said, Gemini has been consistently dependable when I had asks that involved large amounts of code and data. Claude and the OpenAI models struggled with some tasks that Gemini responsively satisfied seemingly without "breaking a sweat." Lately, it's been GPT-5 for brainstorming/planning, Claude for hammering out some code, Gemini when there is huge data/code requirements. I'm curious if the widened Sonnet 4 context window will change things.
    • llm_nerd 1y ago
      Opus 4.1 is a much better model for coding than Sonnet. The latter is good for general queries / investigations or to draw up some heuristics. I have paid subscriptions to both Gemini Pro and Claude. Hugely worthwhile expense professionally.
    • faangguyindia 1y ago
      when gemini 2.5 pro gets stuck, i often use deep seek r1 in architect mode and qwen3 in coder mode in aider and it solves all the problem last month i ran into some wicked dependency bug and only chatgpt could solve it which i am guessing is the case because it has hot data from github? On the other hand, i really need a tool like aider where i can use various models in "architect" and "coder" mode. what i've found is better reasoning models tend to be bad at writing actual code, and models like qwen3 coder seems better. deep seek r1 will not write reliable code but it will reason well and map out the path forward. i wouldn't be surprised if sonnets success was doing EXACTLY this behind the scenes. but now i am looking for pure models who do not use this black magic hack behind API. I want more control at tool end where i can alter the prompts and achieve results i want this is one reason i do not use claude code etc.... aider is 80% of what i want wish it had more of what i want though. i just don't know why no one has build a perfect solution to this yet. Here are things i am missing in aider 1. Automatic model switching, use different models for asking questions about the code, different one for planning a feature, different one for writing actual code. 2. Self determine, if a feature needs a "reasoning" model or coding model will suffice. 3. be able to do more, selectively send context and drop the files we don't need. Intelligently add files to context which will be touched by the feature, not after having done all code planning asking to add files, then doing it all over again with more context available.
• artursapek 1y ago
  Eagerly waiting for them to do this with Opus
  • irthomasthomas 1y ago
    Imagine paying $20 a prompt?
    • artursapek 1y ago
      If I can give it a detailed spec, walk away and do something else for 20 minutes, and come back to work that would have taken me 2 hours, then that's a steal.
      • dbbk 1y ago
        You can just do this now though. In fact you could go a step further and set up the GitHub Action, then you can kick off Claude from the iOS GitHub app from the beach and review the PR when it's done.
    • datadrivenangel 1y ago
      Depending on how many prompts per hour you're looking at, that's probably same order of magnitude as expensive SAAS. A fancy CRM seat can be ~$2000 per month (or more), which assuming 50 hours per week x 4 weeks per month is $10 per hour ($2000/200 hours). A lot of money, but if it makes your sales people more productive, it's a good investment. Assuming that you're paying your sales people say 240K per year, ($20,000 per month), then the SAAS cost is 10% of their salary. This explains DataDog pricing. Maybe it will give a future look at AI pricing.
• penguin202 1y ago
  But will it remember any of it, and stop creating new redundant files when it can't find or understand what its looking for?
• 1xer 1y ago
  moaaaaarrrr
• aliljet 1y ago
  This is definitely one of my CORE problem as I use these tools for "professional software engineering." I really desperately need LLMs to maintain extremely effective context and it's not actually that interesting to see a new model that's marginally better than the next one (for my day-to-day). However. Price is king. Allowing me to flood the context window with my code base is great, but given that the price has substantially increased, it makes sense to better manage the context window into the current situation. The value I'm getting here flooding their context window is great for them, but short of evals that look into how effective Sonnet stays on track, it's not clear if the value actually exists here.
  • rootnod3 1y ago
    Flooding the context also means increasing the likelihood of the LLM confusing itself. Mainly because of the longer context. It derails along the way without a reset.
    • aliljet 1y ago
      How do you know that?
      • EForEndeavour 1y ago
        https://onnyunhui.medium.com/evaluating-long-context-lengths-in-llms-challenges-and-benchmarks-ef77a220d34d https://onnyunhui.medium.com/evaluating-long-context-lengths...
      • bigmadshoe 1y ago
        https://research.trychroma.com/context-rot https://research.trychroma.com/context-rot
        • joenot443 1y ago
          This is a good piece. Clearly it's a pretty complex problem and the intuitive result a layman engineer like myself might expect doesn't reflect the reality of LLMs. Regex works as reliably on 20 characters as it does 2m characters; the only difference is speed. I've learned this will probably _never_ be the case with LLMs, there will forever exist some level of epistemic doubt in its result. When they announced Big Contexts in 2023, they referenced being able to find a single changed sentence in the context's copy of Great Gatsby[1]. This example seemed _incredible_ to me at the time but now two years later I'm feeling like it was pretty cherry-picked. What does everyone else think? Could you feed a novel into an LLM and expect it to find the single change? [1] https://news.ycombinator.com/item?id=35941920 https://news.ycombinator.com/item?id=35941920
• rootnod3 1y ago
  So, more tokens means better but at the same time more tokens means it distracts itself too much along the way. So at the same time it is an improvement but also potentially detrimental. How are those things beneficial in any capacity? What was said last week? Embrace AI or leave? All I see so far is: don't embrace and stay.
  • rootnod3 1y ago
    So, I see this got downvoted. Instead of just downvoting, I would prefer to have a counter-argument. Honestly. I am on the skeptic side of LLM, but would not mind being turned to the other side with some solid arguments.
• pupppet 1y ago
  How does anyone send these models that much context without it tripping over itself? I can't get anywhere near that much before it starts losing track of instruction.
  • 9wzYQbTYsAIc 1y ago
    I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts. Right now I’m experimenting with using separate .plan files for tracking key instructions across domains like architecture and feature decisions.
    • CharlesW 1y ago
      > I’ve been having decent luck telling it to keep track of itself in a .plan file, not foolproof, of course, but it has some ability to “preserve context” between contexts. This is the way. Not only have I had good luck with both a TASKS.md and TASKS-COMPLETE.md (for history), but I have an .llm/arch full of AI-assisted, for-LLM .md files (auth.md, data-access.md, etc.) that document architecture decisions made along the way. They're invaluable for effectively and efficiently crossing context chasms.
    • collinvandyck76 1y ago
      Yeah, this. Each project I work on has it's own markdown file named for the ticket or the project. Committed on the branch, and I have claude rewrite it with the "current understanding" periodically. After compacting, I have it re-read the MD file and we get started again. Quite nice.
  • olddustytrail 1y ago
    I think it's key to not give it contradictory instructions, which is an easy mistake to make if you forget where you started. As an example, I know of an instance where the LLM claimed it had tried a test on its laptop. This obviously isn't true so the user argued with it. But they'd originally told it that it was a Senior Software Engineer so playing that role, saying you tested locally is fine. As soon as you start arguing with those minor points you break the context; now it's both a Software Engineer and an LLM. Of course you get confused responses if you do that.
• greenfish6 1y ago
  Yes, but if you look in the rate limit notes, the rate limit is 500k tokens / minite for tier 4, which we are on. Given how stingy anthropic has been with rate limit increases, this is for very few people right now
• alvis 1y ago
  Context window after certain size doesn’t bring in much benefit but higher bill. If it still keeps forgetting instructions it would be just much easier to be ended up with long messages with higher context consumption and hence the bill I’d rather having an option to limit the context size
  • EcommerceFlow 1y ago
    It does if you're working with bigger codebases. I've found copy/pasting my entire codebase + adding a <task> works significantly better than cursor.
    • spiderice 1y ago
      How does one even copy their entire codebase? Are you saying you attach all the files? Or you use some script to copy all the text to your clipboard? Or something else?
      • EcommerceFlow 1y ago
        I created a script that outputs the entire codebase to a text file (also allows me to exclude files/folders/node_modules), separating and labeling each file in the program folder. I then structure my prompts around like so: <project_code> ``` ``` </project_code> <heroku_errors> " " </heroku_errors> <task> " " </task> I've been using this with Google Ai studio and it's worked phenomenally. 1 million tokens is A LOT of code, so I'd imagine this would work for lots n lots of project type programs.
      • swader999 1y ago
        Repomix, there's a cli and an MCP.
• andrewstuart 1y ago
  Oh man finally. This has been such a HUGE advantage for Gemini. Could we please have zip files too? ChatGPT and Gemini both unpack zip files via the chat window. Now how about a button to download all files?
• qsort 1y ago
  I won't complain about a strict upgrade, but that's a pricy boi. Interesting to see differential pricing based on size of input, which is understandable given the O(n^2) nature of attention.
• isoprophlex 1y ago
  1M of input... at $6/1M input tokens. Better hope it can one-shot your answer.
  • elitan 1y ago
    have you ever hired humans?
    • bicepjai 1y ago
      Depends on which human you tried :) Donot underestimate yourself !
• deleted 1y ago
  [deleted]
• rafaelero 1y ago
  god they keep raising prices
• henriquegodoy 1y ago
  Thats incredible to see how ai models are improving, i'm really happy with this news. (imo it's more impactful than the release of gpt5) now, we need more tokens per second, and then the self-improvement of the model will accelerate.
• lherron 1y ago
  Wow, I thought they would feel some pricing pressure from GPT5 API costs, but they are doubling down on their API being more expensive than everyone else.
  • sebzim4500 1y ago
    I think it's the right approach, the cost of running these things as coding assistants is negligable compared to the benefit of even a slight model improvement.
  • AtNightWeCode 1y ago
    GPT5 API uses more tokens for answers of the same quality as previous versions. Fell into that trap myself. I use both Claude and OpenAI right now. Will probably drop OpenAI since they are obviously not to be trusted considering the way they do changes.
• shamano 1y ago
  1M tokens is impressive, but the real gains will come from how we curate context—compact summaries, per-repo indexes, and phase resets. Bigger windows help; guardrails keep models focused and costs predictable.
• jbellis 1y ago
  Just completed a new benchmark that sheds some light on whether Anthropic's premium is worth it. (Short answer: not unless your top priority is speed.) https://brokk.ai/power-rankings https://brokk.ai/power-rankings
  • 24xpossible 1y ago
    Why no Grok 4?
    • Zorbanator 1y ago
      You should be able to guess.
    • jeffhuys 1y ago
      People hate it because it had less filters and media caught on, so they told people to hate it. It’s actually the best one right now, or close to. For my uses (code and queries) nothing comes even close. Once people look past the “but ELoN mUssKkkk!!!”, they’ll be surprised.
    • jbellis 1y ago
      the accompanying blog post explains: xAI did not respond to our requests for a grok 4 quota that would allow us to run the evaluation
  • rcanepa 1y ago
    I recently switched to the $200 CC subscription and I think I will stay with it for a while. I briefly tested whatever version of ChatGPT 5 comes with the free Cursor plan and it was unbearably slow. I could not really code with it as I was constantly getting distracted while waiting for a response. So, speed matters a lot for some people.
• Someone1234 1y ago
  Before this they supposedly had a longer context window than ChatGPT, but I have workloads that abuse the heck out of context windows (100-120K tokens). ChatGPT genuinely seems to have a 32K context window, in the sense that is legitimately remembers/can utilize everything within that window. Claude previously had "200K" context windows, but during testing it wouldn't even hit a full 32K before hitting a wall/it forgetting earlier parts of the context. They also have extremely short prompt limits relative to the other services around, making it hard to utilize their supposedly larger context windows (which is suspicious). I guess my point is that with Anthropic specifically, I don't trust their claims because that has been my personal experience. It would be nice if this "1M" context window now allows you to actually use 200K though, but it remains to be seen if it can even do that. As I said with Anthropic you need to verify everything they claim.
  • Etheryte 1y ago
    Strong agree, Claude is very quick to forget things like "don't do this", "never do this" or things it tried that were wrong. It will happily keep looping even in very short conversations, completely defeating the purpose of using it. It's easy to game the numbers, but it falls apart in the real world.
    • joquarky 1y ago
      I've found it better to use antonyms than negations most situations.
      • typpilol 1y ago
        Same here. Always tell them the way you want it done. For example: Instead of "don't modify the tests" It should be: analyze the test output and fix the bug in the source code. The test is built correctly. Not the best but you get the idea. The one problem with this is if you don't know how to do something properly. Like if you're just writing in your prompt "generate 90% test coverage" , you give it a lot more leeway to do whatever it wants. And that's how you end up with the source code being modified to fit the test vice versa
• lvl155 1y ago
  Only time this is useful is to do init on a sizable code base or dump a “big” csv.
• film42 1y ago
  The 1M token context was Gemini's headlining feature. Now, the only thing I'd like Claude to work on is tokens counted towards document processing. Gemini will often bill 1/10th the tokens Anthropic does for the same document.
• varyherb 1y ago
  I believe this can be configured in Claude Code via the following environment variable: ANTHROPIC_BETAS="context-1m-2025-08-07" claude
  • falcor84 1y ago
    Have you tested it? I see that this env var isn't specified in their docs https://docs.anthropic.com/en/docs/claude-code/settings#environment-variables https://docs.anthropic.com/en/docs/claude-code/settings#envi...
    • bazhand 1y ago
      Add these settings to your `.claude/settings.json`: ```json { "env": { "ANTHROPIC_CUSTOM_HEADERS": {"anthropic-beta": "context-1m-2025-08-07"}, "ANTHROPIC_MODEL": "claude-sonnet-4-20250514", "CLAUDE_CODE_MAX_OUTPUT_TOKENS": 8192 } } ```
    • varyherb 1y ago
      Yup! Claude Code has a lot of undocumented configuration. Once I saw the beta header value in their docs [1], I tried to see in their source code if there was anyway to specify this flag via env var config. Their source code is already on your computer, just gotta dig through the minified JS :) Try: `cat $(which claude) | grep ANTHROPIC_BETAS` Sibling comment's approach with the other (documented) env var works too. [1] https://docs.anthropic.com/en/docs/build-with-claude/context-windows#1m-token-context-window https://docs.anthropic.com/en/docs/build-with-claude/context...
  • anonym29 1y ago
    Tested this morning. Worked wonderfully, except ran into output issues. Attempted to patched the minified Claude file's CLAUDE_CODE_MAX_OUTPUT_TOKENS hard limit of 32000 on Sonnet to 64000, which worked, and I was able to generate outputs above 32000 tokens, but this coincided with a breakage of the 1m context window for me. Still testing and playing around with this, but this may be getting patched?
• gdudeman 1y ago
  A tip for those who both use Claude Code and are worried about token use (which you should be if you're stuffing 400k tokens into context even if you're on 20x Max): 1. Build context for the work you're doing. Put lots of your codebase into the context window. 2. Do work, but at each logical stopping point hit double escape to rewind to the context-filled checkpoint. You do not spend those tokens to rewind to that point. 3. Tell Claude your developer finished XYZ, have it read it into context and give high level and low level feedback (Claude will find more problems with your developer's work than with yours). If you want to have multiple chats running, use /resume and pull up the same thread. Hit double escape to the point where Claude has rich context, but has not started down a specific rabbit hole.
  • rvnx 1y ago
    Thank you for the tips, do you know how to rollback latest changes ? Trying very hard to do it, but seems like Git is the only way ?
    • gdudeman 1y ago
      Git or my favorite "Undo all of those changes."
      • spike021 1y ago
        this usually gets the job done for me as well
    • SparkyMcUnicorn 1y ago
      I haven't used it, but saw this the other day: https://github.com/RonitSachdev/ccundo https://github.com/RonitSachdev/ccundo
    • rtuin 1y ago
      Quick tip when working with Claude Code and Git: When you're happy with an intermediate result, stage the changes by running `git add` (no commit). That makes it possible to always go back to the staged changes when Claude messes up. You can then just discard the unstaged changes and don't have to roll back to the latest commit.
• ZeroCool2u 1y ago
  It's great they've finally caught up, but unfortunate it's on their mid-tier model only and it's laughably expensive.
• thimabi 1y ago
  Oh, well, ChatGPT is being left in the dust… When done correctly, having one million tokens of context window is amazing for all sorts of tasks: understanding large codebases, summarizing books, finding information on many documents, etc. Existing RAG solutions fill a void up to a point, but they lack the precision that large context windows offer. I’m excited for this release and hope to see it soon on the UI as well.
  • OutOfHere 1y ago
    Fwiw, OpenAI does have a decent active API model family of GPT-4.1 with a 1M context. But yes, the context of the GPT-5 models is terrible in comparison, and it's altogether atrocious for the GPT-5-Chat model. The biggest issue in ChatGPT right now is a very inconsistent experience, presumably due to smaller models getting used even for paid users with complex questions.
    • wahnfrieden 1y ago
      Doesn’t it matter more what context they provide via Claude Code and Codex CLI? And arent they similar anyway there? Because the API with maximum context is very expensive (also not rolled out to everyone)
• kotaKat 1y ago
  A million tokens? Damn, I’m gonna need a lot of quarters to play this game at Chuck-E-Cheese.
• xnx 1y ago
  1M context windows are not created equal. I doubt Claude's recall is as good as Gemini's 1M context recall. https://cloud.google.com/blog/products/ai-machine-learning/the-needle-in-the-haystack-test-and-how-gemini-pro-solves-it https://cloud.google.com/blog/products/ai-machine-learning/t...
  • xnx 1y ago
    Good analysis here: https://news.ycombinator.com/item?id=44878999 https://news.ycombinator.com/item?id=44878999 > the model that’s best at details in long context text and code analysis is still Gemini. > Gemini Pro and Flash, by comparison, are far cheaper
• firasd 1y ago
  A big problem with the chat apps (ChatGPT; Claude.ai) is the weird context window hijinks. Especially ChatGPT does wild stuff.. sudden truncation; summarization; reinjecting 'ghost snippets' etc I was thinking this should be up to the user (do you want to continue this conversation with context rolling out of the window or start a new chat) but now I realized that this is inevitable given the way pricing tiers and limited computation works. Like the only way to have full context is use developer tools like Google AI Studio or use a chat app that wraps the API With a custom chat app that wraps the API you can even inject the current timestamp into each message and just ask the LLM btw every 10 minutes just make a new row in a markdown table that summarizes every 10 min chunk
  • cruffle_duffle 1y ago
    > btw every 10 minutes just make a new row in a markdown table that summarizes every 10 min chunk Why make it time based instead of "message based"... like "every 10 messages, summarize to blah-blah.md"?
    • dev0p 1y ago
      Probably it's more cost effective and less error prone to just dump the message log rather than actively rethink the context window, costing resources and potentially losing information in the process. As the models gets better, this might change.
    • firasd 1y ago
      Sure. But you'd want to help out the LLM with a message count like this is message 40, this is message 41... so when it hits message 50 it's like ahh time for a new summary and call the memory_table function (cause it's executing the earlier standing order in your prompt)
• tosh 1y ago
  How did they do the 1M context window? Same technique as Qwen? As Gemini?
• deadbabe 1y ago
  Unfortunately, larger context isn’t really the answer after a certain point. Small focused context is better, lazily throwing a bunch of tokens in as a context is going to yield bad results.
• ramoz 1y ago
  Awesome addition to a great model. The best interface for long context reasoning has been AIStudio by Google. Exceptional experience. I use Prompt Tower to create long context payloads.
• alienbaby 1y ago
  The fracturing of all the models offered across providers is annoying. The number of different models and the fact a given model will have different capabilities from different providers is ridiculous.
• simianwords 1y ago
  How does "supporting 1M tokens" really work in practice? Is it a new model? Or did they just remove some hard coded constraint?
  • eldenring 1y ago
    Serving a model efficiently at 1M context is difficult and could be much more expensive/numerically tricky. I'm guessing they were working on serving it properly, since its the same "model" in scores and such.
    • simianwords 1y ago
      Thanks - still not clear what they did really. Some inference time hacks?
      • FergusArgyll 1y ago
        That would imply the model always had a 1m token context but they limited it in the api and app? That's strange because they can just charge more for every token past 250k (like google does, I believe). But if not shouldn't it have to be completely retrained model? it's clearly not that - good question!
      • Aeolun 1y ago
        They already had 0.5M context window on the enteprise version.
      • otabdeveloper4 1y ago
        Most likely still 32k tokens under the hood, but with some context slicing/averaging hacks to make inference not error out on infinite input. (That's what I do locally with llama.cpp)
• nickphx 1y ago
  Yay, more room for stray cats.
• chrisweekly 1y ago
  Peer of this post currently also on HN front page, comparing perf for Claude vs Gemini, w/ 1M tokens: https://news.ycombinator.com/item?id=44878999 https://news.ycombinator.com/item?id=44878999
• DiabloD3 1y ago
  Neat. I do 1M tokens context locally, and do it entirely with a single GPU and FOSS software, and have access to a wide range of models of equivalent or better quality. Explain to me, again, how Anthropic's flawed business model works?
  • codazoda 1y ago
    Tell us more?
    • DiabloD3 1y ago
      Nothing really to say, its just like everyone else's inference setups. Select a model that produces good results, has anywhere from 256k to 1M context (ex: Qwen3-Coder can do 1M), is under one of the acceptable open weights licenses, and run it in llama.cpp. llama.cpp can split layers between active and MoE, and only load the active ones into vram, leaving the rest of it available for context. With Qwen3-Coder-30B-A3B, I can use Unsloth's Q4_K_M, consume a mere 784MB of VRAM with the active layers, then consume 27648MB (kv cache) + 3096MB (context) with the kv cache quantized to iq4_nl. This will fit onto a single card with 32GB of VRAM, or slightly spill over on 24GB. Since I don't personally need that much, I'm not pouring entire projects into it (I know people do this, and more data does not produce better results), I bump it down to 512k context and fit it in 16.0GB, to avoid spill over on my 24GB card. In the event I do need the context, I am always free to enable it. I do not see a meaningful performance difference between all on the card and MoE sent to RAM while active is on VRAM, its very much a worthwhile option for home inference. Edit: For completeness sake, 256k context with this configuration is 8.3GB total VRAM, making _very_ budget good inference absolutely possible.
• ffitch 1y ago
  I wonder how modern models fair on NovelQA and FLenQA (benchmarks that test ability to understand long context beyond needle in a haystack retrieval). The only such test on a reasoning model that I found was done on o3-mini-high (https://arxiv.org/abs/2504.21318 https://arxiv.org/abs/2504.21318), it suggests that reasoning noticeably improves FLenQA performance, but this test only explored context up to 3,000 tokens.
• dang 1y ago
  Related ongoing thread: Claude vs. Gemini: Testing on 1M Tokens of Context - https://news.ycombinator.com/item?id=44878999 https://news.ycombinator.com/item?id=44878999 - Aug 2025 (9 comments)
• whalesalad 1y ago
  My first thought was "gg no re" can't wait to see how this changes compaction requirements in claude code.
• pmxi 1y ago
  The reason I initially got interested in Claude was because they were the first to offer a 200K token context window. That was massive in 2023. However, they didn't keep up once Gemini offered a 1M token window last year. I'm glad to see an attempt to return to having a competitive context window.
  • iamnotagenius 1y ago
    they all degrade well before 1M tokens.
• markb139 1y ago
  I’ve tried 2 AI tools recently. Neither could produce the correct code to calculate the CPU temperature on a Raspberry Pi RP2040. The code worked, looked ok and even produced reasonable looking results - until I put a finger on the chip and thus raised the temp. The calculated temperature went down. As an aside the free version of chatGPT didn’t know about anything newer than 2023 so couldn’t tell me about the RP2350
  • anvuong 1y ago
    How can you be sure putting the finger on the chip raise the temp? If you feel hot that means heat from the chip is being transferred to your finger, that may decrease the temp, no?
  • broshtush 1y ago
    From my understanding putting your finger on an uncooled CPU acts like a passive cooler, thus actually decreasing temperature.
  • fwip 1y ago
    I don't think a larger context window would help with that.
    • fpauser 1y ago
      Best comment ;)
  • ghjv 1y ago
    wouldn't your finger have acted as a heat sink, lowering the temp? sounds like the program may have worked correctly. could be worth trying again with a hot enough piece of metal instead of your finger
• logicchains 1y ago
  With that pricing I can't imagine why anyone would use Claude Sonnet through the API when Gemini 2.5 Pro is both better and cheaper (especially at long-context understanding).
  • CuriouslyC 1y ago
    Claude is a good deal with the $20 subscription giving a fair amount of sonnet use with Code. It's also got a very distinct voice as far as LLMs go, and tends to produce cleaner/clearer writing in general. I wouldn't use the API in an application but the subscription feels like a pretty good deal.
• siva7 1y ago
  Ah, so claude code on subscription will become a crippled down version
• joduplessis 1y ago
  As far as coding goes Claude seems to be the most competent right now, I like it. GPT5 is abysmal - I'm not sure if they're bugs, or what, but the new release takes a good few steps back. Gemini still a hit and miss - and Grok seems to be a poor man's Claude (where code is kind of okay, a bit buggy and somehow similar to Claude).
  • wahnfrieden 1y ago
    Are you evaluating gpt5-thinking on high mode, via API or Codex CLI on Pro tier? Just wondering what specifically you compared since those factors affect its performance and context
• brokegrammer 1y ago
  Many people are confused about the usefulness of 1M tokens because LLMs often start to get confused after about 100k. But this is big for Claude 4 because it uses automatic RAG when the context becomes large. With optimized retrieval thanks to RAG, we'll be able to make good use of those 1M tokens.
  • m4r71n 1y ago
    How does this work under the hood? Does it build an in-memory vector database of the input sources and runs queries on top of that data to supplement the context window?
    • brokegrammer 1y ago
      No idea how it's implemented because it's proprietary. Details here: https://support.anthropic.com/en/articles/11473015-retrieval-augmented-generation-rag-for-projects https://support.anthropic.com/en/articles/11473015-retrieval...
    • menaerus 1y ago
      RAG commonly implies some sort of vector database to be built and which will then be used for response augmentation. If it operates over the repo, I believe it will index your codebase using those vector embeddings.
• Balgair 1y ago
  Wow! As a fiction writer/noodler this is amazing. I can put not just a whole book in as before, not just a whole series, but the entire corpus of authors in. I mean, from the pov of biography writers, this is awesome too. Just dump it all in, right? I'll have to switch using to Sonnet 4 now for workflows and edit my RAG code to be longer windows, a lot longer
• irthomasthomas 1y ago
  Brain: Hey, you going to sleep? Me: Yes. Brain: That 200,001st token cost you $600,000/M.
• qwertox 1y ago
  > desperately need LLMs to maintain extremely effective context Last time I used Gemini it did something very surprising: instead of providing readable code, it started to generate pseudo-minified code. Like on CSS class would become one long line of CSS, and one JS function became one long line of JS, with most of the variable names minified, while some remained readable, but short. It did away with all unnecessary spaces. I was asking myself what is happening here, and my only explanation was that maybe Google started training Gemini on minified code, on making Gemini understand and generate it, in order to maximize the value of every token.
• ericol 1y ago
  "...in API" That's a VERY relevant clarification. this DOESN'T apply to web or app users. Basically, if you want a 1M context window you have to specifically pay for it.
• sporkland 1y ago
  Does anyone have data on how much better these 1M token context models produce better results than the more limited windows alongside certain RAG implementations? Or how much better in the face of RAG the 200k vs 1M token models perform on a benchmark?
• poniko 1y ago
  [Claude usage limit reached. Your limit will reset at..] .. eh lunch is a good time to go home anyways..
• chmod775 1y ago
  For some context, only the tweaks files and scripting parts of Cyberpunk 2077 are ~2 million LOC.
• not_that_d 1y ago
  My experience with the current tools so far: 1. It helps to get me going with new languages, frameworks, utilities or full green field stuff. After that I expend a lot of time parsing the code to understand what it wrote that I kind of "trust" it because it is too tedious but "it works". 2. When working with languages or frameworks that I know, I find it makes me unproductive, the amount of time I spend writing a good enough prompt with the correct context is almost the same or more that if I write the stuff myself and to be honest the solution that it gives me works for this specific case but looks like a junior code with pitfalls that are not that obvious unless you have the experience to know it. I used it with Typescript, Kotlin, Java and C++, for different scenarios, like websites, ESPHome components (ESP32), backend APIs, node scripts etc. Botton line: usefull for hobby projects, scripts and to prototypes, but for enterprise level code it is not there.
  • jeremywho 1y ago
    My workflow is to use Claude desktop with the filesystem mcp server. I give claude the full path to a couple of relevant files related to the task at hand, ie where the new code should hook into or where the current problem is. Then I ask it to solve the task. Claude will read the files, determine what should be done and it will edit/add relevant files. There's typically a couple of build errors I will paste back in and have it correct. Current code patterns & style will be maintained in the new code. It's been quite impressive. This has been with Typescript and C#. I don't agree that what it has produced for me is hobby-grade only...
    • taberiand 1y ago
      I've been using it the same way. One approach that's worked well for me is to start a project and first ask it to analyse and make a plan with phases for what needs to be done, save that plan into the project, then get it to do each phase in sequence. Once it completes a phase, have it review the code to confirm if the phase is complete. Each phase of work and review is a new chat. This way helps ensure it works on manageable amounts of code at a time and doesn't overload its context, but also keeps the bigger picture and goal in sight.
• TZubiri 1y ago
  Remember kids, just because you CAN doesn't mean you SHOULD
• mrcwinn 1y ago
  This tells me they've gotten very good at caching and modeling the impact of caching.
• fpauser 1y ago
  O observed that claude produces a lot of bloat. Wonder how such llm generated projects age.
• howinator 1y ago
  I could be wrong, but I think this pricing is the first to admit that cost scales quadratically with number of tokens. It’s the first time I’ve seen nonlinear pricing from an LLM provider which implicitly mirrors the inference scaling laws I think we're all aware of.
  • jpau 1y ago
    Google[1] also has a "long context" pricing structure. OpenAI may be considering offering similar since they do not offer their priority processing SLAs[2] for context >128K. [1] https://cloud.google.com/vertex-ai/generative-ai/pricing https://cloud.google.com/vertex-ai/generative-ai/pricing [2] https://openai.com/api-priority-processing/ https://openai.com/api-priority-processing/
  • energy123 1y ago
    Is this marginal pricing or if you go from 200,000 to 200,001 tokens your total costs double?
• reverseblade2 1y ago
  Does this cover subscription?
  • anonym29 1y ago
    API only for now, but at the very bottom of the post: "We're also exploring how to bring long context to other Claude products." So, not yet, but maybe someday?
• _joel 1y ago
  Fantastic, use up your quota even more quickly. :)
• phyzix5761 1y ago
  What I've found with LLMs is they're basically a better version of Google Search. If I need a quick "How do I do..." or if I need to find a quick answer to something its way more useful than Google and the fact that I can ask follow up questions is amazing. But for any serious deep work it has a long way to go.
  • mr_moon 1y ago
    I feel exactly the same way. why skim and sift 15 different stackoverflow posts when an LLM can pick out exactly the info I need? I don't need to spin up an entire feature in a few seconds. I need help understanding where something is broken; what are some opinions o best practice; or finding out what a poorly written snippet is doing. context still v important for this though and I appreciate cranking that capacity. "read 15000 stackoverflow posts for me please"
    • anvuong 1y ago
      The action of sifting through through poop to find gold actually positively develops my critical thinking skill. I, too, went through a phase of just asking LLM for a specific concept instead of Googling it and weave through dozens of wiki pages or niche mailing list discussions. It did improve my productivity but I feel like it dulls my brain. So recently I have to tone that down and force myself to go back to the old way. Maybe too much of a good thing is bad.
  • Whatarethese 1y ago
    This is my primary use of AI. Looking for a new mountain bike and using AI to list and compare parts of the bike and which is best for my use case scenario. Works pretty well so far.
  • throawaywpg 1y ago
    Google always planned search to be just a stopgap
• deleted 1y ago
  [deleted]
• meander_water 1y ago
  I like to spend a lot of time in "Ask" mode in Cursor. I guess the equivalent in Claude code is "plan" mode. Where I have minimal knowledge about the framework or language, I ask a lot of questions about how the implementation would work, what the tradeoffs are etc. This is to minimize any misunderstanding between me and the tool. Then I ask it to write the implementation plan, and execute it one by one. Cursor lets you have multiple tabs open so I'll have a Ask mode and Agent mode running in parallel. This is a lot slower, and if it was a language/framework I'm familiar with I'm more likely to execute the plan myself.
• itissid 1y ago
  My experience with Claude code beyond building anything bigger than a webpage, a small API, a tutorial on CSS etc has been pretty bad. I think context length is a manageable problem, but not the main one. I used it to write a 50K LoC python code base with 300 unit tests and it went ok for the first few weeks and then it failed. This is after there is a CLAUDE.md file for every single module that needs it as well as detailed agents for testing, design, coding and review. I won't going into a case by case list of its failures, The core of the issue is misaligned incentives, which I want to get into: 1. The incentives for coding agent, in general and claude, are writing LOTS of code. None of them — O — are good at the planning and verification. 2. The involvement of the human, ironically, in a haphazard way in the agent's process. And this has to do with how the problem of coding for these agents is defined. Human developers are like snow flakes when it comes to opinions on software design, there is no way to apply each's preference(except paper machet and superglue SO, Reddit threads and books) to the design of the system in any meaningful way and that makes a simple system way too complex or it makes a complex problem simplistic. - There is no way to evolve the plan to accept new preferences except text in CLAUDE.md file in git that you will have to read through and edit. - There is no way to know the near term effect of code choices now on 1 week from now. - So much code is written that asking a person to review it in case you are at the envelope and pushing the limit feels morally wrong and an insane ask. How many of your Code reviews are instead replaced by 15-30 min design meetings to instead solicit feedback on design of the PR — because it so complex — and just push the PR into dev? WTF am I even doing I wonder. - It does not know how far to explore for better rewards and does not know it better from local rewards, Resulting in commented out tests and deleting arbitrary code, to make its plan "work". In short code is a commodity for CEOs of Coding agent companies and CXOs of your company to use(sales force has everyone coding, but that just raises the floor and its a good thing, it does NOT lower the bar and make people 10x devs). All of them have bought into this idea that 10x is somehow producing 10x code. Your time reviewing and unmangling and mainitaining the code is not the commodity. It never ever was.
• lpa22 1y ago
  One of the most helpful usages of CC so far is when I simply ask: "Are there any bugs in the current diff" It analyzes the changes very thoroughly, often finds very subtle bugs that would cost hours of time/deployments down the line, and points out a bunch of things to think through for correctness.
  • swyx 1y ago
    maybe want to reify that as a claude code hook!
  • bertil 1y ago
    That matches my experience with non-coding tasks: it’s not very creative, but it’s a comprehensive critical reader.
  • neucoas 1y ago
    I am trying this tomorrow
    • lpa22 1y ago
      Let me know how it goes. It’s a game changer
  • KTibow 1y ago
    I'm surprised that works even without telling it to think/think hard/think harder/ultrathink.
  • GuB-42 1y ago
    I added this to my toolbox in addition to traditional linters. My experience is that it is about 10% harmful, 80% useless and 10% helpful. Which is actually great, the 10% is worth it, but it is far from a hands off experience. By harmful I mean something like suggesting a wrong fix to code that works, it usually happens when I am doing something unusual or counter intuitive, for example having a function "decrease_x" that (correctly) adds 1 to x. It may hint for better documentation, but you have to be careful not to go on autopilot and just do what it says. By useless I mean something like "you didn't check for null" even though the variable can't be null or is passed to a function that handles the "null" case gracefully. In general, it tends to be overly defensive and following the recommendations would lead to bloated code. By helpful I mean finding a real bug. Most of them minor, but for some, I am glad I did that check. LLMs complement traditional linters well, but they don't replace them.
• i_have_an_idea 1y ago
  While this is cool, can anything be done about the speed of inference? At least for my use, 200K context is fine, but I’d like to see a lot faster task completion. I feel like more people would be OK with the smaller context if the agent acts quickly (vs waiting 2-3 mins per prompt).
  • jeffhuys 1y ago
    There’s work being done in this field - I saw a demo using the same method as stable diffusion does, but then for text. Was extremely fast (3 pages of text in like a second). It’ll come.
  • wahnfrieden 1y ago
    Meanwhile the key is to become proficient at using worktrees to parallelize agents instead of working serially with them
    • i_have_an_idea 1y ago
      Sounds nice, in theory, but in practice I want to iterate on one, perhaps, two tasks at a time, and keep a good understanding of what the agent is doing, so that I can prevent it from going off the rails, making bad decisions and then building on them even further. Worktrees and parallel agents do nothing to help me with that. It's just additional cognitive load.
• maxnevermind 1y ago
  Does very large context significantly increase a response time? Are there any benchmarks/leader-boards estimating different models in that regard?
• hoppp 1y ago
  So I can upload 1M tokens per prompt but pay $3 per 1M input tokens? Its really expensive to use.
  • Aeolun 1y ago
    Only the first time. After that it’s 0.3 per 1M input tokens (cached).
• psyclobe 1y ago
  Isn’t Opus better? Whenever I run out of Opus tokens and get kicked down to Sonnet it’s quite a shock sometimes. But man I’m at the perfect stage in my career for these tools. I know a lot, I understand a lot, I have a lot of great ideas-but I’m getting kinda tired of hammering out code all day long. Now with Claude I am just busting ass executing in all these ideas and tests and fixes-never going back!
  • Aeolun 1y ago
    Haha, I think I recognize that. I’m just worried my actual skills will athrophy while I use Claude Code like I’m a manager on steroids.
    • stavros 1y ago
      By definition the thing that atrophies is the thing you never need to use.
• as367 1y ago
  That is an unfortunate logo.
  • wiseowise 1y ago
    This is something that I wish I would unremember.
• tomsanbear 1y ago
  I just want a better way to invalidate old context... It's great that I can fit more context, but the main challenge is claude getting sidetracked with 10 invalid grep calls, pytest dumping a 10k token stack trace etc.... And yes the ability to go back in time via esc+esc is great but I want claude to read the error stack learn from it and purge from its context or at least let me lobotomize ot selectively... Learning and discarding the raw output from tool calls feels like the missing piece here still.
  • typpilol 1y ago
    Vscode recently rolled out checkpoints where you can go back to a previous state of the conversation. But it's still not enough. We honestly need to be able to see exactly what's in the context and be able to manually edit it.
• aledalgrande 1y ago
  I hope that they are going to put something in Claude Code to display if you're entering the expensive window. Sometime I just keep the conversation going. I wouldn't want that to burn my Max credits 2x faster.
  • terminalshort 1y ago
    Yeah, that 1 MM tokens is a $15 (IIRC) API call. That's gonna add up quick! My favorite hypothetical AI failure scenario is that LLM agents eventually achieve human level general intelligence, but have to burn so many tokens to do it that they actually become more expensive than a human.
    • k9294 1y ago
      I believe Claude Code uses cache aggressively, so this 1kk tokens will be 90% discounted or do I miss something?
• socrateslee 1y ago
  It's like double "double the dose"
• forgingahead 1y ago
  Wish it was on the web app as well!
• williamtrask 1y ago
  Claude is down. EDIT: for the moment... it supports 0 tokens of context xD
• nojs 1y ago
  Currently the quality seems to degrade long before the context limit is reached, as the context becomes “polluted”. Should we expect the higher limit to also increase the practical context size proportionally?
• m13rar 1y ago
  This is amazing. shout out to anthropic for doing this. I would like to have a CLAUDE Model which is not nerfed with ethics and values to please the users and write overtly large plans to impress the user.
  • elcritch 1y ago
    I'm finding GPT5 to be more succinct and on par with Claude Code so far. They're really toned down the obsequiousness.
• truelson 1y ago
  We do know Parkinson’s Law ( https://en.m.wikipedia.org/wiki/Parkinson%27s_law https://en.m.wikipedia.org/wiki/Parkinson%27s_law ) will apply to all this, right?
• simon_rider 1y ago
  feels like we just traded "not enough context" for "too much noise." The million-token window is cool for marketing, but until retrieval and summarization get way better, it’s like dumping the entire repo on a junior dev’s desk and saying "figure it out." They’ll spend half their time paging through irrelevant crap, and the other half hallucinating connections. Bigger context is only a net win if the model can filter, prioritize, and actually reason over it
  • whalesalad 1y ago
    I can't tell you the number of times I had almost reached utopia only to hit compaction limits. Post-compaction I am usually dead in the water and the spiraling or repetition begins. Claude has a hard time compacting/remembering/flaggign a-ha moments from the session. Stuff that is important in the context of the task, but not appropriate for CLAUDE.md for instance. I have been thinking for months that if the context window was 2-3x larger, I would be unstoppable. So happy for this change, and excited to test it this week.
• MagicMoonlight 1y ago
  It’s a stupid metric because nothing in the real world has half a million words of context. So all they’re doing is feeding it imagined slop, or sticking together random files.
  • zaptrem 1y ago
    It's useful for hours-long long-context debugging sessions in Claude Code, etc.
• cintusshied 1y ago
  For folks using LLMs for big coding projects, what's your go-to workflow for deciding which parts of the codebase to feed the model?
  • aitchnyu 1y ago
    Aider automatically makes an outline of your whole codebase, which takes fraction of the tokens of the real files. https://aider.chat/docs/repomap.html https://aider.chat/docs/repomap.html
• mrcwinn 1y ago
  I wish they’d fix other things faster. Still can’t upload an Excel file in the iOS app, even with analyst mode enabled. The voice mode feels like it’s 10 years behind OpenAI (no realtime, for example). And Opus 4.1 still occasionally goes absolutely mental and provides much worse analysis than o3 or GPT5-thinking. Rooting for Anthropic. Competition in this space is good. I watched an interview with Dario recently where he said he wasted a “product guy” and it really shows.
• cognix_dev 1y ago
  To be honest, I am not particularly interested in whether the next model is better than the previous one. Rather than being superior, it is important that it maintains context and has enough memory capacity to not interfere with work. I believe that is what makes the new model competitive.
• CodeCompost 1y ago
  In Visual Studio as well?
• omlelelom_kimox 1y ago
  HM
• throwmeaway222 1y ago
  Why do I get the feeling that HN devs on here want to just feed it their entire folder, source, binaries everything and have it make changes in seconds.
• Roark66 1y ago
  I noticed the quality of answer degrades horribly beyond few thousands of tokens. Maybe 10k. Is anyone actually successfully using these 100k+ token contexts for anything?
• nprateem 1y ago
  Anyone else found Claude has become hugely more stupid recently? It used to always pitch answers at the right level, but recently it just seems to have left its common sense at the door. Gemini just gives much better answers for non-technical questions now.
• amelius 1y ago
  1M? 640K ought to be enough for anybody ... right?
• doppelgunner 1y ago
  Great! Now we can use AI to read and think like a specific "book".
• hassadyb 1y ago
  i personally use it in my codding tasks such ana amazing and powerful llm
• muzani 1y ago
  Of course Bolt is the customer spotlight. These vibe coding tools chuck the entire codebase and charge for tokens used. By 10k lines of code or so, these apps were not able to fit.
• ndkap 1y ago
  Does anybody know which technology most of these companies that support 1M tokens use? Or is it all hidden?
• t43562 1y ago
  I think this highlights some problems with software development in general. i.e. the code isn't enough - you need to have domain knowledge too and a lot of knowledge about how and why the company needs things done in some way or another. You might imagine that dumping the contents of your wiki and all your chat channels into some sort of context might do it but that would miss the 100s of verbal conversations between people in the company. It would also fall foul of the way everything tends to work in any way you can imagine except what the wiki says. Even if you transcribed all the voice chats and meetings and added it in, it challenges a human to work out what is going on. No-context human developers are pretty useless too.
• nothercastle 1y ago
  How much will it cost when you get near the 1m context mark? Its got to be in thousands per query