23 ms·
Claude 3.7 Sonnet and Claude Code
- deleted 2y ago[deleted]
- bnc319 2y agoPretty amazing how DeepSeek started the visual reasoning trend, xAI featured it in their latest release, and now Anthropic does the same.
- anjel 2y agoI took DS visual reasoning to be an elegant misdirect from how much slower DS returns your query's output.
- TechDebtDevin 2y agoYou almost have to do this, or atleast some sort of progress bar, else people will think their requests failed and spam the server.
- nomel 2y agoI thought my internet cut out the first time I used o1.
- t55 2y agoAnthropic doubling down on code makes sense, that has been their strong suit compared to all other models Curious how their Devin competitor will pan out given Devin's challenges
- malux85 2y agoI thought the same thing, I have 3 really hard problems that Claude (or any model) hasn’t been able to solve so far and I’m really excited to try them today
- ru552 2y agoConsidering that they are the model that powers a majority of Cursor/Windsurf usage and their play with MCP, I think they just have to figure out the UX and they'll be fine.
- weinzierl 2y agoIt's their strong suit no doubt, but sometimes I wish the chat would not be so eager to code. It often throws code at me when I just want a conceptual or high level answer. So often that I routinely tell it not to.
- KerryJones 2y agoI complain about this all the time, despite me saying "ask me questions before you code" or all these other instructions to code less, it is SO eager to code. I am hoping their 3.7 reasoning follows these instructions better
- vessenes 2y agoWe should remember 3.5 was trained in an era when ChatGPT would routinely refuse to code at all and architected in an era when system prompts were not necessarily very effective. I bet this will improve, especially now that Claude has its own coding and arch cli tool.
- NitpickLawyer 2y ago> I just want a conceptual or high level answer I've found claude to be very receptive to precise instructions. If I ask for "let's first discuss the architecture" it never produces code. Aider also has this feature with /architect
- ap-hyperbole 2y agoI added custom instruction under my Profile settings in the "personal preferences" text box. Something along the lines of "I like to discuss things before wanting the code. Only generate code when I prompt for it. Any question should be answered to as a discussion first and only when prompted should the implementation code be provided". It works well, occasionally I want to see the code straight away but this does not happen as often.
- perdomon 2y agoI get this as well, to the point where I created a specific project for brainstorming without code — asking for concepts, patterns, architectural ideas without any code samples. One issue I find is that sometimes I get better answers without using projects, but I’m not sure if that’s everyone experience.
- KaoruAoiShiho 2y agoThey cited Cognition (Devin's maker) in this blog post which is kinda funny.
- Flux159 2y agoIt's interesting that Anthropic is making their own coding agent with Claude Code - is this a sign of them looking to move up the stack and more into verticals that model wrapper startups are in?
- madduci 2y agoGitHub copilot has now introduced Claude as model as well
- vessenes 2y agoThis makes sense to me: sell razor blades. Presumably Claude has a large developer distribution channel so they will keep eyeballing what to ‘give away’ that turns the dials on inference billing. I’d guess this will keep raising the bar for paid or open source competitors, so probably good for end users esp given they aren’t a monopoly by any means.
- estsauver 2y agoThe docs for Claude code don't seem to be up yet but are linked here: http://docs.anthropic.com/s/claude-code http://docs.anthropic.com/s/claude-code I'm not sure if it's a broken link in the blog post or just hasn't been published yet.
- jumploops 2y agoSaw the same thing, but looks to be up now!
- tablet 2y agoThe progress in AI area is insane. I can't keep up with all the news. And I have work to do...
- amelius 2y agoIt stopped being revolutionary and is now mostly evolutionary, though.
- szundi 2y ago[dead]
- dingnuts 2y agoit's been evolutionary for a long time. I fine-tuned a GPT-2 based chat bot that could form complete sentences back in like 2017 It's been so long that I'm not even certain which YEAR I set that up.
- falcor84 2y agoWhere do you draw the line? If going from forming sentences to achieving medal level success on IMO questions, doing extensive web research on its own and writing entire SaaS apps based on a prompt in under 10 years is just "evolutionary", then it's one heck of an evolution.
- nomel 2y agoIt's always been the case that people in to tech see a smooth slope rather than some sort of discontinuity, like you might perceive if you stepped back a bit. That's why you can go laugh at "thing makes a billion dollars even though nerds say it's obvious and incremental" type posts going back 25 years. iPhone is a great one.
- famouswaffles 2y ago>I fine-tuned a GPT-2 based chat bot that could form complete sentences back in like 2017. GPT-2 was a 2019 release lol.
- TIPSIO 2y ago"Make me a website about books. Make it look like a designer and agency made it. Use Tailwind." https://play.tailwindcss.com/tp54wfmIlN https://play.tailwindcss.com/tp54wfmIlN Getting way better at UI.
- deleted 2y ago[deleted]
- flir 2y agoThat's not hideous. Derivative, but that's the nature of the beast.
- jasonjmcghee 2y agoI feel like something isn't working... when i try to click anything it just reloads. i can't see the collections
- handfuloflight 2y agoAs a designer and agency... this is extremely basic... but so was the prompt.
- punkpeye 2y agoI cannot believe that others just casually dismiss this as 'basic', when just a few years ago this would have taken someone a full day of work.
- djeastm 2y agoI mean, it is basic. Templates have been around for decades and this looks like a template from 2007 that someone filled in with their own copy. That might take like an hour, maybe? And presumably the person who wants the page done will have to customize this text, too.
- lysace 2y agoIt's fascinating how close these companies are to each other. Some company comes up with something clever/ground-breaking and everyone else has implemented it a few weeks later. Hard not to think of Kurzweil's Law of Accelerating Returns.
- mechagodzilla 2y agoIt does seem like it will be very, very hard for the companies training their own models to recoup their investment when the capabilities of open-weight models catch up so quickly - general purpose LLMs just seem destined to be a cheap commodity.
- szundi 2y ago[dead]
- jsheard 2y agoWell, the companies releasing open weights also need to recoup their investments at some point, they can't coast on VC hype forever. Huge models don't grow on trees.
- mechagodzilla 2y agoOr, like Meta, they make their money elsewhere and just seem interested in wrecking the economics of LLMs. As soon as an open-weight model is released, it basically sets a global floor that says "Models with similar or worse performance effectively have zero value," and that floor has been rising incredibly quickly. I'd be surprised if the vast, vast majority of queries ChatGPT gets couldn't get equivalently good results from llama3/deepseek/qwen/mistral models, even for those paying for the pro versions.
- Philpax 2y agoEh, to some extent - there's still a pretty significant cost to actually running inference for those models. For example, no consumer can run DeepSeek v3/r1 - that requires tens, possibly hundreds, of thousands of dollars of hardware to run. There's still room for other models, especially if they have different performance characteristics that make them suitable to run under consumer constraints. Mistral has been doing quite well here.
- ctoth 2y agoI've been using O3-mini with reasoning effort set to high in Aider and loving the pricing. This looks as though it'll be about three times as expensive. Curious to see which falls out as most useful for what over the next month!
- rahimnathwani 2y agoAro using o3-mini for editing or just architect in architect-editor mode?
- vessenes 2y agoIt is .. not a great architect. I have high hopes for 3.7 though - even 3.5 architect matched with 3.5 coding is generally better than 3.5 coding alone.
- rs_rs_rs_rs_rs 2y agoHope it's worth the money because it's quite expensive.
- kruxigt 2y ago[dead]
- wetpaws 2y ago[dead]
- m3kw9 2y agoWonder if Aider will copy some of these features
- sergiotapia 2y agoAlready available in Cursor! https://x.com/cursor_ai/status/1894093436896129425 https://x.com/cursor_ai/status/1894093436896129425 (although I do not see it)
- jedberg 2y agoLast week when Grok launched the consensus was that its coding ability was better than Claude. Anyone have a benchmark with this new model? Or just warm feelings?
- esafak 2y agoThey merely claimed that. I have not seen many people confirm that it is the best, let alone a consensus. I don't believe it is even available through an API yet.
- minihat 2y agoGrok 3 with thinking is comparable to o1 for writing complex algorithms. However, Grok sometimes loses the context where o1 seems not to. For this reason I still mostly use o1. I have found both o1 and Grok 3 to be substantially better than any Claude offering.
- ianhawes 2y ago> Include the beta header output-128k-2025-02-19 in your API request to increase the maximum output token length to 128k tokens for Claude 3.7 Sonnet. This is pretty big! Previously most models could accept massive input tokens but would be restricted to 4096 or 8192 output tokens.
- thegeomaster 2y agoThis amounts to a cost-saving measure - you can generate arbitrarily many tokens by appending the output and re-invoking the model.
- ungreased0675 2y agoAwesome. Claude is significantly better than other models at code assistant tasks, or at least in the way I use it.
- jasondigitized 2y agoTotally agree. I continue to be blown away at how good it is at understanding, explaining, and writing code. Got an obscure error? Give Claude enough context and it is pretty dang good and getting you on glide slope.
- TekMol 2y ago[flagged]
- jonas21 2y ago> Why would my phone number be any of their business? Preventing abuse? It's much harder to create a throwaway phone number than a throwaway email address. > OpenAI does the logical thing. Let's me enter my credit card and I'm good to go. I will stay with them. You'd rather hand over your credit card than your phone number? I think most people would see it the other way around.
- rcstank 2y agoCredit card is easily changed. Phone number is much more difficult.
- whywhywhywhy 2y agoNot difficult at all for anyone actually wanting to abuse it
- behnamoh 2y agoYou shouldn't have to want to abuse something for it to be inappropriate to be asked.
- whywhywhywhy 2y agoI agree just debunking the excuse that's always wheeled out for this overreach.
- crazygringo 2y agoYou credit card is also easily charged. Your phone number isn't. What is a company going to do with your phone number that you're worried about...?
- brendoelfrendo 2y ago
- bbor 2y agoJust as humans use a single brain for both quick responses and deep reflection, we believe reasoning should be an integrated capability of frontier models rather than a separate model entirely. Interesting. I've been working on exactly this for a bit over two years, and I wasn't surprised to see UAI finally getting traction from the biggest companies -- but how deep do they really take it...? I've taken this philosophy as an impetus to build an integrated system of interdependent hierarchical modules, much like Minsky's Society of Mind that's been popular in AI for decades. But this (short, blog) post reads like it's more of a behavioral goal than a design paradigm. Anyone happen to have insight on the details here? Or, even better, anyone from Anthropic lurking in these comments that cares to give us some hints? I promise, I'm not a competitor! Separately, the throwaway paragraph on alignment is worrying as hell, but that's nothing new. I maintain hope that Anthropic is keeping to their founding principles in private, and tracking more serious concerns than "unnecessary refusals" and prompt injection...
- Alex-Programs 2y agoIIRC there's some reasoning in old Sonnet too, they're just expanding that. Perhaps that's part of why it was so good for a while. https://www.reddit.com/r/ClaudeAI/comments/1iv356t/is_sonnet_35_actually_a_pseudoreasoning_model/ https://www.reddit.com/r/ClaudeAI/comments/1iv356t/is_sonnet...
- isoprophlex 2y agoYES. I've tried them all but Sonnet is still the model I'm most productive with, even better than the o1/o3 models. Wish I could find the link to enroll in their Claude Code beta...
- frankfrank13 2y agohere -- https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
- isoprophlex 2y agoThanks!
- waltercool 2y agoJust like OpenAI or Grok, there is no transparency and no way for self-hosting purposes. Your input and confidential information can be collected for training purposes. I just don't trust those companies when you use their servers. This is not a good approach to LLM democratization.
- azinman2 2y agoI wouldn’t assume there’s no way to self host — it just costs a lot more than open weights. Anthropic claims they don’t train on their inputs. I haven’t seen any reason to disbelieve them.
- waltercool 2y agoBut there is no way to know if their claims are true either. Your inputs are processed into their servers, then you get a response. Whatever happens in the middle, only Anthropic knows. We don't even know of governments are actually pushing AI companies to enforce censorship or spying people, like we seen recently at UK government getting into Apple E2E encryption. This criticism is valid for the business who wants to use AI to improve coding, code analysis or code review, documentation, emails, etc, but also for that individual who don't want to rely on 3rd party companies for AI usage.
- simonw 2y agoYou can sign a contract with Anthropic that fully bakes their promise not to train on your input. You can also access Claude via both AWS Bedrock and Google Vertex, both of which come with very robust guarantees about how your data is used.
- deleted 2y ago[deleted]
- wewewedxfgdf 2y agoNothing in the Claude API release notes. https://docs.anthropic.com/en/release-notes/api https://docs.anthropic.com/en/release-notes/api I really wish Claude would get Projects and Files built into its API, not just the consumer UI.
- thanhhaimai 2y ago> Third, in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs. Company: we find that optimizing for LeetCode level programming is not a good use of resources, and we should be training AI less on competition problems. Also Company: we hire SWEs based on how much time they trained themselves on LeetCode /joke of course
- Svoka 2y agoMy manager explained to me that LeetCode is proving that you are willing to dance the dance. Same as PhD requirements etc - you probably won't be doing anything related and definitely nothing related to LeetCode, but you display dedication and ability. I kinda agree that this is probably reason why companies are doing it. I don't like it, but this is besides the matter. Using Claude other models in interviews probably won't be allowed any time soon, but I do use it the work. So it does make sense.
- deleted 2y ago[deleted]
- nico 2y agoAnd it's also the reality of hiring practices for most VC-backed and public companies Some try to do something more like "real-world" tasks, but those end up either being either just toy problems, or long take homes Personally, I feel the most important things to prioritize when hiring are: is the candidate going to get along with their teammates (colleagues, boss, etc), and do they have the basic skills to relatively quickly learn their jobs once they start?
- EliasWatson 2y agoI asked it for a self-portrait as a joke and the result is actually pretty impressive. Prompt: "Draw a SVG self-portrait" https://claude.site/artifacts/b10ef00f-87f6-4ce7-bc32-80b3ee1c8bd5 https://claude.site/artifacts/b10ef00f-87f6-4ce7-bc32-80b3ee... For comparison, this is Sonnet 3.5's attempt: https://claude.site/artifacts/b3a93ba6-9e16-4293-8ad7-398a5e1c547b https://claude.site/artifacts/b3a93ba6-9e16-4293-8ad7-398a5e...
- orangesun 2y agoNew mascot! Just make it the Anthropic orange
- deleted 2y ago[deleted]
- punkpeye 2y agoI kinda get how LLMs work with language, but it beyond blows me my mind trying to understand how an LLM can draw SVG. There are just so many dimensions to understanding how SVG converts to an image. Even as a human I don't think I could do anywhere close to that result in first attempt.
- frankfrank13 2y agoTried claude code, and have an empty unresponsive terminal. Looks cool in the demo though, but not sure this is going to perform better than Cursor, and shipping this as an interactive CLI instead of an extension is... a choice
- toddmorey 2y agoI think it's a smart starting point as it's compatible with all IDEs. Iterate and learn and then later wrap the functionality up into IDE plugins.
- apsec112 2y agoThey don't say this, but from querying it, they also seem to have updated the knowledge cutoff from April 2024 ("3.6") to October 2024 (3.7)
- KerryJones 2y agoThanks for noting this -- it's actually pretty important in my work.
- sunaookami 2y agoIt's in the Model Card: https://assets.anthropic.com/m/785e231869ea8b3b/original/claude-3-7-sonnet-system-card.pdf https://assets.anthropic.com/m/785e231869ea8b3b/original/cla... >Claude 3.7 Sonnet is trained on a proprietary mix of publicly available information on the Internet as of November 2024
- paradite 2y agoIt's there in the docs (Model comparison table) https://docs.anthropic.com/en/docs/about-claude/models/all-models https://docs.anthropic.com/en/docs/about-claude/models/all-m...
- rahimnathwani 2y agoI'm curious how Claude Code compares to Aider. It seems like they have a similar user experience.
- azinman2 2y agoTo me the biggest surprise was seeking grok dominate in all of their published benchmarks. I haven’t seen any benchmarks of it yet (which I take with a giant heap of salt), but it’s still interesting nevertheless. I’m rooting for Anthropic.
- pertymcpert 2y agoIndeed. I wonder what the architecture for Claude and Grok3 is. If they're still dense models was the MoE excitement with R1 was a tad premature...
- phillipcarter 2y agoNeither a statement for or against Grok or Anthropic: I've now just taken to seeing benchmarks as pretty lines or bars on a chart that are in no way reflective of actual ability for my use cases. Claude has consistently scored lower on some benchmarks for me, but when I use it in a real-world codebase, it's consistently been the only one that doesn't veer off course or "feel wrong". The others do. I can't quantify it, but that's how it goes.
- vessenes 2y agoO1 pro is excellent at figuring out complex stuff that Claude misses. It’s my go to mid level debug assistant when Claude spins
- maeil 2y agoIve found the same but find o3-mini just as good as that. Sonnet is far better as a general model, but when it's an open-ended technical question that isn't just about code, o3-mini figures it out while Sonnet sometimes doesn't. In those cases o3 is less inclined to go with purely the most "obvious" answer when it's wrong.
- OsrsNeedsf2P 2y agoI have never, in frontend, backend, or Android, had O1 pro solve a problem Claude 3.5 could not. I've probably tried it close to 20 times now as well
- cyounkins 2y agoI don't yet see it in Bedrock in us-east-1 or us-east-2
- punkpeye 2y agoIf you are open to alternatives https://glama.ai/models/claude-3-7-sonnet-20250219 https://glama.ai/models/claude-3-7-sonnet-20250219
- elliot07 2y agoThe cost is absurd (compared to other LLM providers these days). I asked 3 questions and the cost was ~0.77c. I do like how this is implemented as a bash tool and not an editor replacement though. Never leaving Vim! :P
- koakuma-chan 2y agoYep, my experience as well. It's just not worth it.
- koakuma-chan 2y agoIt burns through tokens like crazy on a small code base https://i.imgur.com/16GCxiy.png https://i.imgur.com/16GCxiy.png
- nomel 2y agoThat 0.77 can save hours of work though, fighting with or being misdirected by other LLM. And, relative to hourly rate, or a cup of coffee, it's incredibly insignificant, if just used for the heavy questions. My LLM client can switch between whatever models, mid conversation. So I'll have a question or two in the more expensive, then drop down to the cheaper for explanations/questions that help me understand. Rewind time, then hit the more expensive models with relevant prompts. At the edges, it really ends up being "this is the only model that can do this".
- elliot07 2y agoAfter playing with it for a few days, I agree the cost is easily worth the $$. It's almost in a different playing field of "normal" LLM usage.. and more so in the "good productivity tools" realm.
- photon_collider 2y agoNice to see a new release from Anthropic. Yet, this only makes me even more curious of when we'll see a new Claude Opus model.
- bakugo 2y agoFunny enough, 3.7 Sonnet seems to think it's Opus right now: > "thinking": "I am Claude, an AI assistant created by Anthropic. I believe the specific model is Claude 3 Opus, which is Anthropic's most capable model at the time of my training. However, I should simply identify myself as Claude and not mention the specific model version unless explicitly asked for that level of detail."
- Alex-Programs 2y agoI doubt we will. The state of the art seem to have moved away from the GPT-4 style giant and slow models to smaller, more refined ones - though Groq might be a bit of a return to the "old ways"? Personally I'm hoping they update Haiku at some point. It's not quite good enough for translation at the moment, while Sonnet is pretty great and has OK latency (https://nuenki.app/blog/llm_translation_comparison https://nuenki.app/blog/llm_translation_comparison)
- modeless 2y agoI updated Cursor to the latest 0.46.3 and manually added "claude-3.7-sonnet" to the model list and it appears to work already. "claude-3.7-sonnet-thinking" works as well. Apparently controls for thinking time will come soon: https://x.com/sualehasif996/status/1894094715479548273 https://x.com/sualehasif996/status/1894094715479548273
- Hadriel 2y agoso do you think its a better experience with Cursor using 3.7 or just the 3.7 terminal experience?
- deleted 2y ago[deleted]
- punkpeye 2y agohttps://glama.ai/models/claude-3-7-sonnet-20250219 https://glama.ai/models/claude-3-7-sonnet-20250219 Will be interesting to see how this gets adopted in communities like Roo/Cline, which currently account for the most token usage among Glama gateway user base.
- bcherny 2y agoHi everyone! Boris from the Claude Code team here. @eschluntz, @catherinewu, @wolffiex, @bdr and I will be around for the next hour or so and we'll do our best to answer your questions about the product.
- deleted 2y ago[deleted]
- frankfrank13 2y agoCongrats on the launch! You said its an important tool for you (Claude Code) how does this fit in with Co-Pilot, Cursor, etc. Do you/your teammates only rely on Claude Code? What do you reach for for different tasks?
- bcherny 2y agoClaude Code is super popular internally at Anthropic. Most engineers like to use it together with an IDE like Cursor, Windsurf, VS Code, Zed, Xcode, etc. Personally I usually start most coding tasks in Code, then move to an IDE for finishing touches.
- 420gunna 2y agoAre you guys paying Claude for its assistance with your products
- pookieinc 2y agoThe biggest complaint I (and several others) have is that we continuously hit the limit via the UI after even just a few intensive queries. Of course, we can use the console API, but then we lose ability to have things like Projects, etc. Do you foresee these limitations increasing anytime soon? Quick Edit: Just wanted to also say thank you for all your hard work, Claude has been phenomenal.
- eschluntz 2y agoWe are definitely aware of this (and working on it for the web UI), and that's why Claude Code goes directly through the API!
- TriangleEdge 2y agoThis AI race is happening so fast. Seems like it to me anyway. As a software developer/engineer I am worried about my job prospects.. time will tell. I am wondering what will happen to the west coast housing bubbles once software engineers lose their high price tags. I guess the next wave of knowledge workers will move in and take their place?
- ilrwbwrkhv 2y ago[flagged]
- gchokov 2y agoThis is BS and you are not listening and watching carefully.
- dingnuts 2y agoOK then show me a model that can answer honestly and correctly about whether or not it knows something.
- medvezhenok 2y agoShow me a human that can answer honestly and correctly about whether they know something.
- croes 2y agohttps://news.ycombinator.com/item?id=43155825 https://news.ycombinator.com/item?id=43155825
- lukaslalinsky 2y agoEven the best LLMs today are just junior devs with a lot of knowledge. They make a lot of the same mistakes junior devs would do. Even the responses, when you point out those mistakes, are the same. If anything, it's a tool for junior devs to get better and spend more time on the architecture. Using AI code without fully understanding it (ie operated by a non-programmer) is just recipe for disaster.
- shortrounddev2 2y agoDoes claude have a vscode plugin yet? I dropped github copilot because I didnt want so many subscriptions
- dugmartin 2y agoYou can use the Roo Code extension and point it most any api, including Anthropic: https://marketplace.visualstudio.com/items?itemName=RooVeterinaryInc.roo-cline https://marketplace.visualstudio.com/items?itemName=RooVeter...
- punkpeye 2y agoBig fan of Roo. Highly recommend them. Easy to setup and easy to use. I am keen to see if Roo integrates the new Anthropic's coding assistant or if they become competing offerings.
- visarga 2y agoUse Windsurf, a VSCode fork, it defaults on Claude as LLM.
- wolffiex 2y agoTry running Claude Code in your VS Code terminal! Just don't paste too much text :) https://stackoverflow.com/questions/41714897/character-line-limit-when-pasting-in-vscode-terminal https://stackoverflow.com/questions/41714897/character-line-...
- jumploops 2y ago> "[..] in developing our reasoning models, we’ve optimized somewhat less for math and computer science competition problems, and instead shifted focus towards real-world tasks that better reflect how businesses actually use LLMs.” This is good news. OpenAI seems to be aiming towards "the smartest model," but in practice, LLMs are used primarily as learning aids, data transformers, and code writers. Balancing "intelligence" with "get shit done" seems to be the sweet spot, and afaict one of the reasons the current crop of developer tools (Cursor, Windsurf, etc.) prefer Claude 3.5 Sonnet over 4o.
- bicx 2y agoClaude 3.5 has been fantastic in Windsurf. However, it does cost credits. DeepSeek V3 is now available in Windsurf at zero credit cost, which was a major shift for the company. Great to have variable options either way. I’d highly recommend anyone check out Windsurf’s Cascade feature for agentic-like code writing and exploration. It helped save me many hours in understanding new codebases and tracing data flows.
- ai-christianson 2y agoI'm working on an OSS agent called RA.Aid and 3.7 is anecdotally a huge improvement. About to push a new release that makes it the default. It costs money but if you're writing code to make money, it's totally worth it.
- throwup238 2y agoDeepSeek’s models are vastly overhyped (FWIW I have access to them via Kagi, Windsurf, and Cursor - I regularly run the same tests on all three). I don’t think it matters that V3 is free when even R1 with its extra compute budget is inferior to Claude 3.5 by a large margin - at least in my experience in both bog standard React/Svelte frontend code and more complex C++/Qt components. After only half an hour of using Claude 3.7, I find the code output is superior and the thinking output is in a completely different universe (YMMV and caveat emptor). For example, DeepSeek’s models almost always smash together C++ headers and code files even with Qt, which is an absolutely egregious error due to the meta-object compiler preprocessor step. The MOC has been around for at least 15 years and is all over the training data so there’s no excuse.
- d_watt 2y agoI'm about 50kloc into a project making a react native app / golang backend for recipes with grocery lists, collaborative editing, household sharing, so a complex data model and runtime. Purely from the experiment of "what's it like to build with AI, no lines of code directly written, just directing the AI." As I go through features, I'm comparing a matrix of Cursor, Cline, and Roo, with the various models. While I'm still working on the final product, there's no doubt to me that Sonnet is the only model that works with these tools well enough to be Agentic (rather than single file work). I'm really excited to now compare this 3.7 release and how good it is at avoiding some of the traps 3.5 can fall into.
- thebigspacefuck 2y agoThis has been my experience as well. Why do the others suck so bad?
- d_watt 2y agoI wonder how much it's self fulfilling, where the developers of the agents are tuning their prompts / tool calls to sonnet.
- wokwokwok 2y ago"no lines of code directly written, just directing the AI" /skeptical face. Without fail, every. single. person. I've met who says that, actually means "except for the code that I write", or "except for how I link the code it build together by hand". If you are 50kloc in to a large complex project that you have literally written none of, and have, eg. used cursor to generate the code without any assistance... well, you should start a startup. ...because, that's what devin was supposed to be, and it was enormously and famously terrible at it. So that would be either a) terribly exciting, or b) hyperbole.
- M4v3R 2y agoI’m currently doing something very similar to what GP is doing - I’m building a hobby project that’s a desktop app with web frontend. It’s a map editor with a 3D view. My estimate is that 80-90% of the code was written by AI. Sure, I did have to intervene or write some more complex parts myself but it’s still exciting to me that in many cases it took just a single prompt to add a new feature to it or change existing behavior. Judging from the complexity of the project it would take me in the past 4-5x longer if I were to write it completely by hand. It’s a game changer for me.
- ndm000 2y agoHave there been any updates to Claude 3.5 Sonnet pricing? I can't find that anywhere even though Claude 3.7 Sonnet is now at the same price point. I could use 3.5 for a lot more if it's cheaper.
- minimaxir 2y agoNo changes to Claude 3.5 Sonnet pricing despite the new model. https://www.anthropic.com/pricing#anthropic-api https://www.anthropic.com/pricing#anthropic-api
- ramesh31 2y agoWell there goes my evening
- anonzzzies 2y agoWe have used claude almost exclusively since 3.5 ; we regularly run our internal benchmark (coding) against others, but it's mostly just a waste of time and money. Will be testing 3.7 the coming days to see how it stacks up!
- hubraumhugo 2y agoYou can get your HN profile analyzed by it and it's pretty funny :) https://hn-wrapped.kadoa.com/ https://hn-wrapped.kadoa.com/ I'm using this to test the humor of new models.
- aprilthird2021 2y agoIt's very funny! But it's also clear that it only used a small subset of my comments to generate everything. Still thanks for sharing!
- deleted 2y ago[deleted]
- Philpax 2y agoSeems broken? Getting > An error occurred in the Server Components render. The specific message is omitted in production builds to avoid leaking sensitive details. A digest property is included on this error instance which may provide additional details about the nature of the error.
- ANewFormation 2y agoI did multiple accounts with no problem, but in trying to do you I got the same error. You've broke the system.
- Philpax 2y agoNew benchmark for good posting, I'll take it!
- ghxst 2y agoWorked for me, seems to be case sensitive (?) I'll post these incase I just got lucky and it still doesn't work for you. https://hn-wrapped.kadoa.com/Philpax?share https://hn-wrapped.kadoa.com/Philpax?share > You explain WebAssembly memory management with such passion that we're worried you might be dating your pointer allocations. > Your comments about multiplayer game architecture are so detailed, we suspect you've spent more time debugging network code than maintaining actual human connections. > You track AI model performance metrics more closely than your own bank account. DeepSeek R1 knows your preferences better than your significant other. I like your interests :)
- newbie578 2y agoScary to watch the pace of progress and how the whole industry is rapidly shifting. I honestly didn’t believe things would speed up this much.
- DavidPP 2y agoHaven't had time to try it out, but I've built myself a tool to tag my bookmarks and it uses 3.5 Haiku. Here is what it said about the official article content: I apologize, but the URL and page description you provided appear to be fictional. There is no current announcement of a Claude 3.7 Sonnet model on Anthropic's website. The most recent Claude 3 models are Claude 3 Haiku, Sonnet, and Opus, released in March 2024. I cannot generate a description for a non-existent product announcement. I appreciate their stance on safety, but that still made me laugh.
- dzhiurgis 2y agoAnyone else noticed all the reasoning models kinda catch up on claude and claude itself turned to crap last week?
- punkpeye 2y agoI have observed some unusual behavior. I wonder if it simply due to reprioritization of resources. Presumably, there is some parameter that determines how long a model is allowed to use resources for, which would get tapered in preparation for a demand surge of another model.
- kmlx 2y agoClaude 3.5 sonnet has been my go to for coding tasks, it’s just so much better than the others. but I’ve tried using the api in production and had to drop it due to daily issues: https://status.anthropic.com/ https://status.anthropic.com/ compare to https://status.openai.com/ https://status.openai.com/ any idea when we’ll see some improvements in api availability or will the focus be more on the web version of claude?
- msp26 2y agoDoes it show the raw "reasoning" tokens or is it a summary? Edit: > we’ve decided to make its thought process visible in raw form.
- koakuma-chan 2y agoWhere did 3.6 go?
- danielbln 2y agoAllegedly many people called new newest 3.5 revision 3.6, so Anthropic just rolled with it and called this 3.7.
- meetpateltech 2y agoWhen you ask: 'How many r's are in strawberry?' Claude 3.7 Sonnet generates a response in a fun and cool way with React code and a preview in Artifacts check out some examples: [1]https://claude.ai/share/d565f5a8-136b-41a4-b365-bfb4f4400df5 https://claude.ai/share/d565f5a8-136b-41a4-b365-bfb4f4400df5 [2]https://claude.ai/share/a817ac87-c98b-4ab0-8160-feefd7f798e8 https://claude.ai/share/a817ac87-c98b-4ab0-8160-feefd7f798e8
- jasonjmcghee 2y agoI'm guessing this is an easter egg, but this was a huge gripe I had with artifacts and eventually disabled it (now impossible to disable afaict) as I'd ask question completely unrelated to code or clearly not wanting code as an output, and I'd have to wait for it to write a program (which you can't stop afaict, it stops the current artifact then starts a new one) (still claude sonnet is my go-to and favorite model)
- falcor84 2y agoA shame the underlying issue still persists: > There is exactly 1 'r' in "blueberry" [0] [0] https://claude.ai/share/9202007a-9d85-49e6-9883-a8d8305cd29f https://claude.ai/share/9202007a-9d85-49e6-9883-a8d8305cd29f
- OsrsNeedsf2P 2y agoThis test has always been so stupid since models work at the token level. Claude 3.5 already 5xs your frontend dev speed but people still say "hurr durr it can't count strawberry" as if that's a useful problem
- dannyw 2y agoThe problem also comes to LLMs being confidently wrong when it’s wrong.
- bufferoverflow 2y agoThis test isn't stupid. If it can't count the number of letters in a text, can you rely on it with more important calculations?
- anti-soyboy 2y agoOpenAI should be worried as they products are weak
- batterylake 2y agoHi Claude Code team, excited for the launch! How well does Claude Code do on tasks which rely heavily on visual input such as frontend web dev or creating data visualizations?
- wolffiex 2y agoAs a CLI, this tool is most efficient when it can see text outputs from the commands that it runs. But you can help it with visual tasks by putting a screenshot file in your project directory and telling claude to read it, or by copying an image to your clipboard and pasting it with CTRL+V
- batterylake 2y agoCool, thanks!
- simion314 2y agoWhy not accepting other payment methods like PayPal/venmo ? Steam, Netflix have developers managed to integrate those payment methods so I conclude that Anthropic,Google, MS, OpenAI don't really need the money from the user but just hunting from big investors.
- siva7 2y agoWill Claude Code also be available with Pro Subscription?
- _joel 2y agoI've been using 3.5 with Roocode for the past couple of weeks and I've found it really quite powerful. Making it write tests and run them as part of the flow is with vscode windows pinging about is neat too.
- forrestthewoods 2y agoClaude is the best example of benchmarks not being reflective of reality. All the AI labs are so focused on improving benchmark scores but when it comes to providing actual utility Claude has been the winner for quite some time. Which isn’t to say that benchmarks aren’t useful. They surely are. But labs are clearly both overtraining and overindexing on benchmarks. Coming from gamedev I’ve always been significantly more yolo trust your gut than my PhD co-workers. Yes data is good. But I think the industry would very often be better off trusting guts and not needing a big huge expensive UX study or benchmark to prove what you can plainly see.
- Alifatisk 2y agoWhy is Claude-3.5-Haiku considered PRO and Claude-3.7-Sonnet is for free users?
- alecco 2y agoWho do I have to kill to get Claude Code access?
- xd1936 2y ago$ npm install -g @anthropic-ai/claude-code $ claude
- ckbishop 2y agoWell, I used 3.5 via Cursor to do some coding earlier today, and the output kind of sucked. Ran it through 3.7 a few minutes ago, and it's much more concise and makes sense. Just a little anecdotal high five from me.
- freediver 2y agoKagi LLM benchmark updated with general purpose and thinking mode for Sonnet 3.7. https://help.kagi.com/kagi/ai/llm-benchmark.html https://help.kagi.com/kagi/ai/llm-benchmark.html Appears to be second most capable general purpose LLM we tried (second to gemini 2.0 pro, in front of gpt-4o). Less impressive in thinking mode, about at the same level as o1-mini and o3-mini (with 8192 token thinking budget). Overall a very nice update, you get higher quality and higher speed model at same price. Hope to enable it in Kagi Assistant within 24h!
- jjice 2y agoThank you to the Kagi team for such fast turn around on new LLMs being accessible via the Assistant! The value of Kagi Assistant has been a no-brainer for me.
- hackernewds 2y ago[flagged]
- jjice 2y agoI find that giving encouraging messages when you're grateful is a good thing for everyone involved. I want the devs to know that their work is appreciated. Not everything is a tactical operation to get more subscription purchases - sometimes people like the things they use and want to say thanks and let others know.
- zaphod420 2y agoSome of us just actally really like kagi...
- flixing 2y agoDo you think kagi is the right Eval tool? If so,why?
- freediver 2y ago
- slantedview 2y agoAs a Claude Pro user, one of the biggest problems I have with day to day use of Sonnet is running out of tokens, and having to wait several hours. Would this new deep thinking capability just hit this problem faster?
- k8sToGo 2y agoHave you tried just using the API and pay as you go?
- mvdtnz 2y agoThat doesn't answer his very specific question.
- djeastm 2y agoThis thread is giving me a flashback to Stack Overflow
- k8sToGo 2y agoI think I might have just misunderstood his question then. I assumed the limitations come from the subscription plan
- paradite 2y agoIt's actually fairly easy to setup a 3rd party app to use Claude via API, to get extremely generous limits. I wrote a step-by-step guide for the app I built: https://prompt.16x.engineer/guide/claude https://prompt.16x.engineer/guide/claude
- ttoinou 2y agoPay 20 usd and ask them by email to get upgraded. I got to Tier 4 in a day
- grav 2y agoClaude 3.7 Sonnet seems to have a context window of 64.000 via the API: max_tokens: 4242424242 > 64000, which is the maximum allowed number of output tokens for claude-3-7-sonnet-20250219 I got a max of 8192 with Claude 3.5 sonnet.
- koakuma-chan 2y agoContext window is how long your prompt can be. Output tokens is how long its response can be. What you sent says its response can be 64k tokens at maximum.
- epistasis 2y agoIt's pretty fascinating to refresh the usage page on the API site while working [0]. After initialization it was up to 500k tokens ($1.50). After a few questions and a small edit, I'm up to over a million tokens (>$3.00). Not sure if the amount of code navigation and typing saved will justify the expense yet. It'll take a bit more experimentation. In any case, the default API buy of $5 seems woefully low to explore this tool. [0] https://console.anthropic.com/settings/usage https://console.anthropic.com/settings/usage
- koakuma-chan 2y agoIt also produces terrible code even though it's supposed to be good for front-end development.
- trekkie1024 2y agoCould you share an example?
- koakuma-chan 2y agoTLDR: told it to implement a grid view as an alternative to the existing list view, and specifically told it to DRY the code. What it did? Copy and pasted the list view implementation (definitely not DRY), and tried to make it a grid, and even though it is a grid, it looks terrible (https://i.imgur.com/fJiSjq4.png https://i.imgur.com/fJiSjq4.png). I don't understand how people use cursor and all that other shit when it cannot follow such simple instructions. Prompt (Claude Code): Implement an alternative grid view that the users can switch to. Follow the existing code style with empty comments and line breaks for improved code readability. Use snake case. DRY the code, avoid repetition of code. Do not change the font size or weight. Output: https://github.com/mayo-dayo/app/compare/0.4...claude-code-grid-view https://github.com/mayo-dayo/app/compare/0.4...claude-code-g...
- koakuma-chan 2y agoIt also keeps adding aspect-ratio to every single image it finds in my code base.
- highfrequency 2y agoAwesome work. When CoT is enabled in Claude 3.7 (not the new Claude Code), is the model now able to compile and run code as part of its thought process? This always seemed like very low hanging fruit to me, given how common this pattern is: ask for code, try running it, get an error (often from an outdated API in one of the packages used), paste the error back to Claude, have Claude immediately fix it. Surely this could be wrapped into the reasoning iterations?
- falcor84 2y agoWhy can't they count to 4? I accepted it when Knuth did it with TeX's versioning. And I sort of accept it with Python (after the 2-3 transition fiasco), but this is getting annoying. Why not just use natural numbers for major releases?
- jjice 2y agoI think I heard on a podcast with some of their team that they want 4 to be a massive jump. If I recall, they said that they want Haiku (the smallest of their current gen models) to be as good as Opus (the highest version, although there isn't one in the 3.5+ line) of the previous generation.
- sensanaty 2y agoYou'd think all these companies would have a single good naming convention, amazingly they don't. I suspect it's half on purpose so they can nerf the models without anyone suspecting once the hype dies down, since with every one of these models the latter version of the "same" version is worse than the launch version
- nurettin 2y agoWhat I love about their API is the tools array. Given a json schema describing your functions, it will output tool usage appropriate for the prompt. You can return tool results per call, and it will generate a dialog and additional tool calls based on those results.
- Uninen 2y agoI'm somewhat impressed from the very first interaction I had with Claude 3.7 Sonnet. I prompted it to find a problem in my codebase where a CloudFlare pages function would return 500 + nonsensical error and an empty response in prod. Tried to figure this out all Friday. It was super annoying to fix as there's no way to add more logging or have any visibility to the issue as the script died before outputting anything. Both o1, o3 and Claude 3.5 failed to help me in any way with this, but Claude 3.7 not only found the correct issue with first answer (after thinking 39 seconds) but then continued to write me a working function to work around the issue with the second prompt. (I'm going to let it write some tests later but stopped here for now.) I assume it doesn't let me to share the discussion as I connected my GitHub repo to the conversation (a new feature in the web chat UI launched today) but I copied it as a gist here: https://gist.github.com/Uninen/46df44f4307d324682dabb7aa6e10a34 https://gist.github.com/Uninen/46df44f4307d324682dabb7aa6e10...
- Uninen 2y agoOne thing about the reply gives away why Claude is still basically clueless about Actual Thinking; it suggested me to move the HTML sanitization to the frontend. It's in the CF function because it would be trivial to bypass it in the frontend making it easy to post literally anything in the db. Even a junior developer would understand this.
- gen3 2y agoYou could move the sanitation to the front end securely, it would just need to be right before render (after fetching the data to the browser). Some UI libraries do this automatically (like React) and the dompurify can run in the browser for this task. It could have done a better job outlining how to do it properly
- smt88 2y agoGP was talking about input sanitization, not output
- 2y ago
- umaar 2y agoDrawing an SVG of a pelican on a bicycle. Claude 3.7 edition: https://x.com/umaar/status/1894114767079403747 https://x.com/umaar/status/1894114767079403747
- redox99 2y agoClaude 3.5/3.6/3.7 seems too good at SVG compared to other models. I'd wager they did a bit of training specifically on that.
- pcwelder 2y agoClaude code terminal ux feels great. It has some well thought out features like restarting conversation with compressed context. Great work guys. However, I did get stuck when I asked it to run `npm create vite@latest todo-app` because it needs interactivity.
- g8oz 2y agoCongratulations on the release! While team members are monitoring this discussion let me add that a relatively simple improvement I’d like to see in the UI is the ability to export a chat to markdown or XML.
- bhouston 2y agoI wonder how similar Claude Code is to https://mycoder.ai https://mycoder.ai - which also uses Claude in an agentic fashion? It seems quite similar: https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
- jsemrau 2y agoWhat I found one of the most interesting takeaways from Huggingface's GAIA is that the agent would provide better result when the agent "reasoned" the response to the task in code.
- wewewedxfgdf 2y agoWhat makes software "agentic" instead of just a computer program? I hear lots of talk about agents and can't see them as being any different from an ordinary computer program.
- Uninen 2y agoThe Anthropic models comparison table has been updated now. Interesting new things at least the maximum output tokens upped from 8k to 64k and the knowledge cutoff date from April 2024 to October 2024. https://docs.anthropic.com/en/docs/about-claude/models/all-models#model-comparison-table https://docs.anthropic.com/en/docs/about-claude/models/all-m...
- bcherny 2y agoThanks everyone for all your questions! The team and I are signing off. Please drop any other bugs or feature requests here: https://github.com/anthropics/claude-code https://github.com/anthropics/claude-code. Thanks and happy coding!
- anotherpaulg 2y agoClaude 3.7 Sonnet scored 60.4% on the aider polyglot leaderboard [0], WITHOUT USING THINKING. Tied for 3rd place with o3-mini-high. Sonnet 3.7 has the highest non-thinking score, taking that title from Sonnet 3.5. Aider 0.75.0 is out with support for 3.7 Sonnet [1]. Thinking support and thinking benchmark results coming soon. [0] https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/ [1] https://aider.chat/HISTORY.html#aider-v0750 https://aider.chat/HISTORY.html#aider-v0750
- bearjaws 2y agoThanks for all the work on aider, my favorite AI tool.
- bt1a 2y agoIt really is best in slot. Owe it to git, which has a particular synergy with a hallucination-prone but correctable system
- doctoboggan 2y agoI like Aider but I've turned off auto-commit. I just can't seem to let the AI actually commit code for me. Do you regularly let Aider commit for you? How much do you review the code written by it?
- sejje 2y agoI don't let it auto commit, either. I don't like committing in a broken state, and the llm breaks things plenty often.
- MyOutfitIsVague 2y agoWhat's wrong with committing in a broken state if you squash those into a working state before pushing?
- hankchinaski 2y agoIt’s amazingly good, but it will be scaringly good when there will be a way to include the entire codebase in the context and let it create and run various parts of a large codebase autonomously. Right now I can only do patch work and give specific code snippets to make it work. Excited to try this new version out, I’m sure I won’t be disappointed, Edit: I just tried claude code CLI and it's a good compromise, it works pretty well, it does the discovery by itself instead of loading the whole codebase into context
- flutas 2y agoFWIW, there's a project to turn it into something similar, though I think it's lacking the "entire in context" part and runs into rate limits quick with Claude. https://github.com/All-Hands-AI/OpenHands https://github.com/All-Hands-AI/OpenHands The few times I've tested it out though it fails fairly quick and gets hung up (usually on setting up the project while testing with Kotlin / Go).
- thefourthchime 2y agoCursor AI is getting there.
- hankchinaski 2y agocursor is just a wrapper to the apis and is unnecessarily expensive, I use zed editor with custom API keys and it works super well
- knes 2y agoat Augment (https://augmentcode.com https://augmentcode.com) we were one of the partner who tested 3.7 pre-launch. And it has been a pretty significant increase in quality and code understanding. Happy to answer some questions FYI, We use Claude 3.7 has part of the new features we are shipping around Code Agent & more.
- ginkgotree 2y agoBeen using 3.5 sonnet for a mobile app build the past month. Havent had much time to get a good sense of 3.7 improvements, but I have to say the dev experience improvement of Claude Code right in my shell is fantastic. Loving it so far
- ismaelvega 2y agoAny plans to make some HackerRank Astra bench?
- specto 2y agoI've had a personal subscription to Claude for a while now. I would love if that also gave me access to some amount of API calls.
- mirekrusin 2y agoOk, just got documentation and fixed two bugs in my open source project. $1.42 This thing is a game changer.
- ramesh31 2y agoIt would be reeeaaally nice if someone built Claude Code into a Cline/Aider type extension...
- bittermandel 2y agoClaude Code works pretty OK so far, but Bash doesn't work straight up. Just sits and waits, even when running something basic like "!echo 123".
- leyoDeLionKin 2y agoI cancelled after I hit the limit, plus you have very limited support here in europe
- 0xcb0 2y agoI can just say that this is awesome. I just did spend 10$ and a handful of querys to init up a app idea I had in a while. The basic idea is working, it handled everything for me. From setting up the node environment. Creating the directories, files, patching the files, running code, handling errors, patching again. From time to time it fails to detect its own faults. But when I pinpoint it, it get it most of the time. And the UI is actually more pretty than I would have crafted in v1 When this get's cheaper, and better with each iteration, everybody will have a full dev team for a couple of bucks.
- wellthisisgreat 2y agoWhat’s the privacy like for Claude Code? Is it memorizing all the codebase?
- punkpeye 2y agoIt is whatever their API privacy policy is, i.e. private by default.
- j_maffe 2y agoIt redid half of my BSc thesis in less than 30s :| https://claude.ai/share/ed8a0e55-633f-4056-ba70-772ab5f5a08b https://claude.ai/share/ed8a0e55-633f-4056-ba70-772ab5f5a08b edit: Here's the output figure https://i.imgur.com/0c65Xfk.png https://i.imgur.com/0c65Xfk.png edit 2: Gemini Flash 2 failed miserably https://g.co/gemini/share/10437164edd0 https://g.co/gemini/share/10437164edd0
- ThouYS 2y agomaster and phd next!
- akreal 2y agoCould this (or something similar) be found in public access/some libraries?
- j_maffe 2y agoThere is only a single paper that has published a similar derivation but with a critical mistake. To be fair there are many documented examples of how to derive parametric relationships in linkages and can be quite methodical. I think I could get Gemini or 3.5 to do it but not single shot/ultra fast like here.
- crm9125 2y agoYes usually most of the topics covered in undergraduate studies are well documented and understood and therefore will likely be part of the training data of the AI. Once you get to graduate studies that's where the material coverage is a little more sparse/niche (though usually still not groundbreaking), and for a PhD. coverage is mostly non-existent since the point is to expand upon current knowledge within the field and many topics are being explored for the first time.
- TacticalCoder 2y ago[dead]
- dev0p 2y agoThe quality of the code is so much better! The UI seems to have an issue with big artifacts but the model is noticeably smarter. Congratulations on the release!
- unshavedyak 2y agoAre you using Claude Code or just the UI? Trying to figure out if anyone actually has Code yet hah. edit: Oh, there's a link to "joining the preview" which points to: https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview https://docs.anthropic.com/en/docs/agents-and-tools/claude-c...
- gigatexal 2y agoHow is the code generation? Open ai was generating good looking terraform but it was hallucinating on things that were incorrect.
- Copenjin 2y agoVery good, Code is extremely nice but as others have said, if you let it go on its own it burns through your money pretty fast. I've made it build a web scraper from scratch, figuring out the "API" of a website using a project from github in another language to get some hints, and while in the end everything was working, I've seen 100k+ tokens being sent too frequently for apparently simple requests, something feels off, it feels like there are quite a few opportunities to reduce token usage.
- deleted 2y ago[deleted]
- joelthelion 2y agoIt probably makes sense to continue using third party tools such as aider, for now. Anthropic doesn't have a lot of incentives to reduce token usage.
- jamiedumont 2y agoThat was my sense too. Used it for a few similar programs today (like converting HTML to Markdown but parsing certain <figure> elements to shortcodes) and scaffolding a Rust web app. It's done a reasonable job — but rips through credit, often changing its mind. Even strong-arming it into choosing an approach, it wanted to flip-flop between using regex and lol_html to parse the HTML whenever it came across a difficulty. If you're a US developer on whatever multiple of $ to the £ that I earn it might make sense, but burning through $100p/h for a pair programmer is a bit rich for my blood.
- taosx 2y agoThe model is expensive, it almost reaches what I charge per hour. If used right it can be a productivity increase otherwise if you trust it, it WILL introduce silent bugs. So if I have to go over the code line by line I'd prefer to use the cheapest viable model: deepseek, gemini any other free self-hosted models. Congratz to the team!
- vbezhenar 2y agoSo far only o1 pro was breathtaking for me few times. I wrote a kind of complex code for MCU which deals with FRAM and few buffers, juggling bytes around in a complex fashion. I was very not sure in this code, so I spent some time with AI chats asking them to review this code. 4o, o3-mini and claude were more or less useless. They spot basic stuff like this code might be problematic for multi-thread environment, those are obvious things and not even true. o1 pro did something on another level. It recognized that my code uses SPI to talk to FRAM chip. It decoded commands that I've used. It understood the whole timeline of using CS pin. And it highlighted to me, that I used WREN command in a wrong way, that I must have separated it from WRITE command. That was truly breathtaking moment for me. It easily saved me days of debugging, that's for sure. I asked the same question to Claude 3.7 thinking mode and it still wasn't that useful. It's not the only occasion. Few weeks before o1 pro delivered me the solution to a problem that I considered kind of hard. Basically I had issues accessing IPsec VPN configured on a host, from a docker container. I made a well thought question with all the information one might need and o1 pro crafted for me magic iptables incarnation that just solved my problem. I spent quite a bit of time working on this problem, I was close but not there yet. I often use both ChatGPT and Claude comparing them side by side. For other models they are comparable and I can't really say what's better. But o1 pro plays above. I'll keep trying both for the upcoming days.
- Hadriel 2y agoask the same question to grok 3 and report back :)
- dkulchenko 2y agoHave you tried comparing with 3.7 via the API with a large thinking budget yet (32k-64k perhaps?), to bring it closer to the amount of tokens that o1-pro would use? I think claude.ai’s web app in thinking mode is likely defaulting to a much much smaller thinking budget than that.
- davidbarker 2y agoClaude 3.5 Sonnet is great, but on a few occasions I've gone round in circles on a bug. I gave it to o1 pro and it fixed it in one shot. More generally, I tend to give o1 pro as much of my codebase as possible (it can take around 100k tokens) and then ask it for small chunks of work which I then pass to Sonnet inside Cursor. Very excited to see what o3 pro can do.
- danieldevries 2y agoJust tried Claude code. First impressions, it seems rather expensive. I prefer how Aider allows finer control over which files to add, or to use a sub-tree of a git repo. Also, It feels like the API calls when using Claude code are much faster then when using 3.7 on Aider. Giving bandwidth priority?
- RomanPushkin 2y ago> strong improvements in coding and front-end web development The best part
- Daniel_Van_Zant 2y agoBeing able to control how many tokens are spent on thinking is a game-changer. I've been building fairly complex, efficient, systems with many LLMs. Despite the advantages, reasoning models have been a no-go due to how variable the cost is, and how hard that makes it to calculate a final per-query cost for the customer. Being able to say "I know this model can always solve this problem in this many thinking tokens" and thus limiting the cost for that component is huge.
- jahooma 2y agoYup, it's just what we wanted for our coding agent. Codebuff can enter a "Deep thinking" mode and we can tell it to burn a lot of tokens hahaha.
- syndicatedjelly 2y agoClaude Code is pretty sick. I love the terminal integration, I like being able to stay on the keyboard and not have to switch UIs. It did a nice job learning my small Django codebase and helping me finish out a feature that I wasn't sure how to complete.
- unsupp0rted 2y agoAnybody else noticing that in Cursor, Claude Sonnet 3.7 is thinking much slower than Claude Sonnet 3.5 did?
- nomel 2y agoClaude 3.5 was not a thinking model. It's thinking time was 0s.
- unsupp0rted 2y agoOkay, if we're being pedantic, then anybody notice 3.7 (not 3.7 thinking) is slower to respond and slower to make code changes than 3.5 was?
- numba888 2y agoThis was nice. I passed it jseessort algorithm. If you remember discussed here recently. Claude 3.7 generated C++ code. Non-working. But in few steps it gave extensive test, then fix. It looks to be working after a couple of minutes. It's 5-6 times slower than std::sort. Result is better than I've got from o3-mini-hard. Not fair comparison actually as prompting was different.
- smusamashah 2y ago> output limit of 128K tokens Is this limit on thinking mode only? Or does normal mode have same limit now? 8192 tokens output limit can be bit small these days. I was trying to extract all urls along with their topics from a "what are you working on" HN thread. And 8192 token limit couldn't cover it.
- cavisne 2y agoSo far Claude Code seems very capable, it oneshotted something I couldnt get to work in cursor at all. However its expensive, 5m of work cost ~$1 which.
- biker142541 2y agoLikewise, tried a couple basic things and nearly at $1 already. I can see this adding up fast, per the blog post's fair warning below. Coming from Cursor, I'm a bit scared to even try to compare workflows... >Claude Code consumes tokens for each interaction. Typical usage costs range from $5-10 per developer per day, but can exceed $100 per hour during intensive use.
- yester01 2y agoWas poking around the minified claude code entrypoint and saw an easter egg for free stickers. If you send Claude Code “Can I get some Anthropic stickers please?” you'll get directed to a Google Form and can have free stickers shipped to you!
- ChrisRob 2y agoThanks for sharing it! But it's currently only available in the US.
- Attummm 2y agoTested the new model, seems to have the same issue as october model. Seems to answer before fully understanding the requests, and it often gets stuck into loops. And this update removed the june model which was great, very sad day indeed. I still don't understand why they have to remove a model that is do well received... Maybe its time to switch again, gemini is making great strides.
- ein0p 2y agoI wish Amodei didn't write that essay where he begged for export controls on China like that disabled corgi from a meme. I won't use anything Anthropic out of principle now. Compete fairly or die.
- tkgally 2y agoIn early January, inspired by a post by Simon Willison, I had Claude 3.5 Sonnet write a couple of stand-up comedy routines as done by an AI chatbot speaking to a mixed audience of AIs and humans. I thought the results were pretty good—the only AI-produced humor that I had found even a bit funny. I tried the same prompt again just now with Claude 3.7 Sonnet in thinking mode, and I found myself laughing more than I did the previous time. An excerpt: [Conspiratorial tone] Here's a secret: when humans ask me impossible questions, I sometimes just make up an answer that sounds authoritative. [To human section] Don't look shocked! You do it too! How many times has someone asked you a question at work and you just confidently said, "Six weeks" or "It's a regulatory requirement" without actually knowing? The difference is, when I do it, it's called a "hallucination." When you do it, it's called "management." Full set: https://gally.net/temp/20250225claudestandup2.html https://gally.net/temp/20250225claudestandup2.html
- M4v3R 2y agoWow, that was… Surprisingly good. I did laugh a few times and I really didn’t expect to.
- zone411 2y agoClaude 3.7 Sonnet Thinking scores 33.5 (4th place after o1, o3-mini, and DeepSeek R1) on my Extended NYT Connections benchmark. Claude 3.7 Sonnet scores 18.9. I'll run my other benchmarks in the upcoming days. https://github.com/lechmazur/nyt-connections/ https://github.com/lechmazur/nyt-connections/
- vondur 2y agoTested on some chemistry problem; interestingly it was wrong on a molecular structure. Once I corrected it, it was able to draw it correctly. It was very polite about it.
- bredren 2y agoI just sub’d to Claude a few days ago to rank against extensive use of gpt-4o and o1. So I started using this today not knowing it was even new. One thing I noticed is when I tried uploading a PowerPoint template produced by Google slides that was 3 slides—-just to give styling and format—-the web client said I’d exceeded line limit by 1200+%. Is that intentional? I wanted Claude to update the deck with content I provided in markdown but it could seemingly not be done, as the line overflow error prevented submission.
- bpbp-mango 2y agoUsing 3.7 today via the web UI and it feels far lazier than 3.5 was
- navin1110 2y agoHuh
- jimmcslim 2y agoWill aider and Claude Code meaningfully interpret a wireframe/mockup I put in the context as a PNG file? Or several mockups in a PDF? What success have people seen in this area?
- kashnote 2y agoKinda related: anyone know if there is an autocomplete plugin for Neovim on par with Cursor? I really want to use this new model in nvim to suggest next changes but none of the plugins I’ve come across are as good as Cursor’s.
- whywhywhywhy 2y agoWatching Claude Code fumble around trying to edit text and double checking the hex output of a .cpp file and cd around a folder all while burning actual dollars and context is the opposite of endearing.
- zora_goron 2y agoDoes anyone know how this “user decides how much compute” is implemented architecturally? I assume it’s the same underlying model, so what factor pushes the model to <think> for longer or shorter? Just a prompt-time modification or something else?
- epolanski 2y agoI am noticing a good dose of hallucination for 3.7 thinking in cursor. 3.7 seems more reliable.
- AlfeG 2y agoAhha, recently my daugher come to me with 3rd grade math problem. "Without rearranging the digits 1 2 3 4 5, insert mathematical operation signs and, if necessary, parentheses between them so that the resulting expression equals 40 and 80. The key is that you can combine digits (like 12+3/45) but you cannot change their order from the original sequence 1,2,3,4,5" Grok3, Claude, Deepseek, Qwen all failed to solve this problem. Resulting in some very very wrong solutions. While Grok3 were admit it fail and don't provide answers all other AI's are provided just plain wrong answers, like `12 * 5 = 80` ChatGPT were able to solve for 40, but not able to 80. YandexGPT solved those correctly (maybe it were trained on same Math books) Just checked Grok3 few more times. It were able to solve correctly for 80.
- sizzle 2y agoThis is what they are expecting 3rd graders to solve in math? Pretty hard for that age?
- AlfeG 2y agoThey have a lessons before on order of the expressions and some similiar problems. They were able to solve for 80, but stuck on 40 and asked me.
- bfm 2y agoNeither Claude Sonet 3.5 or 3.7 could solve this correctly unless you add to the prompt “ Prove it with the js analysis tool, please use an efficient combinatorial algorithm to find the solution”… and I had to correct 3.7 because it was not following the instructions as 3.5 did
- Pannoniae 2y agoLooks correct to me on 3.7 extended (albeit with loooots of thinking) although I'm incredibly exhausted so I might not be mathing correctly: https://claude.ai/share/dfb37c1a-f6a8-45a1-b987-e6d28e205080 https://claude.ai/share/dfb37c1a-f6a8-45a1-b987-e6d28e205080
- 2y ago
- Madmallard 2y agoIs it actually good at solving complex code or is it just garbage and people are lying about it as usual? In my experience EXTENSIVELY using claude 3.5 sonnet you basically have to do everything complex or you're just introducing massive amounts of slop code into your code base that while functional is nowhere near good. And for anything actually complex like requires a lot of context to make a decision and has to be useful to multiple different parts, it's just hopelessly bad.
- Snuggly73 2y agoI've played with it the whole day (so take it with a grain of salt). My gut feeling is that it can produce a bigger ... "thing". I am calling it a "thing", because it looks very much as what you want, but the bigger it is - the more the chances of it being subtly (or not) wrong. I usually ask the models to extend a small parser/tree-walking interpreter with a compiler/VM. Up until Claude 3.7 the models would propose something lazy and obviously incomplete. 3.7 generated something that looks almost right, mostly works, but is so overcomplicated and broken in such a way, that I rather delete it and write it from scratch. Trying to get the model to fix it resulted in running in circles, spitting out pieces of code that didn't fit the existing ones etc. Not sure if I prefer the former or the latter tbh.
- dsincl12 2y agoIs it just me who get the feeling that Claude 3.7 is worse than 3.5? I really like 3.5 and can be productive with it, but with Claude 3.7 it can't fix even simple things. Last night I sat for 30 minutes just to try to get the new model to remove a instructions section from a Next.js page. It was an isolated component on the page named InstructionsComponent. Failed non-stop, didn't matter what I did, it could not do it. 3.5 did it first try, I even mistyped instructions and the model fixed the correct thing anyway.
- OhioMan2943 2y agoI don't think it's worse but it's like losing a friend in a small way. It's not the same assistant you talked to previously
- deleted 2y ago[deleted]
- matthewconnell 2y ago[dead]
- datadeft 2y agoI am not sure how good these Exercism tasks are for measuring how good at a model with coding. My experience is that these models could write a simple function and get it right if it does not require any out of the box thinking (so essentially offloading the boilerplate part of coding). When it comes to think creatively and have a much better solution to a specific task that would require the think 2-3 steps ahead than they are not suitable.
- berkes 2y agoI think many of the "AI can do coding" narratives don't see what coding means in real situations. It's finding out why "jbdoe1337" added this large if/else around the entire function body back in 2016 - it seems important business logic, but the commit just says "updated code". And how the h*ll this interaction between the conf.ini files, the conf/something.json and the ENV vars works. Why sometimes the ENV var overrides a value in the ini and why its sometimes the other way around. But also finding that when you clean it up, everything falls apart. It's discussing with the stakeholders why "adding a delete button" isn't as easy as just putting a button there, but that it means designing a whole cascading deletion strategy and/or trashcan and/or soft-delete and/or garbage-collection. It's finding out why - again - the grumb pipeline crashes with the typebar checker, when used through mpm-yearn package manager. Both in containers and on a osx machine but not on Linux Brobuntu 22.12 LTLS. It's moving stuff in the right abstraction layer. It's removing abstractions while introducing others. KISS vs future flexibility. It's gut feeling when to apply DRY and when to embrace it. And then, if your lucky, churning out boilerplate or new code for 120 minutes a week. I'm glad that this 120 minutes can be improved with AI and become 20 minutes. Truly. But this is not what (senior?) programmers do. Despite what the hyped up AI press makes us believe. It only shows they have no idea what the "real" problems and time-consumers are for programmers.
- datadeft 2y agoExactly. People sold on AI replacing software engineers are missing the point. It is almost the say that better laptops are replacing software engineers. LLMs are just tools that make you faster. Finding bugs, writing documentation, etc. are very nice to accelerate but creative thinking is also a big part of the job.
- deleted 2y ago[deleted]
- deleted 2y ago[deleted]
- casey2 2y agoThis is what we meant by "AI can only get better from here" or "Right now AI is the worst it will ever be"
- Yustynn 2y agoI feel like 3.7's personality is neutered, and frankly, the personality was the biggest selling point for me
- jaybro867 2y ago[dead]
- qoez 2y agoCan't wait to try this in 6 months when it arrives in europe and the competition has superior models available before then
- createaccount99 2y agoWhy would they release Claude Code as closed source? Let's hope DeepSeek-r2 delivers, Anthropic is dead. I mean, it's a tool designed to eat itself. Absurd to close source.
- mark_l_watson 2y agoI like Claude Sonnet and use it 4 or 5 times a week via ChatLLM to generate code. I started setting up for Claude Code this morning, then remembered how pissed I was at their CEO for the really lame anti-open source and anti-open weight models he was making publicly after the DeepSeek-R rollout - I said NOPE and didn’t install Claude Code. CEOs should really watch what they say in public. Anyway, this is all just my opinion.
- james_marks 2y agoAnecdotal cost impact- After toying with Claude Code for the afternoon, my Anthropic spend just went from $20/mo to $10/day. Still worth it, but that’s a big jump.
- smithcoin 2y agoSo it’s an order of magnitude more effective?
- james_marks 2y agoPerhaps a magnitude more effective than copy/paste, perhaps not. But do I get more than $300/month of value from it, per developer? Almost certainly. The bottleneck was already checking the work for correctness and building my own trust / familiarity with new code. So it's made that problem slightly more pronounced, as it generates more code faster, with more surface area to debug when many new changes arrive at once.
- smithcoin 2y agoYou described the price increasing from $10/month to $300 (which is an order of magnitude). What I meant was when you are saying it equals out (in terms of cost for you) you would expect a similar increase in performance. If that makes sense.
- james_marks 2y agoI understand, but it's the wrong question IMO. It was dramatically under-priced before, and less so now.
- chipgap98 2y agoYeah I think that's very doable for a business but it gets expensive if you are just tinkering
- npace12 2y agoThe source maps were included in an earlier release. I extracted the source code here if anyone is curious: https://github.com/dnakov/claude-code https://github.com/dnakov/claude-code
- kaveh_h 2y agoI saw that Claude 3.7 Sonnet both regular and thinking was available for Github Copilot (Pro) 5 hours ago for me, I enabled it and tried it out a couple of times, but for the past hour the option has disappeared. I'm situated in Europe (Sweden), anyone else having the same experience?
- gunalx 2y agoIt seemed kinda buggy as well with latencies
- shekhargulati 2y agoI asked Claude 3.7 Sonnet to generate an SVG illustration of Maha Kumbh. The generated SVG includes a Shivling (https://en.wikipedia.org/wiki/Lingam https://en.wikipedia.org/wiki/Lingam) and also depicts Naga Sadhus well. Both Grok 3 and OpenAI o3 failed miserably. You can view the generated SVG and the exact prompt here: https://shekhargulati.com/2025/02/25/can-claude-3-7-sonnet-generate-svg-illustration-for-maha-kumbh/ https://shekhargulati.com/2025/02/25/can-claude-3-7-sonnet-g...
- __MatrixMan__ 2y agoIt's smarter, but it also feels more aggressive than 3.5. I'm finding I need to tell it not to do superfluous things more often
- simonw 2y agoI got this working with my LLM tool (new plugin version: llm-anthropic 0.14) and figured out a bunch of things about the model in the process. My detailed notes are here: https://simonwillison.net/2025/Feb/25/llm-anthropic-014/ https://simonwillison.net/2025/Feb/25/llm-anthropic-014/ One of the most exciting new capabilities is that this model has a 120,000 token output limit - up from just 8,000 for the previous Claude 3.5 Sonnet model and way higher than any other model in the space. It seems to be able to use that output limit effectively. Here's my longest result so far, though it did take 27 minutes to finish! https://gist.github.com/simonw/854474b050b630144beebf06ec4a2f52 https://gist.github.com/simonw/854474b050b630144beebf06ec4a2...
- Citizen_Lame 2y agoHow much did it cost?
- mrbonner 2y ago$1.8
- rvnx 2y agoI have a very long request to do like this, did you use a specific CLI tool ? (Thank you in advance)
- simonw 2y agoI used my own CLI tool LLM, which can handle these long requests in streaming mode (Anthropic won't let you do a non-streaming request for long output replies like this). uv tool install llm llm install llm-anthropic llm keys set anthropic # paste in API key llm -m claude-3.7-sonnet -o thinking 1 'your prompt goes here'
- rvnx 2y agoThank you very much
- melvinroest 2y agoSo I tried schemesh [1] with it. That was a rough ride, wow. schemesh is lisp in your shell. Most of the bash syntax remains. Claude was okay with lisp, but understanding the gist of schemesh, it fount it really hard - even when I supplied the git source code. ChatGPT O3 (high) had similar issues. [1] https://news.ycombinator.com/item?id=43061183 https://news.ycombinator.com/item?id=43061183
- 0x1ceb00da 2y agoVideo killed the radio star
- erichocean 2y agoPlease provide the ability to diff file versions within the browser. I really want to be able to see what specifically is changing, not just the entire new file. Also, if the user provides a file for modification, make that available as Version 0 (or whatever), so we can diff against that.
- biker142541 2y agoSadly, Claude 3.7 is still failing pretty hard on Svelte 5 even when provided latest docs in context. It just fails more confidently and further into otherwise decent code than 3.5. Ex: built a much more complex initial app, but used runes incorrectly and continued to use <slot>. Even when prompted with update doc snippets, it couldn't dig itself out of its hole. We really still need a better unified workflow for working on the cutting edge of tech with LLMs, imo. This problem is the same with other frameworks/technologies undergoing recent changes.
- AtlasBarfed 2y ago"port sed to java with all options and capabilities" Still is very underwhelming. I like this because it isn't a difficult problem, it should be up the alley of a "language model" to translate computer languages, but it is a fairly complex problem with lots of options and parse annoyances. Addresses can be pretty complex with regex in line selections/subsetting. Scripts are supported. Probably turing complete considering the pattern space as storage and looping/jump constructs. In an experience reminescent of "can I have L2 support please" most AIs give a kind of milquetoast slightly above average IQ responses to various questions. I wonder if there should be standard "please give me more complicated/erudite/involved explanations/documents/code from the get-go to not bother with the boring prompts.
- cadamsdotcom 2y agoFinally got access to the preview just now. Let's fire it up. "Type /init to set up your repository" OK, /init <enter> "OK, I created CLAUDE.md, session cost so far is $0.1764" QUIT QUIT QUIT QUIT QUIT Seventeen cents just to initialize yourself, Claude. No. I feel like I touched a live wire. It's about 2 orders of magnitude (100x) too expensive.
- mandeepbagga 2y ago[dead]
- Darius95yo 2y agoHey
- Darius95yo 2y agoHelo
- Darius95yo 2y agoHello
- epictwow 2y agoDesigning some chartreuse lamps, quickly zip away from their radiuses. I just successfully exited the "lime light"!