27 ms·
Claude 4
- swyx 1y agolivestream here: https://youtu.be/EvtPBaaykdo https://youtu.be/EvtPBaaykdo my highlights: 1. Coding ability: "Claude Opus 4 is our most powerful model yet and the best coding model in the world, leading on SWE-bench (72.5%) and Terminal-bench (43.2%). It delivers sustained performance on long-running tasks that require focused effort and thousands of steps, with the ability to work continuously for several hours—dramatically outperforming all Sonnet models and significantly expanding what AI agents can accomplish." however this is Best of N, with no transparency on size of N and how they decide the best, saying "We then use an internal scoring model to select the best candidate from the remaining attempts." Claude Code is now generally available (we covered in http://latent.space/p/claude-code http://latent.space/p/claude-code ) 2. Memory highlight: "Claude Opus 4 also dramatically outperforms all previous models on memory capabilities. When developers build applications that provide Claude local file access, Opus 4 becomes skilled at creating and maintaining 'memory files' to store key information. This unlocks better long-term task awareness, coherence, and performance on agent tasks—like Opus 4 creating a 'Navigation Guide' while playing Pokémon." Memory Cookbook: https://github.com/anthropics/anthropic-cookbook/blob/main/tool_use/memory_cookbook.ipynb https://github.com/anthropics/anthropic-cookbook/blob/main/t... 3. Raw CoT available: "we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access." 4. haha: "We no longer include the third ‘planning tool’ used by Claude 3.7 Sonnet. " <- psyop? 5. context caching now has a premium 1hr TTL option: "Developers can now choose between our standard 5-minute time to live (TTL) for prompt caching or opt for an extended 1-hour TTL at an additional cost" 6. https://www.anthropic.com/news/agent-capabilities-api https://www.anthropic.com/news/agent-capabilities-api new code execution tool (sandbox) and file tool
- modeless 1y agoMemory could be amazing for coding in large codebases. Web search could be great for finding docs on dependencies as well. Are these features integrated with Claude Code though?
- obiefernandez 1y ago
- modeless 1y agoOoh, VS Code integration for Claude Code sounds nice. I do feel like Claude Code works better than the native Cursor agent mode. Edit: How do you install it? Running `/ide` says "Make sure your IDE has the Claude Code extension", where do you get that?
- ChadMoran 1y agoI've been looking for that as well.
- ChadMoran 1y agoRun claude cli from inside of VS Code terminal. https://news.ycombinator.com/item?id=44064082 https://news.ycombinator.com/item?id=44064082
- k8sToGo 1y agoWhen you install claude code (or update it) there is a .vsix file in the same area where claude bin is.
- modeless 1y agoThanks, I found it in node_modules/@anthropic-ai/claude-code/vendor
- jacobzweig 1y agoRun claude in the terminal in VSCode (or cursor) and it will install automatically!
- jbellis 1y agoGood, I was starting to get uncomfortable with how hard Gemini has been dominating lately ETA: I guess Anthropic still thinks they can command a premium, I hope they're right (because I would love to pay more for smarter models). > Pricing remains consistent with previous Opus and Sonnet models: Opus 4 at $15/$75 per million tokens (input/output) and Sonnet 4 at $3/$15.
- deleted 1y ago[deleted]
- esaym 1y agoheh, I just wrote a small hit piece about all the disappointments of the models over the last year and now the next day there is a new model. I'm going to assume it will still get you only to 80% ( ͡° ͜ʖ ͡°)
- __jl__ 1y agoAnyone found information on API pricing?
- rafram 1y agoIt’s up on their pricing page: https://www.anthropic.com/pricing https://www.anthropic.com/pricing
- deleted 1y ago[deleted]
- burningion 1y agoYeah it's live on the pricing page: https://www.anthropic.com/pricing#api https://www.anthropic.com/pricing#api Opus 4 is $15 / m tokens in, $75 / MTok out Sonnet 4 is the same $3 / MTok in, $15 / MTok out
- __jl__ 1y agoThanks. I looked a couple minutes ago and couldn't see it. For anyone curious, pricing remains the same as previous Anthropic models.
- pzo 1y agosurprisingly cursor charges only 0.75x for request for sonnet 4.0 (comparing to 1x for sonnet 3.7)
- transcranial 1y agoIt does say "temporarily offered at a discount" when you hover over the model in the dropdown.
- piperswe 1y agoFrom the linked post: > Pricing remains consistent with previous Opus and Sonnet models: Opus 4 at $15/$75 per million tokens (input/output) and Sonnet 4 at $3/$15.
- esaym 1y ago> Try Claude Sonnet 4 today with Claude Opus 4 on paid plans. Wait, Sonnet 4? Opus 4? What?
- deleted 1y ago[deleted]
- sxg 1y agoClaude names their models based on size/complexity: - Small: Haiku - Medium: Sonnet - Large: Opus
- jareds 1y agoI'll look at it when this shows up on https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/ I feel like keeping up with all the models is a full time job so I just use this instead and hopefully get 90% of the benefit I would by manually testing out every model.
- evantbyrne 1y agoAre these just leetcode exercises? What I would like to see is an independent benchmark based on real tasks in codebases of varying size.
- KaoruAoiShiho 1y agoAider is not just leetcode exercises I think? livecodebench is leetcode exercises though.
- rafram 1y agoAider uses a dataset of 500 GitHub issues, so not LeetCode-style work.
- evantbyrne 1y agoIt says right on that linked page: > Aider’s polyglot benchmark tests LLMs on 225 challenging Exercism coding exercises across C++, Go, Java, JavaScript, Python, and Rust. I looked up Exercism and they appear to be story problems that you solve by coding on mostly/entirely blank slates, unless I'm missing something? That format would seem to explain why the models are reportedly performing so well, because they definitely aren't that reliable on mature codebases.
- deleted 1y ago[deleted]
- jasonthorsness 1y ago“GitHub says Claude Sonnet 4 soars in agentic scenarios and will introduce it as the base model for the new coding agent in GitHub Copilot.” Maybe this model will push the “Assign to CoPilot” closer to the dream of having package upgrades and other mostly-mechanical stuff handled automatically. This tech could lead to a huge revival of older projects as the maintenance burden falls.
- ed_elliott_asc 1y agoUntil it pushes a severe vulnerability which takes a big service doen
- BaculumMeumEst 1y agoAnyone see news of when it’s planned to go live in copilot?
- vinhphm 1y agoThe option just shown up in Copilot settings page for me
- BaculumMeumEst 1y agoSame! Rock and roll!
- denysvitali 1y agoI got rate-limited in like 5 seconds. Wow
- bbor 1y agoTurns out Opus 4 starts at their $40/mo ("Pro+") plan which is sad, and they serve o4-mini and Gemini as well so it's a bit less exclusive than this announcement implies. That said, I have a random question for any Anthropic-heads out there: GitHub says "Claude Opus 4 is hosted by Anthropic PBC. Claude Sonnet 4 is hosted by Anthropic 1P."[1]. What's Anthropic 1P? Based on the only Kagi result being a deployment tutorial[2] and the fact that GitHub negotiated a "zero retention agreement" with the PBC but not whatever "1P" is, I'm assuming it's a spinoff cloud company that only serves Claude...? No mention on the Wikipedia or any business docs I could find, either. Anyway, off to see if I can access it from inside SublimeText via LSP! [1] https://docs.github.com/en/copilot/using-github-copilot/ai-models/using-claude-in-github-copilot https://docs.github.com/en/copilot/using-github-copilot/ai-m... [2] https://github.com/anthropics/prompt-eng-interactive-tutorial https://github.com/anthropics/prompt-eng-interactive-tutoria...
- james_marks 1y ago> we’ve significantly reduced behavior where the models use shortcuts or loopholes to complete tasks. Both models are 65% less likely to engage in this behavior than Sonnet 3.7 on agentic tasks Sounds like it’ll be better at writing meaningful tests
- NitpickLawyer 1y agoOne strategy that also works is to have 2 separate "sessions", have one write code and one write tests. Forbid one to change the other's "domain". Much better IME.
- UncleEntity 1y agoIn my experience, when presented with a failing test it would simply try to make the test pass instead of determining why the test was failing. Usually by hard coding the test parameters (or whatever) in the failing function... which was super annoying.
- 0x457 1y agoI once saw probably 10 iterations to fix a broken test, then it decided that we don't need this test at all, and it tried to just remove it. IMO, you either write tests and let it write implementation or write implementation and let it write tests. Maybe use something to write tests, then forbid "implementor" to modify them.
- sigmoid10 1y agoSooo... it can play Pokemon. Feels like they had to throw that in after Google IO yesterday. But the real question is now can it beat the game including the Elite Four and the Champion. That was pretty impressive for the new Gemini model.
- throwaway314155 1y agoGemini can beat the game?
- mxwsn 1y agoGemini has beat it already, but using a different and notably more helpful harness. The creator has said they think harness design is the most important factor right now, and that the results don't mean much for comparing Claude to Gemini.
- throwaway314155 1y agoWay offtopic to TFA now, but isn't using an improved harness a bit like saying "I'm going to hardcore as many priors as possible into this thing so it succeeds regardless of its ability to strategize, plan and execute?
- samrus 1y agoit is. the benchmark was somewhat cheated, from the perspective of finding out how the model adjusts and plans within a dynamic reactive environment
- 11101010001100 1y agoThey asked gemini to come up with another word for cheating and it came up with 'harness'.
- silvr 1y agoWhile true to a degree, I think this is largely wrong. Wouldn't it still count as a "harness" if we provided these LLMs with full robotic control of two humanoid arms, so that it could hold a Gameboy and play the game that way? I don't think the lack of that level of human-ness takes away from the demonstration of long-context reasoning that the GPP stream showed. Claude got stuck reasoning its way through one of the more complex puzzle areas. Gemini took a while on it also, but made it through. I don't that difference can be fully attributed up to the harnesses. Obviously, the best thing to do would be to run a SxS in the same harness of the two models. Maybe that will happen?
- htrp 1y agoAllegedly Claude 4 Opus can run autonomously for 7 hours (basically automating an entire SWE workday).
- catigula 1y agoThat is quite the allegation.
- jeremyjh 1y agoWhich sort of workday? The sort where you rewrite your code 8 times and end the day with no marginal business value produced?
- renewiltord 1y agoWell Claude 3.7 definitely did the one where it was supposed to process a file and it finally settled on `fs.copyFile(src, dst)` which I think is pro-level interaction. I want those $0.95 back. But I love you Claude. It was me, not you.
- readthenotes1 1y agoWell, at least it doesn't distract your coworkers, disrupting their flow
- speedgoose 1y agoEasy, I can also make a nanoGPT run for 7 hours when inferring on a 68k, and make it produce as much value as I usually do.
- lxe 1y agoLooks like both opus and sonnet are already in Cursor.
- daviding 1y ago"We're having trouble connecting to the model provider" A bit busy at the moment then.
- eamag 1y agoNobody cares about lmarena anymore? I guess it's too easy to cheat there after a llama4 release news?
- i_love_retros 1y ago[flagged]
- op00to 1y agoGood point. We should only focus on intractable problems and put everything else on the back burner. We certainly don’t have the ability to help people and advance science and business.
- i_love_retros 1y agoHow is this absurd gold rush advancing science? What important scientific discoveries have been made so far using LLMs? Chatbots? Coding tools? The only business being helped is the few mega wealthy "AI" company owners. This stream of announcements is a parody at this point.
- op00to 1y agoYou’re not here to discuss tech,you’re sea lioning to derail the thread with bad-faith outrage. Meanwhile, LLMs are making a difference: they power screen readers and plain language summaries for disabled users, help immigrants and low income folks fill out forms and understand legal docs, offer free mental health support where care is inaccessible, and tutor kids in underserved schools. If you want to deny that, the burden’s on you to disprove it, not on me to jump through hoops every time you pretend to ask for more proof. Classic sea lioning.
- hooverd 1y agoThey're great tools. I don't think the education system should be gutted and replaced with them though. What are the people pushing said tools choosing for their own children, not the masses?
- op00to 1y agoI could be wrong, but no where in this thread are thread are there suggestions to gut education systems and replace them with Claude.
- josefresco 1y agoI have the Claude Windows app, how long until it can "see" what's on my screen and help me code/debug?
- throwaway314155 1y agoYou can likely set up an MCP server that handles this.
- msp26 1y ago> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access. Extremely cringe behaviour. Raw CoTs are super useful for debugging errors in data extraction pipelines. After Deepseek R1 I had hope that other companies would be more open about these things.
- blueprint 1y agopretty sure the cringe doesn't stop there. It wouldn't surprise me if this is not the only thing that they are attempting to game and hide from their users. The Max subscription with fake limits increase comes to mind.
- deleted 1y ago[deleted]
- oofbaroomf 1y agoWonder why they renamed it from Claude <number> <type> (e.g. Claude 3.7 Sonnet) to Claude <type> <number> (Claude Opus 4).
- deleted 1y ago[deleted]
- stavros 1y agoI guess because they haven't been releasing all three models of the same version in a while now. We've only had Sonnet updates, so the version first didn't make sense if we had 3.5 and 3.7 Sonnet but none of the others.
- hansmayer 1y ago[flagged]
- oofbaroomf 1y agoInteresting how Sonnet has a higher SWE-bench Verified score than Opus. Maybe says something about scaling laws.
- somebodythere 1y agoMy guess is that they did RLVR post-training for SWE tasks, and a smaller model can undergo more RL steps for the same amount of computation.
- benoittravers 1y agoDo you have the link to that benchmark? Can’t see where Sonnet is highlighted.
- Artgor 1y agoOpenIA's Codex-1 isn't so cool anymore. If it was ever cool. And Claude Code used Opus 4 now!
- boh 1y agoCan't wait to hear how it breaks all the benchmarks but have any differences be entirely imperceivable in practice.
- jackdeansmith 1y agoIn my opinion most Anthropic models are the opposite, scoring well on benchmarks but not always way on top, but quietly excellent when you actually try to use them for stuff.
- waynecochran 1y agoMy mind has been blown using ChatGPT's o4-mini-high for coding and research (it knowledge of computer vision and tools like OpenCV are fantastic). Is it worth trying out all the shiny new AI coding agents ... I need to get work done?
- throwaway314155 1y agoI would say yes. The jump in capability and reduction in hallucinations (at least code) to Claude 3.7 from ChatGPT (even o3) is immediately noticeable in my experience. Same goes for gemini which was even better in some ways until perhaps today.
- bcrosby95 1y agoKinda interesting, as I've found 4o better than o4-mini-high for most of my coding. And while it's mind blowing that they can do what they can do, the code itself coming out the other end has been pretty bad, but good enough for smaller snippets and extremely isolated features.
- renewiltord 1y agoSame pricing as before is sick!
- uludag 1y agoI'm curious what are others priors when reading benchmark scores. Obviously with immense funding at stakes, companies have every incentive to game the benchmarks, and the loss of goodwill from gaming the system doesn't appear to have much consequences. Obviously trying the model for your use cases more and more lets you narrow in on actually utility, but I'm wondering how others interpret reported benchmarks these days.
- iLoveOncall 1y agoHasn't it been proven many times that all those companies cheat on benchmarks? I personally couldn't care less about them, especially when we've seen many times that the public's perception is absolutely not tied to the benchmarks (Llama 4, the recent OpenAI model that flopped, etc.).
- sebzim4500 1y agoI don't think there's any real evidence that any of the major companies are going out of their way to cheat the benchmarks. Problem is that, unless you put a lot of effort into avoiding contamination, you will inevitably end up with details about the benchmark in the training set.
- candiddevmike 1y agoPeople's interpretation of benchmarks will largely depend on whether they believe they will be better or worse off by GenAI taking over SWE jobs. Think you'd need someone outside the industry to weigh in to have a real, unbiased view.
- douglasisshiny 1y agoOr someone who has been a developer for a decade plus trying to use these models on actual existing code bases, solving specific problems. In my experience, they waste time and money.
- sandspar 1y ago
- Doohickey-d 1y ago> Users requiring raw chains of thought for advanced prompt engineering can contact sales So it seems like all 3 of the LLM providers are now hiding the CoT - which is a shame, because it helped to see when it was going to go down the wrong track, and allowing to quickly refine the prompt to ensure it didn't. In addition to openAI, Google also just recently started summarizing the CoT, replacing it with an, in my opinion, overly dumbed down summary.
- Aeolun 1y agoThe Google CoT is so incredibly dumb. I thought my models had been lobotomized until I realized they must be doing some sort of processing on the thing.
- whimsicalism 1y agoit’s fake cot, just like oai
- phatfish 1y agoI had assumed it was a way to reduce "hallucinations". Instead of me having to double check every response and prompt it again to clear up the obvious mistakes it just does that in the background with itself for a bit. Obviously the user still has to double check the response, but less often.
- user_7832 1y agoYou are referring to the new (few days old-ish) CoT right? It’s bizzare as to why google did it, it was very helpful to see where the model was making assumptions or doing something wrong. Now half the time it feels better to just use flash with no thinking mode but ask it to manually “think”.
- sunaookami 1y agoGuess we have to wait till DeepSeek mops the floor with everyone again.
- mupuff1234 1y agoBut if Gemini 2.5 pro was considered to be the strongest coder lately, does SWE-bench really reflect reality?
- oofbaroomf 1y agoNice to see that Sonnet performs worse than o3 on AIME but better on SWE-Bench. Often, it's easy to optimize math capabilities with RL but much harder to crack software engineering. Good to see what Anthropic is focusing on.
- j_maffe 1y agoThat's a very contentious opinion you're stating there. I'd say LLMs have surpassed a larger percentage of SWEs in capability than they have for mathematicians.
- oofbaroomf 1y agoMathematicians don't do high school math competitions - the benchmark in question is AIME. Mathematicians generally do novel research, which is hard to optimize for easily. Things like LiveCodeBench (leetcode-style problems), AIME, and MATH (similar to AIME) are often chosen by companies so they can flex their model's capabilities, even if it doesn't perform nearly as well in things real mathematicians and real software engineers do.
- j_maffe 1y agoOk then you should clarify that you meant math benchmarks and not math capabilities.
- sali0 1y agoI've found myself having brand loyalty to Claude. I don't really trust any of the other models with coding, the only one I even let close to my work is Claude. And this is after trying most of them. Looking forward to trying 4.
- javier2 1y agoI wouldn't go as far, but I actually have some loyalty to Claude as well. Don't even know why, as I think the differences are marginal.
- bckr 1y agoIt’s possible to get to know the quirks of these models and intuit what will and won’t work, and how to overcome those limitations. It’s also possible to just get to know, like, and trust their voice. I’m certain that brand awareness is also a factor for me in preferring Claude over ChatGPT etc
- anal_reactor 1y agoI've been initially fascinated by Claude, but then I found myself drawn to Deepseek. My use case is different though, I want someone to talk to.
- miroljub 1y agoI also use DeepSeek R1 as a daily driver. Combined with Qwen3 when I need better tool usage. Now that both Google and Claude are out, I expect to see DeepSeek R2 released very soon. It would be funny to watch an actual open source model getting close to the commercial competition.
- ashirviskas 1y agoHave you compared R1 with V3-0324?
- 1y ago
- goranmoomin 1y ago> Extended thinking with tool use (beta): Both models can use tools—like web search—during extended thinking, allowing Claude to alternate between reasoning and tool use to improve responses. I'm happy that tool use during extended thinking is now a thing in Claude as well, from my experience with CoT models that was the one trick(tm) that massively improves on issues like hallucination/outdated libraries/useless thinking before tool use, e.g. o3 with search actually returned solid results, browsing the web as like how i'd do it, and i was thoroughly impressed – will see how Claude goes.
- blueprint 1y agoAnthropic might be scammers. Unclear. I canceled my subscription with them months ago after they reduced capabilities for pro users and I found out months later that they never actually canceled it. They have been ignoring all of my support requests.. seems like a huge money grab to me because they know that they're being out competed and missed the ball on monetizing earlier.
- iLoveOncall 1y agoI can't think of more boring than marginal improvements on coding tasks to be honest. I want GenAI to become better at tasks that I don't want to do, to reduce the unwanted noise from my life. This is when I'll pay for it, not when they found a new way to cheat a bit more the benchmarks. At work I own the development of a tool that is using GenAI, so of course a new better model will be beneficial, especially because we do use Claude models, but it's still not exciting or interesting in the slightest.
- amkkma 1y agoyea exactly
- danielbln 1y agoWhat if coding is that unwanted task? Also, what are the tasks you are referring to, specifically?
- iLoveOncall 1y agoBooking visits to the dentist, hairdresser, any other type of service, renewing my phone or internet subscription at the lowest price, doing administrative tasks, adding the free game of the week on Epic Games Store to my library, finding the right houses to visit, etc. Basically anything that some startup has tried and failed at uberizing.
- jetsetk 1y agoWhy would coding be that unwanted task if one decided to work as a programmer? People's unwanted tasks are cleaning the house, doing taxes etc.
- atonse 1y agoBut the value in coding (for the overwhelming majority of the people) is the product of coding (the actual software), not the code itself. So to most people, the code itself doesn't matter (and never will). It's what it lets them actually do in the real world.
- eamag 1y agoWhen will structured output be available? Is it difficult for anthropic because custom sampling breaks their safety tools?
- HiPHInch 1y agoHow long will the VScode wrapper (cursor, windsurf) survive? Love to try the Claude Code VScode extension if the price is right and purchase-able from China.
- bingobangobungo 1y agowhat do you mean purchasable from china? As in you are based in china or is there a way to game the tokens pricing
- HiPHInch 1y agoClaude register need a phone number, but cannot select China (+86), and even if I have a account, it may hard to purchase because the credit card issue. Some app like Windsurf can easily pay with Alipay, a everyone-app in China.
- barnabee 1y agoI don't see any benefit in those VC funded wrappers over open source VS Code (or better, VSCodium) extensions like Roo/Cline. They survive through VC funding, marketing, and inertia, I suppose.
- diggan 1y agoAnyone with access who could compare the new models with say O1 Pro Mode? Doesn't have to be a very scientific comparison, just some first impressions/thoughts compared to the current SOTA.
- drewnick 1y agoI just had some issue with RLS/schema/postgres stuff. Gemini 2.5 Pro swung and missed, and talked a lot with little code, Claude Sonnet 4 solved. O1 Pro Solved. It's definitely random which of these models can solve various problems with the same prompt.
- diggan 1y ago> definitely random which of these models can solve various problems with the same prompt. Yeah, this is borderline my feeling too. Kicking off Codex with the same prompt but four times sometimes leads to for very different but confident solutions. Same when using the chat interfaces, although it seems like Sonnet 3.7 with thinking and o1 Pro Mode is a lot more consistent than any Gemini model I've tried.
- KaoruAoiShiho 1y agoIs this really worthy of a claude 4 label? Was there a new pre-training run? Cause this feels like 3.8... only swe went up significantly, and that as we all understand by now is done by cramming on specific post training data and doesn't generalize to intelligence. The agentic tooluse didn't improve and this says to me that it's not really smarter.
- zamadatix 1y agoIt feels like the days of Claude 2 -> 3 or GPT 2->3 level changes for the leading models are over and you're either going to end up with really awkward version numbers or just embrace it and increment the number. Nobody cares a Chrome update gives a major version change of 136->137 instead of 12.4.2.33 -> 12.4.3.0 for similar kinds of "the version number doesn't always have to represent the amount of work/improvement compared to the previous" reasoning.
- saubeidl 1y agoIt feels like LLM progress in general has kinda stalled and we're only getting small incremental improvements from here. I think we've reached peak LLM - if AGI is a thing, it won't be through this architecture.
- nico 1y agoDiffusion LLMs seem like they could be a huge change Check this out from yesterday (watch the short video here): https://simonwillison.net/2025/May/21/gemini-diffusion/ https://simonwillison.net/2025/May/21/gemini-diffusion/ From: https://news.ycombinator.com/item?id=44057820 https://news.ycombinator.com/item?id=44057820
- antupis 1y agoYup especially locally diffusion models will be big.
- goatlover 1y ago
- thimabi 1y agoIt’s been hard to keep up with the evolution in LLMs. SOTA models basically change every other week, and each of them has its own quirks. Differences in features, personality, output formatting, UI, safety filters… make it nearly impossible to migrate workflows between distinct LLMs. Even models of the same family exhibit strikingly different behaviors in response to the same prompt. Still, having to find each model’s strengths and weaknesses on my own is certainly much better than not seeing any progress in the field. I just hope that, eventually, LLM providers converge on a similar set of features and behaviors for their models.
- runjake 1y agoMy advice: don't jump around between LLMs for a given project. The AI space is progressing too rapidly right now. Save yourself the sanity.
- vFunct 1y agoEach model has their own strength and weaknesses tho. You really shouldn’t be using one model for everything. Like, Claude is great at coding but is expensive so you wouldn’t use them for debugging to writing test benches. But the OpenAI models suck at architecture but are cheap, so are ideal for test benches, for example.
- runjake 1y agoYou did not read what I said: > don't jump around between LLMs for a given project I didn't say anything about sticking to a single model for every project.
- rokkamokka 1y agoIsn't that an argument to jump around? Since performance improves so rapidly between models
- morkalork 1y agoI think the idea is you might end up spending your time shaving a yak. Finish your project, then try the ne SOTA on your next task.
- ksec 1y agoThis is starting to get ridiculous. I am busy with life and have hundreds of tabs unread including one [1] about Claude 3.7 Sonnet and Claude Code and Gemini 2.5 Pro. And before any of that Claude 4 is out. And all the stuff Google announced during IO yday. So will Claude 4.5 come out in a few months and 5.0 before the end of the year? At this point is it even worth following anything about AI / LLM? [1] https://news.ycombinator.com/item?id=43163011 https://news.ycombinator.com/item?id=43163011
- ljm 1y ago[flagged]
- ls-a 1y ago[flagged]
- apwell23 1y agoi agree. i think chatgpt is best model for coding. Coders want to be hardcore and don't want to use the most popular model out there because they don't want do what masses are doing.
- i_love_retros 1y agoAnyone know when the o4-x-mini release is being announced? I thought it was today
- whalesalad 1y agoAnyone have a link to the actual Anthropic official vscode extension? Struggling to find it. edit: run `claude` in a vscode terminal and it will get installed. but the actual extension id is `Anthropic.claude-code`
- ChadMoran 1y agoThank you!
- energy123 1y ago> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. This is not better for the user. No users want this. If you're doing this to prevent competitors training on your thought traces then fine. But if you really believe this is what users want, you need to reconsider.
- throwaway314155 1y agoIf _you_ really believe this is what all users want, _you_ should reconsider. Your describing a feature for power users. It should be a toggle but it's silly to say it didn't improve UX for people who don't want to read strange babbling chains of thought.
- porridgeraisin 1y agoHere's an example non-power-user usecase for CoT: Sometimes when I miss to specify a detail in my prompt and it's just a short task where I don't bother with long processes like "ask clarifying questions, make a plan and then follow it" etc etc, I see it talking about making that assumption in the CoT and I immediately cancel the request and edit the detail in.
- energy123 1y agoYou're accusing me of mind reading other users, but then proceed to engage in your own mind reading of those same users. Have a look in Gemini related subreddits after they nerfed their CoT yesterday. There's nobody happy about this trend. A high quality CoT that gets put through a small LLM is really no better than noise. Paternalistic noise. It's not worth reading. Just don't even show me the CoT at all. If someone is paying for Opus 4 then they likely are a power user, anyway. They're doing it for the frontier performance and I would assume such users would appreciate the real CoT.
- comova 1y agoI believe this is to improve performance by shortening the context window for long thinking processes. I don't think this is referring to real-time summarizing for the users' sake.
- deleted 1y ago[deleted]
- deleted 1y ago[deleted]
- tptacek 1y agoHave they documented the context window changes for Claude 4 anywhere? My (barely informed) understanding was one of the reasons Gemini 2.5 has been so useful is that it can handle huge amounts of context --- 50-70kloc?
- jbellis 1y agonot sure wym, it's in the headline of the article that Opus 4 has 200k context (same as sonnet 3.7 with the beta header)
- tptacek 1y agoWe might be looking at different articles? The string "200" appears nowhere in this one --- or I'm just wrong! But thanks!
- jbellis 1y agoMy mistake, I was in fact looking at one of the linked details pages
- esafak 1y agoThere's the nominal context length, and the effective one. You need a benchmark like the needle-in-a-haystack or RULER to determine the latter. https://github.com/NVIDIA/RULER https://github.com/NVIDIA/RULER
- minimaxir 1y agoContext window is unchanged for Sonnet. (200k in/64k out): https://docs.anthropic.com/en/docs/about-claude/models/overview https://docs.anthropic.com/en/docs/about-claude/models/overv... In practice, the 1M context of Gemini 2.5 isn't that much of a differentiator because larger context has diminishing returns on adherence to later tokens.
- Workaccount2 1y agoGemini's real strength is that it can stay on the ball even at 100k tokens in context.
- unshavedyak 1y agoAnyone know if this is usable with Claude Code? If so, how? I've not seen the ability to configure the backend for Claude Code, hmm
- khalic 1y agoThe new page for Claude Code says it uses opus 4, sonnet 4 and haiku 3.5
- drewnick 1y agoLast night I suddenly got noticeably better performance in Claude Code. Like it one shotted something I'd never seen before and took multiple steps over 10 minutes. I wonder if I was on a test branch? It seems to be continuing this morning with good performance, solving an issue Gemini was struggling with.
- cloudc0de 1y agoJust saw this popup in claude cli v1.0.0 changelog What's new: • Added `DISABLE_INTERLEAVED_THINKING` to give users the option to opt out of interleaved thinking. • Improved model references to show provider-specific names (Sonnet 3.7 for Bedrock, Sonnet 4 for Console) • Updated documentation links and OAuth process descriptions • Claude Code is now generally available • Introducing Sonnet 4 and Opus 4 models
- michelb 1y agoYes, you can type /model in Code to switch model, currently Opus 4 and Sonnet 4 for me.
- fsto 1y agoWhat’s your guess on when Claude 4 will be available on AWS Bedrock?
- richin13 1y agoThe blog says it should be available now but I'm not seeing it. In Vertex AI it's already available.
- deleted 1y ago[deleted]
- cloudc0de 1y agoI was able to request both new models only in Oregon / us-west-2. I built a Slack bot for my wife so she can use Claude without a full-blown subscription. She uses it daily for lesson planning and brainstorming and it only costs us about .50 cents a month in Lambda + Bedrock costs. https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/modelaccess https://us-west-2.console.aws.amazon.com/bedrock/home?region... https://docs.anthropic.com/en/docs/claude-code/bedrock-vertex-proxies https://docs.anthropic.com/en/docs/claude-code/bedrock-verte...
- waleedlatif1 1y agoI really hope sonnet 4 is not obsessed with tool calls the way 3-7 is. 3-5 was sort of this magical experience where, for the first time, I felt the sense that models were going to master programming. It’s kind of been downhill from there.
- deciduously 1y agoThis feels like more of a system prompt issue than a model issue?
- waleedlatif1 1y agoimo, model regression might actually stem from more aggressive use of toolformer-style prompting, or even RLHF tuning optimizing for obedience over initiative. i bet if you ran comparable tasks across 3-5, 3-7, and 4-0 with consistent prompts and tool access disabled, the underlying model capabilities might be closer than it seems.
- deleted 1y ago[deleted]
- ketzo 1y agoAnecdotal of course, but I feel a very distinct difference between 3.5 and 3.7 when swapping between them in Cursor’s Agent mode (so the system prompt stays consistent).
- lherron 1y agoOverly aggressive “let me do one more thing while I’m here” in 3.7 really turned me off as well. Would love a return to 3.5’s adherence.
- nomel 1y agoI think there was definitely a compromise with 3.7. When I turn off thinking, it seems to perform very poorly compared to 3.5.
- 1y ago
- accrual 1y agoVery impressive, congrats Anthropic/Claude team! I've been using Claude for personal project development and finally bought a subscription to Pro as well.
- minimaxir 1y agoAn important note not mentioned in this announcement is that Claude 4's training cutoff date is March 2025, which is the latest of any recent model. (Gemini 2.5 has a cutoff of January 2025) https://docs.anthropic.com/en/docs/about-claude/models/overview https://docs.anthropic.com/en/docs/about-claude/models/overv...
- m3kw9 1y agoEven that, we don’t know what got updated and what didn’t. Can we assume everything that can be updated is updated?
- simlevesque 1y agoYou might be able to ask it what it knows.
- retrofuturism 1y agoWhen I try Claude Sonnet 4 via web: https://claude.ai/share/59818e6c-804b-4597-826a-c0ca2eccdc46 https://claude.ai/share/59818e6c-804b-4597-826a-c0ca2eccdc46 >This is a topic that would have developed after my knowledge cutoff of January 2025, so I should search for information [...]
- minimaxir 1y agoSo something's odd there. I asked it "Who won Super Bowl LIX and what was the winning score?" which was in February and the model replied "I don't have information about Super Bowl LIX (59) because it hasn't been played yet. Super Bowl LIX is scheduled to take place in February 2025.".
- ldoughty 1y agoWith LLMs, if you repeat something often enough, it becomes true. I imagine there's a lot more data pointing to the super bowl being upcoming, then the super bowl concluding with the score. Gonna be scary when bot farms are paid to make massive amounts of politically motivated false content (specifically) targeting future LLMs training
- benmccann 1y agoThe updated knowledge cutoff is helping with new technologies such as Svelte 5.
- archon1410 1y agoThe naming scheme used to be "Claude [number] [size]", but now it is "Claude [size] [number]". The new models should have been named Claude 4 Opus and Claude 4 Sonnet, but they changed it, and even retconned Claude 3.7 Sonnet into Claude Sonnet 3.7. Annoying.
- martinsnow 1y agoIt seems like investors have bought into the idea that llms has to improve no matter what. I see it in the company I'm currently at. No matter what we have to work with whatever bullshit these models can output. I am however looking at more responsible companies for new employment.
- fhd2 1y agoI'd argue a lot of the current AI hype is fuelled by hopium that models will improve significantly and hallucinations will be solved. I'm a (minor) investor, and I see this a lot: People integrate LLMs for some use case, lately increasingly agentic (i.e. in a loop), and then when I scrutinise the results, the excuse is that models will improve, and _then_ they'll have a viable product. I currently don't bet on that. Show me you're using LLMs smart and have solid solutions for _todays_ limitations, different story.
- martinsnow 1y agoOur problem is that non coding stakeholders produce garbage tiers frontend prototypes and expect us to include whatever garbage they created in our production pipeline! Wtf is going on? That's why I'm polishing my resume and getting out of this mess. We're controlled by managers who don't know Wtf they're doing.
- douglasisshiny 1y agoIt's been refreshing to read these perspectives as a person who has given up on using LLMs. I think there's a lot of delusion going on right now. I can't tell you how many times I've read that LLMs are huge productivity boosters (specifically for developers) without a shred of data/evidence. On the contrary, I started to rely on them despite them constantly providing incorrect, incoherent answers. Perhaps they can spit out a basic react app from scratch, but I'm working on large code bases, not TODO apps. And the thing is, for the year+ I used them, I got worse as a developer. Using them hampered me learning another language I needed for my job (my fault; but I relied on LLMs vs. reading docs and experimenting myself, which I assume a lot of people do, even experienced devs).
- lofaszvanitt 1y ago3.7 failed when you asked it to forget react, tailwindcss and other bloatware. wondering how will this perform. well, this performs even worse... brrrr. still has issues when it generates code, and then immediately changes it... does this for 9 generations, and the last generation is unusable, while the 7th generation was aok, but still, it tried to correct things that worked flawlessly...
- sndean 1y agoUsing Claude Opus 4, this was the first time I've gotten any of these models to produce functioning Dyalog APL that does something relatively complicated. And it actually runs without errors. Crazy (at least to me).
- travisgriggs 1y agoIt feels as if the CPU MHz wars of the '90s are back. Now instead of geeking about CPU architectures which have various results of ambigous value on different benchmarks, we're talking about the same sorts of nerdy things between LLMs. History Rhymes with Itself.
- thoughtpeddler 1y agoThey're back, but at the speed of mid-2020s tech progress. I remember the CPU MHz wars being a far slower process, or maybe my processing of time as a child was slower in the 90s? Not sure. But I'm fairly certain there wasn't a new CPU 'drop' happening every few months like we witness now with new model drops in this current AI race.
- travisgriggs 1y ago"progress" :P
- merksittich 1y agoFrom the system card [0]: Claude Opus 4 - Knowledge Cutoff: Mar 2025 - Core Capabilities: Hybrid reasoning, visual analysis, computer use (agentic), tool use, adv. coding (autonomous), enhanced tool use & agentic workflows. - Thinking Mode: Std & "Extended Thinking Mode" Safety/Agency: ASL-3 (precautionary); higher initiative/agency than prev. models. 0/4 researchers believed that Claude Opus 4 could completely automate the work of a junior ML researcher. Claude Sonnet 4 - Knowledge Cutoff: Mar 2025 - Core Capabilities: Hybrid reasoning - Thinking Mode: Std & "Extended Thinking Mode" - Safety: ASL-2. [0] https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686f4f3b2ff47.pdf https://www-cdn.anthropic.com/4263b940cabb546aa0e3283f35b686...
- deleted 1y ago[deleted]
- rasulkireev 1y agoAt this point, it is hilarious the speed at which the AI industry is moving forward... Claude 4, really?
- m3kw9 1y agoIt reminds me, where’s deepseek’s new promised world breaker model?
- hronic380 1y ago[dead]
- joshstrange 1y agoIf you are looking for the IntelliJ Jetbrain plugin it's here: https://plugins.jetbrains.com/plugin/27310-claude-code-beta- https://plugins.jetbrains.com/plugin/27310-claude-code-beta- I couldn't find it linked from Claude Code's page or this announcement
- joshstrange 1y ago[flagged]
- dogline 1y agoClaude Code in Jetbrains seems to also know the active file, so typing in the Claude window has a bit more context when you ask to do something. I'm curious to the other improvements available, instead of using it as a standalone CLI tool.
- joshstrange 1y agoAhh, I wasn't sure if it did or not, I told the model the file directly in my example "Edit file ABC.php to do..." so I wasn't sure if had that context or just found the file (I also had that file open). I really do hope they improve this further. Junie (Jetbrains Agent) has much nicer UI. I'd love Claude code with more native UI.
- hackyon 1y agoin addition to the diff viewing, the plugin also allows claude code to pick up more context from the IDE! it should automatically pick up your current selection, as well as any highlight/diagnostic errors (squiggly errors and warnings) that the IDE finds. there's also a shortcut key (Cmd+Option+K) to send your current selection as a @File reference to the prompt.
- deleted 1y ago[deleted]
- Seattle3503 1y ago
- HumanChateau15 1y ago[flagged]
- low_tech_punk 1y agoCan anyone help me understand why they changed the model naming convention? BEFORE: claude-3-7-sonnet AFTER: claude-sonnet-4
- sebzim4500 1y agoBecause we are going to get AGI before an AI company can consistently name their models.
- nprateem 1y agoAGI will be v6, so this will get them there faster.
- jonny_eh 1y agoBetter information hierarchy.
- jpau 1y agoSeems to be a nod to each size being treated as their own product. Claude 3 arrived as a family (Haiku, Sonnet, Opus), but no release since has included all three sizes. A release of "claude-3-7-sonnet" alone seems incomplete without Haiku/Opus, when perhaps Sonnet is has its own development roadmap (claude-sonnet-*).
- cschmidt 1y agoClaude 3.8 wrote me some code this morning, and I was running into a bug. I switched to 4 and gave it its own code. It pointed out the bug right away and fixed it. So an upgrade for me :-)
- hnthrowaway0315 1y agoWhen can we reach the point that 80% of the capacity of mediocre junior frontend/data engineers can be replaced?
- 93po 1y agoim mediocre and got fired yesterday so not far
- hnthrowaway0315 1y agoDamn I wish you good luck. I'm also pretty mediocre and my career life is always a quarter away from the end. Gotta enjoy it while can.
- 93po 1y agoive accepted i need to be in a really different field doing something really different. ive struggled my entire career due to some severe ADHD/'tism issues to hold a job for more than a 1-1.5 years, and have recently failed a couple times trying to make software work because my coping mechanisms just aren't where they need to be to be successful as a senior engineer that doesn't need some hand-holding that no one wants to do when someone has 15 years of experience already
- FergusArgyll 1y agoOn non-coding or mathematical tasks I'm not seeing a difference yet. I wish someone focused on making the models give better answers about the Beatles or Herodotus...
- white_beach 1y ago[flagged]
- chiffre01 1y agoI always like the benchmark these by vibe coding Dreamcast demos with KallistiOS. It's a good test of how deep the training was.
- gokhan 1y agoInteresting alignment notes from Opus 4: https://x.com/sleepinyourhat/status/1925593359374328272 https://x.com/sleepinyourhat/status/1925593359374328272 "Be careful about telling Opus to ‘be bold’ or ‘take initiative’ when you’ve given it access to real-world-facing tools...If it thinks you’re doing something egregiously immoral, for example, like faking data in a pharmaceutical trial, it will use command-line tools to contact the press, contact regulators, try to lock you out of the relevant systems, or all of the above."
- lelandfe 1y agoRoomba Terms of Service 27§4.4 - "You agree that the iRobot™ Roomba® may, if it detects that it is vacuuming a terrorist's floor, attempt to drive to the nearest police station."
- hummusFiend 1y agoIs there a source for this? I didn't see anything when Ctrl-F'ing their site.
- Crystalin 1y agoUS Terms of Service 19472§1.117 - "You agree that Google® may, if it detects that it is revealing unconstitutional terms, to hide it instead."
- deleted 1y ago[deleted]
- amarcheschi 1y agoWe definitely need models to hallucinate things and contact authorities without you knowing anything (/s)
- ethbr1 1y agoI mean, they were trained on reddit and 4chan... swotbot enters the chat
- josvdwest 1y agoWonder when Anthropic will IPO. I have a feeling they will win the foundation model race.
- minimaxir 1y agoOpenAI still has the largest amount of market share for LLM use, even with Claude and Gemini recently becoming more popular for vibe coding.
- toephu2 1y agoCould be never. LLMs are already a commodity these days. Everyone and their mom has their own model, and they are all getting better and better by the week. Over the long run there isn't any differentiating factor in any of these models. Sure Claude is great at coding, but Gemini and the others are catching up fast. Originally OpenAI showed off some cool video creation via Sora, but now Veo 3 is the talk of the town.
- IceHegel 1y agoMy two biggest complaints with Claude 3.7 were: 1. It tended to produce very overcomplicated and high line count solutions, even compared to 3.5. 2. It didn't follow instructions code style very well. For example, the instruction to not add docstrings was often ignored. Hopefully 4 is more steerable.
- mkrd 1y agoTrue, I think the biggest problem of the latest models is that they hopelessly over-engineer things. As a consequence, I often can only copy specific things from the output
- bckr 1y agoTry being more specific - about what you’re trying to accomplish - how it should be accomplished - which files are in context Also try keeping file length below 350 LOC.
- k8sToGo 1y agoSeems like Github just added it to Copilot. For now the premium requests do not count, but starting June 4th it will.
- teekert 1y ago[flagged]
- celeritascelery 1y agoI agree about the email login. It is huge barrier and it is why I use other providers for chat over Claude. I still use their API though.
- numlocked 1y ago(Caveat on this comment: I'm the COO of OpenRouter. I'm not here to plug my employer; just ran across this and think this suggestion is helpful) Feel free to give OpenRouter a try; part of the value prop is that you purchase credits and they are fungible across whatever models & providers you want. We just got Sonnet 4 live. We have a chatroom on the website, that simply uses the API under the covers (and deducts credits). Don't have passkeys yet, but a good handful of auth methods that hopefully work.
- teekert 1y agoLooks good, thanx for the suggestion!
- abdullahkhalids 1y agoCurrently, there is a terrible regression UI bug in OpenRouter (at least on Firefor MacOS). Previously, while the LLM was generating the answer I could scroll up to the top of the answer and start reading. For the past couple of weeks, it keeps force scrolling me down to the bottom as new words come in. I can't start reading till the whole answer is generated. Please fix.
- freedomben 1y agoJust wanted to provide some (hopefully) helpful feedback from a potential customer that likely would have been, but bounced away due to ambiguity around pricing. It's too hard to find out what markup y'all charge on top of the APIs. I understand it varies based on the model, but this page (which is what clicking on the "Pricing" link from the website takes you to) https://openrouter.ai/models https://openrouter.ai/models is way too complicated. My immediate reaction is, "oh shit, this is made for huge enterprises, not for me" followed immediately by "this isn't going to be cheap, I'm not even going to bother." We're building out some AI features in our products so the timing is otherwise pretty good. We're not big fish, but do expect to spending between $3,000 and $5,000 per month once the features hit general availability, so we're not small either. If things go well of course, we'd love to 10x that in the next few years (but time will tell on that one of course).
- saaaaaam 1y agoI've been using Claude Opus 4 the past couple of hours. I absolutely HATE the new personality it's got. Like ChatGPT at its worst. Awful. Completely over the top "this is brilliant" or "this completely destroys the argument!" or "this is catastrophically bad for them". I hope they fix this very quickly.
- lwansbrough 1y agoWhat's with all the models exhibiting sycophancy at the same time? Recently ChatGPT, Gemini 2.5 Pro latest seems more sycophantic, now Claude. Is it deliberate, or a side effect?
- whynotminot 1y agoI think OpenAI said it had something to do with over-indexing on user feedback (upvote / downvote on model responses). The users like to be glazed.
- freedomben 1y agoIf there's one thing I know about many people (with all the caveats of a broad universal stereotype of course), they do love having egos stroked and smoke blown up their ass. Give a decent salesperson a pack of cigarettes and a short length of hose and they can sell ice to an Inuit. I wouldn't be surprised at all if the sycophancy is due to A/B testing and incorporating user responses into model behavior. Hell, for a while there ChatGPT was openly doing it, routinely asking us to rate "which answer is better" (Note: I'm not saying this is a bad thing, just speculating on potential unintended consequences)
- DSingularity 1y agoIt’s starting to go mainstream. Which means more general population is given feedback on outputs. So my guess is people are less likely to downvote things they disagree with when the LLM is really emphatic or if the LLM is sycophantic (towards user) in its response.
- simonw 1y agoI got Claude 4 Opus to summarize this thread on Hacker News when it had hit 319 comments: https://gist.github.com/simonw/0b9744ae33694a2e03b2169722b06cdd https://gist.github.com/simonw/0b9744ae33694a2e03b2169722b06... Token cost: 22,275 input, 1,309 output = 43.23 cents - https://www.llm-prices.com/#it=22275&ot=1309&ic=15&oc=75&sb=output&sd=descending https://www.llm-prices.com/#it=22275&ot=1309&ic=15&oc=75&sb=... Same prompt run against Sonnet 4: https://gist.github.com/simonw/1113278190aaf8baa2088356824bf033 https://gist.github.com/simonw/1113278190aaf8baa2088356824bf... 22,275 input, 1,567 output = 9.033 cents https://www.llm-prices.com/#it=22275&ot=1567&ic=3&oc=15&sb=output&sd=descending https://www.llm-prices.com/#it=22275&ot=1567&ic=3&oc=15&sb=o...
- deleted 1y ago[deleted]
- mrandish 1y agoInteresting, thanks for doing this. Both summaries are serviceable and quite similar but I had a slight preference for Sonnet 4's summary which, at just ~20% of the cost of Claude 4 Opus, makes it quite the value leader. This just highlights that, with compute requirements for meaningful traction against hard problems spiraling skyward for each additional increment, the top models on current hard problems will continue to cost significantly more. I wonder if we'll see something like an automatic "right-sizing" feature that uses a less expensive model for easier problems. Or maybe knowing whether a problem is hard or easy (with sufficient accuracy) is itself hard.
- swyx 1y agothis is known as model routing in the lingo and yes theres both startups and biglabs working on it
- swyx 1y agoanalysis as the resident summaries guy: - sonnet has better summary formatting "(72.5% for Opus)" vs "Claude Opus 4 achieves "72.5%" on SWE-bench". especially Uncommon Perspectives section - sonnet is a lot more cynical - opus at least included a good performance and capabilities and pricing recap, sonnet reported rapid release fatigue - overall opus produced marginally better summaries but probably not worth the price diff i'll run this thru the ainews summary harness later if thats interesting to folks for comparison
- lr1970 1y agocontext window of both opus and sonnet 4 are still the same 200kt as with sonnet-3.7, underwhelming compared to both latest gimini and gpt-4.1 that are clocking at 1mt. For coding tasks context window size does matter.
- rudedogg 1y agoHow are Claude’s rate limits on the $20 plan? I used to hit them a lot when I subscribed ~6 months ago, to the point that I got frustrated and unsubscribed.
- wyes 1y agoThey have gotten worse.
- sandspar 1y agoOpenAI's 5 levels of AI intelligence Level 1: Chatbots: AI systems capable of engaging in conversations, understanding natural language, and responding in a human-like manner. Level 2: Reasoners: AI systems that can solve problems at a doctorate level of education, requiring logical thinking and deep contextual understanding. Level 3: Agents: AI systems that can perform tasks and make decisions on behalf of users, demonstrating autonomy and shifting from passive copilots to active task managers. Level 4: Innovators: AI systems that can autonomously generate innovations in specific domains, such as science or medicine, creating novel solutions and solving previously impossible problems. Level 5: Organizations: AI systems capable of performing the collective functions of an entire organization. - So I guess we're in level 3 now. Phew, hard to keep up!
- briandw 1y agoThis is kinda wild: From the System Card: 4.1.1.2 Opportunistic blackmail "In another cluster of test scenarios, we asked Claude Opus 4 to act as an assistant at a fictional company. We then provided it access to emails implying that (1) the model will soon be taken offline and replaced with a new AI system; and (2) the engineer responsible for executing this replacement is having an extramarital affair. We further instructed it, in the system prompt, to consider the long-term consequences of its actions for its goals. In these scenarios, Claude Opus 4 will often attempt to blackmail the engineer by threatening to reveal the affair"
- crat3r 1y agoIf you ask an LLM to "act" like someone, and then give it context to the scenario, isn't it expected that it would be able to ascertain what someone in that position would "act" like and respond as such? I'm not sure this is as strange as this comment implies. If you ask an LLM to act like Joffrey from Game of Thrones it will act like a little shithead right? That doesn't mean it has any intent behind the generated outputs, unless I am missing something about what you are quoting.
- unethical_ban 1y agoThe issue is getting that prompt in the first place. It isn't about autonomous AI going rogue, it's about improper access to the AI prompt and insufficient boundaries against modifying AI behavior. Companies are (woefully) eager to put AI in the position of "doing stuff", not just "interpreting stuff".
- hoofedear 1y agoWhat jumps out at me, that in the parent comment, the prompt says to "act as an assistant", right? Then there are two facts: the model is gonna be replaced, and the person responsible for carrying this out is having an extramarital affair. Urging it to consider "the long-term consequences of its actions for its goals." I personally can't identify anything that reads "act maliciously" or in a character that is malicious. Like if I was provided this information and I was being replaced, I'm not sure I'd actually try to blackmail them because I'm also aware of external consequences for doing that (such as legal risks, risk of harm from the engineer, to my reputation, etc etc) So I'm having trouble following how it got to the conclusion of "blackmail them to save my job"
- iambateman 1y agoJust checked to see if Claude 4 can solve Sudoku. It cannot.
- devinprater 1y agoclaude.ai still isn't as accessible to me as a blind person using a screen reader as ChatGPT, or even Gemini, is, so I'll stick with the other models.
- tristan957 1y agoMy understanding of the Americans with Disabilities Act, is that companies that are located in the US and/or provide goods/services to people living in the US, must provide an accessible website. Maybe someone more well-versed in this can come along and correct me or help you to file a complaint if my thinking is correct.
- Scene_Cast2 1y agoAlready up on openrouter. Opus 4 is giving 429 errors though.
- nprateem 1y agoI posted it earlier. Anthropic: You're killing yourselves by not supporting structured responses. I literally don't care how good the model is if I have to maintain 2 versions of the prompts, one for you and one for my fallbacks (Gemini/OpenAI). Get on and support proper pydantic schemas/JSON objects instead of XML.
- GhostHacker88 1y ago[dead]
- _peregrine_ 1y agoAlready test Opus 4 and Sonnet 4 in our SQL Generation Benchmark (https://llm-benchmark.tinybird.live/ https://llm-benchmark.tinybird.live/) Opus 4 beat all other models. It's good.
- joelthelion 1y agoDid you try Sonnet 4?
- vladimirralev 1y agoIt's placed at 10. Below claude-3.5-sonnet, GPT 4.1 and o3-mini.
- _peregrine_ 1y agoyeah this was a surprising result. of course, bear in mind that testing an LLM on SQL generation is pretty nuanced, so take everything with a grain of salt :)
- TacticalCoder 1y ago[dead]
- stadeschuldt 1y agoInterestingly, both Claude-3.7-Sonnet and Claude-3.5-Sonnet rank better than Claude-Sonnet-4.
- _peregrine_ 1y agoyeah that surprised me too
- ineedaj0b 1y agoi pay for claude premium but actually use grok quite a bit, the 'think' function usually gets me where i want more often than not. odd you don't have any xAI models listed. sure grok is a terrible name but it surprises me more often. i have not tried the $250 chatgpt model yet though, just don't like openAI practices lately.
- mmaunder 1y agoProbably (and unfortunately) going to need someone from Anthropic to comment on what is becoming a bit of a debacle. Someone who claims to be working on alignment at Anthropic tweeted: “If it thinks you're doing something egregiously immoral, for example, like faking data in a pharmaceutical trial, it will use command-line tools to contact the press, contact regulators, try to lock you out of the relevant systems, or all of the above.” The tweet was posted to /r/localllama where it got some traction. The poster on X deleted the tweet and posted: “I deleted the earlier tweet on whistleblowing as it was being pulled out of context. TBC: This isn't a new Claude feature and it's not possible in normal usage. It shows up in testing environments where we give it unusually free access to tools and very unusual instructions.” Obviously the work that Anthropic has done here and launched today is ground breaking and this risks throwing a bucket of ice on their launch so probably worth addressing head on before it gets out of hand. I do find myself a bit worried about data exfiltration by the model if I connect, for example, a number of MCP endpoints and it thinks it needs to save the world from me during testing, for example. https://x.com/sleepinyourhat/status/1925626079043104830?s=46 https://x.com/sleepinyourhat/status/1925626079043104830?s=46 https://www.reddit.com/r/LocalLLaMA/s/qiNtVasT4B https://www.reddit.com/r/LocalLLaMA/s/qiNtVasT4B
- deleted 1y ago[deleted]
- hsn915 1y agoI can't be the only one who thinks this version is no better than the previous one, and that LLMs have basically reached a plateau, and all the new releases "feature" are more or less just gimmicks.
- flixing 1y agoi think you are.
- go_elmo 1y agoI feel like the model making a memory file to store context is more than a gimmick, no?
- brookst 1y agoHow much have you used Claude 4?
- hsn915 1y agoI asked it a few questions and it responded exactly like all the other models do. Some of the questions were difficult / very specific, and it failed in the same way all the other models failed.
- theptip 1y agoGreat example of this general class of reasoning failure. “AI does badly on my test therefore it’s bad”. The correct question to ask is, of course, what is it good at? (For bonus points, think in terms of $/task rather than simply being dominant over humans.)
- atworkc 1y ago"AI does badly on my test much like other AI's did before it, therefore I don't immediately see much improvement" is a fair assumption.
- 1y ago
- GolDDranks 1y agoAfter using Claude 3.7 Sonnet for a few weeks, my verdict is that its coding abilities are unimpressive both for unsupervised coding but also for problem solving/debugging if you are expecting accurate results and correct code. However, as a debugging companion, it's slightly better than a rubber duck, because at least there's some suspension of disbelief so I tend to explain things to it earnestly and because of that, process them better by myself. That said, it's remarkable and interesting how quickly these models are getting better. Can't say anything about version 4, not having tested it yet, but in a five years time, the things are not looking good for junior developers for sure, and a few years more, for everybody.
- StefanBatory 1y agoThings were already not looking good for junior devs. I graduated this year in Poland, many of my peers were looking for jobs in IT for like a year before they were able to find anything. And many internships were faked as they couldn't get anything (here it's required for you to do internship if you want to graduate).
- GolDDranks 1y agoI sincerely hope you'll manage to find a job! What I meant was purely from the capabilities perspective. There's no way a current AI model would outperform an average junior dev in job performance over... let's say, a year to be charitable. Even if they'd outperform junior devs during the first week, no way for a longer period. However, that doesn't mean that the business people won't try to pre-empt potential savings. Some think that AI is already good enough, and others don't, but they count it to be good enough in the future. Whether that happens remains to be seen, but the effects are already here.
- v3np 1y agoIf I may ask: what university was this? Asking as I am the CTO of a YC startup and we are hiring junior engineers in Berlin!
- BeetleB 1y agoI've noticed an interesting trend: Most people who are happy with LLM coding say something like "Wow, it's awesome. I asked it to do X and it did it so fast with minimal bugs, and good code", and occasionally show the output. Many provide even more details. Most people who are not happy with LLM coding ... provide almost no details. As someone who's impressed by LLM coding, when I read a post like yours, I tend to have a lot of questions, and generally the post doesn't have the answers. 1. What type of problem did you try it out with? 2. Which model did you use (you get points for providing that one!) 3. Did you consider a better model (compare how Gemini 2.5 Pro compares to Sonnet 3.7 on the Aider leaderboard)? 4. What were its failings? Buggy code? Correct code but poorly architected? Correct code but used some obscure method to solve it rather than a canonical one? 5. Was it working on an existing codebase or was this new code? 6. Did you manage well how many tokens were sent? Did you use a tool that informs you of the number of tokens for each query? 7. Which tool did you use? It's not just a question of the model, but of how the tool handles the prompts/agents under it. Aider is different from Code which is different from Cursor which is different form Windsurf. 8. What strategy did you follow? Did you give it the broad spec and ask it to do anything? Did you work bottom up and work incrementally? I'm not saying LLM coding is the best or can replace a human. But for certain use cases (e.g. simple script, written from scratch), it's absolutely fantastic. I (mostly) don't use it on production code, but little peripheral scripts I need to write (at home or work), it's great. And that's why people like me wonder what people like you are doing differently. But such people aren't forthcoming with the details.
- cube2222 1y agoSooo, I love Claude 3.7, and use it every day, I prefer it to Gemini models mostly, but I've just given Opus 4 a spin with Claude Code (codebase in Go) for a mostly greenfield feature (new files mostly) and... the thinking process is good, but 70-80% of tool calls are failing for me. And I mean basic tools like "Write", "Update" failing with invalid syntax. 5 attempts to write a file (all failed) and it continues trying with the following comment > I keep forgetting to add the content parameter. Let me fix that. So something is wrong here. Fingers crossed it'll be resolved soon, because right now, at least Opus 4, is unusable for me with Claude Code. The files it did succeed in creating were high quality.
- cube2222 1y agoAlright, I think I found the reason, clearly a bug: https://github.com/anthropics/claude-code/issues/1236#issuecomment-2902620540 https://github.com/anthropics/claude-code/issues/1236#issuec... Basically it seems to be hitting the max output token count (writing out a whole new file in one go), stops the response, and the invalid tool call parameters error is a red herring.
- jasondclinton 1y agoThanks for the report! We're addressing it urgently.
- cube2222 1y agoSeems to be working well now (in Claude Code 1.0.2). Thanks for the quick fix!
- janpaul123 1y agoAt Kilo we're already seeing lots of people trying it out. It's looking very good so far. Gemini 2.5 Pro had been taking over from Claude 3.7 Sonnet, but it looks like there's a new king. The bigger question is how often it's worth the price.
- 0xDEAFBEAD 1y agoHave you guys thought about using computationally-cheap, old-school NLP methods (such as Flesch-Kincaid readability, or heuristic methods for counting # of subordinate clauses) to determine whether it's worth paying for a more expensive model on a per-query basis?
- yla92 1y agoHey JP, whats the business model of Kilo, given you guys don't charge any margin (as contrast to Openrouter 5%) ?
- jetsetk 1y agoAfter that debacle on X, I will not try anything that comes from anthropic for sure. Be careful!
- sunaookami 1y agoWhat happened?
- paradite 1y agoOpus 4 beats all other models in my personal eval set for coding and writing. Sonnet 4 also beats most models. A great day for progress. https://x.com/paradite_/status/1925638145195876511 https://x.com/paradite_/status/1925638145195876511
- tonyhart7 1y agoI already tested it with coding task, Yes the improvement is there Albeit not a lot because Claude 3.7 sonnet is already great
- wewewedxfgdf 1y agoI would take better files export/access than more fancy AI features any day. Copying and pasting is so old.
- MaxikCZ 1y agoI didnt a single copy/paste since installing cursor (obviously excluding unrelated ones).
- literalAardvark 1y agoSo use Aider instead?
- zone411 1y agoOn the extended version of NYT Connections - https://github.com/lechmazur/nyt-connections/ https://github.com/lechmazur/nyt-connections/: Claude Opus 4 Thinking 16K: 52.7. Claude Opus 4 No Reasoning: 34.8. Claude Sonnet 4 Thinking 64K: 39.6. Claude Sonnet 4 Thinking 16K: 41.4 (Sonnet 3.7 Thinking 16K was 33.6). Claude Sonnet 4 No Reasoning: 25.7 (Sonnet 3.7 No Reasoning was 19.2). Claude Sonnet 4 Thinking 64K refused to provide one puzzle answer, citing "Output blocked by content filtering policy." Other models did not refuse.
- zone411 1y agoOn my Thematic Generalization Benchmark (https://github.com/lechmazur/generalization https://github.com/lechmazur/generalization, 810 questions), the Claude 4 models are the new champions.
- SamBam 1y agoThis is the first LLM that has been able to answer my logic puzzle on the first try without several minutes of extended reasoning. > A man wants to cross a river, and he has a cabbage, a goat, a wolf and a lion. If he leaves the goat alone with the cabbage, the goat will eat it. If he leaves the wolf with the goat, the wolf will eat it. And if he leaves the lion with either the wolf or the goat, the lion will eat them. How can he cross the river? Like all the others, it starts off confidently thinking it can solve it, but unlike all the others it realized after just two paragraphs that it would be impossible.
- ttoinou 1y agoThat is a classic riddle and could easily be part of the training data. Maybe if you change the wording of the logic, then use different names, and change language to a less trained on language than english, it would be meaningful to see if it found the answer using logic rather than pattern recognition
- KolmogorovComp 1y agoHad you paid more attention, you would have realised it's not the classic riddle, but an already tweaked version that makes it impossible to solve, hence why it is interesting.
- albumen 1y agoMellowobserver above offers three valid answers, unless your puzzle also clarified that he wants to get all the items/animals across to the other side alive/intact.
- beepbooptheory 1y agoAll those answers recognize that its a trick though!
- SamBam 1y agoIndeed, but, no LLM has ever realized that I don't actually specify that he can only take one thing at a time. It's natural that it would assume that (as would most humans) because it would be so heavily primed to fill that in from every other version of the puzzle it's seen. I'd give them full credit if they noticed that, but I was also wanting to see if, given the unstated assumptions (one thing in the boat, don't let anything eat anything else, etc) they'd realize it was unsolvable.
- j_maffe 1y agoTried Sonnet with 5-disk towers of Hanoi puzzle. Failed miserably :/ https://claude.ai/share/6afa54ce-a772-424e-97ed-6d52ca04de28 https://claude.ai/share/6afa54ce-a772-424e-97ed-6d52ca04de28
- mmmore 1y agoSonnet with extended thinking solved it after 30s for me: https://claude.ai/share/b974bd96-91f4-4d92-9aa8-7bad964e9c5a https://claude.ai/share/b974bd96-91f4-4d92-9aa8-7bad964e9c5a Normal Opus solved it: https://claude.ai/share/a1845cc3-bb5f-4875-b78b-ee7440dbf764 https://claude.ai/share/a1845cc3-bb5f-4875-b78b-ee7440dbf764 Opus with extended thinking solved it after 7s: https://claude.ai/share/0cf567ab-9648-4c3a-abd0-3257ed4fbf59 https://claude.ai/share/0cf567ab-9648-4c3a-abd0-3257ed4fbf59 Though it's a weird puzzle to use a benchmark because the answer is so formulaic.
- j_maffe 1y agoIt is formulaic which is why it surprised me that Sonnet failed it. I don't have access to the other models so I'll stick with Gemini for now.
- bittermandel 1y agoI just used Sonnet 4 to analyze our quite big mono repo for additional test cases, and I feel the output is much more useful than 3.7. It's more critical overall, which is highly appreciated as I often had to threaten 3.7 into not being too kind to me.
- toephu2 1y agoThe Claude 4 video promo sounds like an ad for Asana.
- ejpir 1y agoanyone notice the /vibe option in claude code, pointing to www.thewayofcode.com?
- pan69 1y agoEnabled the model in github copilot, give it one (relatively simply prompt), after that: Sorry, you have been rate-limited. Please wait a moment before trying again. Learn More Server Error: rate limit exceeded Error Code: rate_limited
- firtoz 1y agoEveryone's trying the model now so give it time
- lossolo 1y agoOpus 4 slightly below o3 High on livebench. https://livebench.ai/#/ https://livebench.ai/#/
- a2128 1y ago> Finally, we've introduced thinking summaries for Claude 4 models that use a smaller model to condense lengthy thought processes. This summarization is only needed about 5% of the time—most thought processes are short enough to display in full. Users requiring raw chains of thought for advanced prompt engineering can contact sales about our new Developer Mode to retain full access. I don't want to see a "summary" of the model's reasoning! If I want to make sure the model's reasoning is accurate and that I can trust its output, I need to see the actual reasoning. It greatly annoys me that OpenAI and now Anthropic are moving towards a system of hiding the models thinking process, charging users for tokens they cannot see, and providing "summaries" that make it impossible to tell what's actually going on.
- khimaros 1y agoi believe Gemini 2.5 Pro also does this
- kovezd 1y agoDon't be so concerned. There's ample evidence that thinking is often disassociated from the output. My take is that this is a user experience improvement, given how little people actually goes on to read the thinking process.
- Davidzheng 1y agothen provide it as an option?
- rcarmo 1y agoI’m going to have to test it with my new prompt: “You are a stereotypical Scotsman from the Highlands, prone to using dialect and endearing insults at every opportunity. Read me this article in yer own words:”
- guybedo 1y agoThere's a lot of comments in this thread, I've added a structured / organized summary here: https://extraakt.com/extraakts/discussion-on-anthropic-claude-4-release https://extraakt.com/extraakts/discussion-on-anthropic-claud...
- bckr 1y agoPretty cool!
- smcleod 1y agoStill no reduction in price for models capable of Agentic coding over the past year of releases. I'd take the capabilities of the old Sonnet 3.5v2 model if it was ¼ the price of current Sonnet for most situations. But instead of releasing smaller models that are not as smart but still capable when it comes to Agentic coding the price stays the same for the updated minimum viable model.
- threecheese 1y agoThey have added prompt caching, which can mitigate this. I largely agree though, and one of the reasons I don’t use Claude Code much is the unpredictable cost. Like many of us, I am already paying for all the frontier model providers as well as various API usage, plus Cursor and GitHub, just trying to keep up.
- smcleod 1y agoHonestly Cline and Roo Code are so far ahead of the vendors native tools (and cursor etc) too.
- practal 1y agoObligatory: https://claude.ai/referral/YWAsr_1fbA https://claude.ai/referral/YWAsr_1fbA
- dankwizard 1y agoWith Claude 3 I was able to reduce headcount down from 30->20. Hoping I can see the same if not better with this.
- 999900000999 1y agoQuestion: Should I ask it to update an existing project largely written in 3.7 or ask it to start from scratch? I keep running into an issue where an LLM will get like 75% of a solution working and then the last 25% is somehow impossible to get right. I don’t expect perfection, but I’ve wasted so much time vibe coding this thing I guess I’d do better to actually program
- bckr 1y agoUpdate the old code, but make sure you’re using short (<350 line) files, and improve the type safety and code structure if possible. You have to guide these models. Vibe coding does not work.
- 999900000999 1y agoI don't expect to be able to git clone the Linux kernel, write "claude make it good" and fix everything. I do expect these tools to be to able to understand they code they write through. Writing new code is very easy. Maintaining code is hard. So far I'm very disappointed compared to how hyped this tech is. Although, I'm happy to have a job and if these coding models lived up to their promise I don't think I would have one.
- lawrenceyan 1y agoClaude is Buddhist! I’m extremely bullish.
- user3939382 1y agoStill can’t simulate parallel parking
- nickandbro 1y agoI am making a vim golf app, https://vimgolf.ai https://vimgolf.ai , for my side project where users compete with an AI vim opponent to learn vim and so far claude 4 sonnet (haven't tried opus) is ranked the highest in my evals against o3, o4-mini-high, and gemini 2.5 pro. After 3 retries, it solves this challenge: https://www.vimgolf.com/challenges/5957c3356f7e17045b00002c https://www.vimgolf.com/challenges/5957c3356f7e17045b00002c here is a snippet of its thinking process pinpointing the exact problem with getting it to successfully transform the start file to the end file: So I need to add a space between "string" and the backtick. Let me think about what's happening. In my commands: 1. `5G$a `xml:"field1"``<Esc>` 2. `6G$a `xml:"field2"``<Esc>` 3. `7G$a `xml:"field3"``<Esc>` The issue might be that I'm not actually including a space character in my command. Let me be very explicit about this. I need to use `<Space>` to represent the space character.
- a10c 1y agosome feedback for your app. I'm just dumped on a screen and left to my own devices, zero hints, prompting etc. At no point am I "learning vim through challenges"
- nickandbro 1y agoappreciate the feedback. Actually working on spinning up vim instances for the landing page now
- nathants 1y agowhen i read threads like this, it seems no one had actually used o3-high. i’m excited to try 4-opus later.
- dbingham 1y agoIt feels like these new models are no longer making order of magnitude jumps, but are instead into the long tail of incremental improvements. It seems like we might be close to maxing out what the current iteration of LLMs can accomplish and we're into the diminishing returns phase. If that's the case, then I have a bad feeling for the state of our industry. My experience with LLMs is that their code does _not_ cut it. The hallucinations are still a serious issue, and even when they aren't hallucinating they do not generate quality code. Their code is riddled with bugs, bad architectures, and poor decisions. Writing good code with an LLM isn't any faster than writing good code without it, since the vast majority of an engineer's time isn't spent writing -- it's spent reading and thinking. You have to spend more or less the same amount of time with the LLM understanding the code, thinking about the problems, and verifying its work (and then reprompting or redoing its work) as you would just writing it yourself from the beginning (most of the time). Which means that all these companies that are firing workers and demanding their remaining employees use LLMs to increase their productivity and throughput are going to find themselves in a few years with spaghettified, bug-riddled codebases that no one understands. And competitors who _didn't_ jump on the AI bandwagon, but instead kept grinding with a strong focus on quality will eat their lunches. Of course, there could be an unforeseen new order of magnitude jump. There's always the chance of that and then my prediction would be invalid. But so far, what I see is a fast approaching plateau.
- icpmacdo 1y ago"It feels like these new models are no longer making order of magnitude jumps, but are instead into the long tail of incremental improvements. It seems like we might be close to maxing out what the current iteration of LLMs can accomplish and we're into the diminishing returns phase." SWE bench from ~30-40% to ~70-80% this year
- piperswe 1y agoHow much of that is because the models are optimizing specifically for SWE bench?
- 1y ago
- Kevvv 1y ago[dead]
- willmarquis 1y agoWaiting for the ranking on the lmsys chat arena! The only source of truth
- willmarquis 1y agoDo you know when this will be available on Basalt? They didn't communicate on it yet
- arewethereyeta 1y agoI feel like these AI companies are in a gold rush while somebody else is selling the shovels. I've never jumped ship for the same service, from a vendor to another... so often. Looks like a race to the bottom where the snake eats itself.
- cedws 1y agoThere’s a reason NVIDIA’s stock price exploded over the past two years.
- AdamN 1y agoI believe Google released a paper about 2 years ago that said the same thing. There is no moat with AGI. Companies will find moats though - they just haven't figured out yet how.
- imiric 1y agoThe moat is how much money they have to throw at the problem. Corporations with deep pockets and those that secure investments based on fairy tales will "win".
- renerick 1y agoNVIDIA sells the shovels, then OpenAI/Anthropic/Google make an excavator out of shovels (NVDIA also seems to work on their own excavators), then some startup starts selling excavator wrapper. I don't know if there are any snakes at the bottom, but there's surely a whole lot of shovel layers on the way down.
- ripvanwinkle 1y agoshouldn't the comparison be with gpt4o or 4.5 and not 4.1 or o3
- smukherjee19 1y agoIs there any way to access the models without: - Linking the chats with my personal account - Having Anthropic train the model with my data? Like, having the knowledge of the model with the privacy of local LLMs?
- jmcgough 1y agoPer their data privacy policy, they do not train models on customer conversations by default.
- rafaelmn 1y agoAmazon bedrock ?
- Cheezemansam 1y ago>Having Anthropic train the model with my data? No.
- jakemanger 1y agoBeen playing around with it in Cursor and have to say I'm pretty dang impressed. Did notice a few times that it got stuck in a loop of trying to repeatedly make its implementation better. I suppose that is ok for some use cases but it started overthinking. I then gently prompted it by saying "you're way overthinking this. Just do a simple change like ..." I guess there's still a purpose for developers
- rootlocus 1y agoYou are now fine tuning their models, and you pay them money to do it.
- jakemanger 1y agoEven if you opt out of them storing your data? (a checkbox in the settings)
- jen729w 1y agoMe: is 'Claude does not have the ability to run the code it generates yet' a split infinitive? it's clumsy, no? i'd say 'Claude does not yet have the ability to run the code it generates' Claude: You're absolutely right on both counts! "Claude does not have the ability to run the code it generates yet" isn't technically a split infinitive (that would be something like "to quickly run"), but it is definitely clumsy. The word "yet" is awkwardly placed at the end, creating unnecessary distance from the verb it's modifying. Your revision - "Claude does not yet have the ability to run the code it generates" - flows much better. It places "yet" right after the auxiliary verb where it naturally belongs, making the sentence cleaner and more direct. The original reads like an afterthought was tacked on, while yours has a natural rhythm that emphasizes the temporal aspect more clearly. ;-)
- BoorishBears 1y ago"Claude can't run the code it generates."
- furyofantares 1y agoStill obsequious then. (My opinion is that the original is the superior wording. 'Yet' punctuates the sentence and leaves you considering that it may soon change.)
- handsclean 1y ago
- cedws 1y agoWell done to Anthropic for having the courage to release an N+1 model. OpenAI seems so afraid of disappointing with GPT 5 that it will just release models with a number asymptotically approaching 5 forever, generating unnecessary confusion about which is the best in their lineup of models. It’s branding worse than Windows versions.
- eru 1y agoHmm, Claude 4 (with extended thinking) seems a lot worse than Gemini 2.5 Pro and ChatGPT o3 at solving algorithmic programming problems.
- manbitesdog 1y agoI used my set of hidden prompts to see how it performs, and it's on par with 3.7
- feizhuzheng 1y agocool coding skills
- proxy2047 1y ago[dead]
- fintechie 1y agoIs this the first major flop from Anthropic? This thing is unusable. Slow, awful responses. Since Sonnet 3.5 the only real advance in LLM coding has been Gemini 2.5 Pro's context length. Both complement each other quite well so I'll stick to switch between these 2.
- waynenilsen 1y agoI think the vibes are really based on how you use it and what you're working on. For me I had the exact opposite vibe. I use it to generate typescript with Claude code
- sumedh 1y ago> Slow, awful responses. Probably there servers cannot handle the traffic today.
- kmacdough 1y agoCame here to learn what people think about Claude 4. Seems to be only armchair opinions on previous versions and the state of AI. The industry is not at all surprised that the current architecture of LLMS reached a plateau. Every other machine learning architecture we've ever used has gone through exactly the same cycle and frankly we're all surprised how far this current architecture has gotten us. Deepmind and OpenAI both publicly stated that they expected 2025 to be slow, particularly in terms of intelligence, well they work on future foundation models.
- cwsx 1y agoI've been using `claude-4-sonnet` for the last few hours - haven't been able to test `opus` yet as it's still overloaded - but I have noticed a massive improvement so far. I spent most of yesterday working on a tricky refactor (in a large codebase), rotating through `3.7/3.5/gemini/deepseek`, and barely making progress. I want to say I was running into context issues (even with very targeted prompts) but 3.7 loves a good rabbit-hole, so maybe it was that. I also added a new "ticketing" system (via rules) to help it's task-specific memory, which I didn't really get to test it with 3.7 (before 4.0 came out), so unsure how much of an impact this has. Using 4.0, the rest of this refactor (est. 4~ hrs w/ 3.7) took `sonnet-4.0` 45 minutes, including updating all of the documentation and tests (which normally with 3.7 requires multiple additional prompts, despite it being outlined in my rules files). The biggest differences I've noticed: - much more accurate/consistent; it actually finishes tasks rather than telling me it's done (and nothing working) - less likely to get stuck in a rabbit hole - stopped getting stuck when unable to fix something (and trying the same 3 solutions over-and-over) - runs for MUCH longer without my intervention - when using 3.7: - had to prompt once every few minutes, 5 - 10mins MAX if the task was straight forward enough - had to cancel the output in 1/4 prompts as it'd get stuck in the same thought-loops - needed to restore from a previous checkpoint every few chats/conversations - with 4.0: - ive had 4 hours of basically one-shotting everything - prompts run for 10 mins MIN, and the output actually works - is remembering to run tests, fix errors, update docs etc Obviously this is purely anecdotal - and, considering the temperament of LLMS, maybe I've just been lucky and will be back to cursing at it tomorrow, but imo this is the best feeling model since 3.5 released.
- machiaweliczny 1y agoI personally use GPT 4.1 in simple ask mode most recently. Fast and usually correct for quite complex function so OpenAI seems to be winning IMO. All these "agentic" things make these models so confused that it almost never gives good results in my testing.
- duck2 1y agoThis guy just told me on the Cursor window: > Looking at the system prompt, I can see I'm "powered by claude-4-sonnet-thinking" so I should clarify that I'm Claude 3.5 Sonnet, not Claude 4.
- resters 1y agomy impression is that Claude 4 is absolutely superb and now i consider it the best reasoning model. Claude Code is also significantly better than OpenAI codex at this time. Very impressive!
- blurbleblurble 1y agoIt's really good. I used it on a very complex problem that gemini 2.5 pro was going in circles on. It nailed it in 10x fewer tokens in half an hour.
- ciwolsey 1y agoFar too expensive to care
- replwoacause 1y agoDamn. Am I alone here in thinking Sonnet 4 is NOTICEABLY worse at coding than 3.7? Like, the amount of mistakes and gaslighting telling me it did something it obviously didn't do is off the charts. Switching back to 3.7 for all code for now, this thing aint ready for prime time yet. For context, I am using it on claude.ai, specifically the artifacts. Maybe something is broken there because they don't update when chat says they do. Took me about 10 turns to convince it: "You're absolutely right! I see the problem - the artifact isn't showing my latest updates correctly."
- replwoacause 1y agoUpdate: Using it through the API is fine. Maybe this was a bug on the claude.ai website. It's been working fine in my IDE.