4 ms·
I'm struggling to understand why I'd ever use this instead of just using a lower effort level for opus given on many of the benchmarks listed the cost per task
by Jcampuzano2 4mo ago
I'm struggling to understand why I'd ever use this instead of just using a lower effort level for opus given on many of the benchmarks listed the cost per task rises above opus at anything higher than medium effort.
Only thing I can think of is for when someone is out of opus credits. Of course there are API billing use cases but I'd probably still just use opus on low.
- nicce 4mo agoOlder Opus models will likely get deprecated and then over time this is the cheapest model. That is how prices are currently increased.
- ChrisLTD 4mo agoYeah... Sonnet becomes the new cheap model, and some Fable class model becomes the more expensive/better one.
- theptip 4mo agoWat. Price/perf has been going down massively over the last few years.
- darkwater 4mo agoBecause they still haven't fully captured the market for Agentic Development.
- enraged_camel 4mo agoSpeed is a huge reason. Sometimes you just need some simple tasks get done fast, and waiting 30-60 seconds for opus to even start thinking can really slow things down.
- humanymous 4mo agoOpus with low reasoning effort would be faster than Sonnet with high reasoning. So that won't exactly help. I think it would just be what those models are optimized to perform
- itopaloglu83 4mo agoMore and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things. I think the models are being optimized for wealth extraction from users and companies, instead of solving problems. I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python.
- post-it 4mo ago> I don't know why Opus would try to create an entire library when I told it specifically to do something simple that would take 2-3 lines of Python. Because it reasons in one direction. First it encounters some kind of issue with 2-3 lines of Python that might make it not work, and then it goes onto plan B, which is making a library, but it doesn't circle back and compare the effort of making the library to working around whatever might make the 2-3 lines not work. Except sometimes it does, because it's inscrutable.
- ethbr1 4mo agoSo, as the quip goes, junior level reasoning.
- __natty__ 4mo ago> More and more I find myself trying to stop Opus from doing something stupid, and at every turn I need to tell it to stop overcomplicating things Yeah, that’s my thoughts as well. I feel it’s great for benchmarks and some tasks while in other it tries to spend as much tokens as possible, tries to overcomplicate task and needs seconds or third round of steering that costs. With the scale Anthropic operates I bet it’s huge amount of extra money just to make sure their model works.
- Aeolun 4mo agoIt’s really weird when you go to one of the open models and suddenly the same context window stretches nearly 3-4 times as long.
- 4mo ago
- SirMaster 4mo agoMaybe it's not for you? I don't pay, so I can't even use Opus... So this is an upgrade over Sonnet 4.6 for me.
- phainopepla2 4mo agoLooking at some of the agentic coding benchmarks on the system card[0], pages 117-118, it seems that running it at low outperforms Sonnet 4.6 at any level, and is a good deal cheaper as well. So on low it could be a good workhorse for an Opus-planned task. [0] https://www.anthropic.com/claude-sonnet-5-system-card https://www.anthropic.com/claude-sonnet-5-system-card
- port11 4mo agoThat is certainly an improvement then. Sonnet 4.6 is a great everyday agent for the limited Pro plan, but it’s not much better than M3 or Kimi 2.7, both significantly cheaper models.
- c0m47053 4mo agoSpecific task based benchmarks don't reflect a lot of day to day agentic use cases in my experience. If you are working on a series of discrete tasks and can clear context after each one and move to the next, you might get that sort of efficiency from Opus low effort. I often find that when working through a real problem, iterating and discovering, context length can creep up, and that is where opus tends to get expensive.
- theptip 4mo agoAre we reading the same chart? They have Sonnet <= high as Pareto dominant on $/perf. You have to test each task obviously but it is not a bad model on its face.
- frozeus 4mo agoThey have updated it
- LUmBULtERA 4mo agoDid Anthropic have Opus 4.8 and Sonnet 5 switched in the Agentic Search chart at first?
- fluidcruft 4mo agoNo, and the original had everything more expensive. There's a comparison here: https://www.reddit.com/r/ClaudeAI/comments/1ukgqwr/looks_like_anthropic_quietly_updated_the_sonnet_5/ https://www.reddit.com/r/ClaudeAI/comments/1ukgqwr/looks_lik... The explanation Anthropic gave for the update doesn't address how the x-axis needed to range up to $50 previously and only $10 now. In any case the pass rates are also lower. Probably the difference between whatever it is people notice when they say models become "nerfed".
- LUmBULtERA 4mo agoHuh, super interesting. Thanks!
- theptip 4mo agoHa! So we were not looking at the same chart. That makes more sense. > Anthropic did post an official explanation, stating the original chart used a "simpler methodology" that "underestimated Sonnet 5's performance." The new chart supposedly uses their "standard methodology." Oops!
- fluidcruft 4mo agoIf you are out of Opus credits, you are out of all model credits.
- southforgeai 4mo agoI concur. I already use Opus 4.8 for almost all my tasks and this gives me almost no reason to try Sonnet 5.
- licjon 4mo agoIs there a router or wrapper that provides a real-time cost estimation for alternative settings? Obviously, you can't predict exact output tokens without running the inference, but a tool that calculates the exact input cost across models and applies a historical average for the output tokens could be useful. Like, you run a task on Sonnet, and it estimates: "Based on your input tokens and a 1:1 output ratio, this would have cost $X on Opus at a low effort level."
- annjose 4mo agoNot sure of any out-of-the-box tool. But Anthropic has a token count API which gives a near estimate of the input tokens for messages [1]. So this API can be used in a UserPromptSubmit hook [2] in the harness, get the token count for any model, calculate the cost and compare. [1] https://platform.claude.com/docs/en/build-with-claude/token-counting https://platform.claude.com/docs/en/build-with-claude/token-... [2] https://code.claude.com/docs/en/hooks https://code.claude.com/docs/en/hooks
- hollownobody 4mo agoI would use Sonnet instead of Opus because it's faster. Isn't it? It's a smaller model