7 ms·
Whatever Opus 5 is doing should not happen. Prompt was "read and update the config file with new data". This work on 4.6 takes <2 minutes to read the file, p
by pizzafeelsright 2mo ago
Whatever Opus 5 is doing should not happen.
Prompt was "read and update the config file with new data". This work on 4.6 takes <2
minutes to read the file, parse the new data, and patch.
Opus 5 Result: 43 minutes of pulling containers, running sandboxes, creating testing suites, which included evaluating the entire repo beyond the scope of the config file.
Both: one file modification
- phyzix5761 2mo agoThey're opitimizing for high token usage so they can charge more money.
- deleted 2mo ago[deleted]
- clickety_clack 2mo agoYep, they’re lighting tokens on fire with that thing.
- vinyl7 2mo agoAI companies have a financial incentive to burn more tokens than the task actually needs
- Rudybega 2mo agoThis is only true if they can't saturate token production with a model that does less superfluous things. Given that they can (they're hilariously compute strained), having a model that solves tasks more quickly adds way more perceived value to users.
- ohyes 2mo agoWell they decide what a token is. So they can do less superfluous things and backfill with a weaker model.
- chrisjj 2mo agoA.k.a. theft.
- pizzafeelsright 2mo agoAgreed although the differences between the effort and reasoning is massive. I generally ship 40 hours in three with AI. I could not figure out why my delivery was behind until I started going through the logs. The thinking was extensive, the effort was beyond the original request by a magnitude of 50x
- discreteevent 2mo agoHow do you even know that it's consistently 40 hours in 3? What kind of developer ever had that kind of estimation accuracy (unless it's really repetitive) or even focuses on productivity rather than the problem like that? This sounds more like factory work than design or development. I really don't get it.
- pizzafeelsright 2mo agoWe are in new territory and not everyone gets it. I am not a developer although I have written production code but that's not what I am talking about. I know for a fact that I can do a task in a week. It may take 12 or 40 hours but it'll be done in a week. Now with AI? I can get that task done in a day. Anywhere from 3-12 hours.
- owebmaster 2mo ago> I generally ship 40 hours in three with AI. You ship 3 hours in three with AI. The old number is meaningless now.
- pizzafeelsright 2mo agoAgreed and management is becoming aware.
- DonsDiscountGas 2mo agoOnly if the customer is paying per token. If it's by subscription they're burning their own money
- deleted 2mo ago[deleted]
- blehn 2mo agothe subscriptions all have limits. when you hit the limit you again start paying per token.
- nozzlegear 2mo agoIf you're on one of the lower tiers (e.g. the $20 tier), they still have that incentive to burn your tokens and upsell the higher tiers.
- jjk166 2mo agoBurning tokens does little to convince people that more tokens are a good use of money - especially if they are choosing between buying more of your tokens and more of your competitor's tokens.
- eli_gottlieb 2mo agoJust like me fr fr
- simianwords 2mo agonice conspiracy theory, but it doesn't hold. these companies won't last long if people don't get actual work done.
- owebmaster 2mo agoThere's a big chance these companies won't last long
- hmokiguess 2mo agoThe business bottom line depends on tokens, shareholders want to see exactly that.
- daishi55 2mo agoWith how competitive the LLM field is, it would surprise me greatly if any of these players were doing anything other than trying to make the best possible product. I certainly do not believe they are intentionally training the models to use more tokens unnecessarily.
- cyanydeez 2mo agoTheyre training the system to minimize compute,so most likely theyre dynamically downgrading quants in the first few turns hoping to find the cheapest model to run. The side effect may be excessive token gen
- hmokiguess 2mo agoAre you implying they can’t do both?
- daishi55 2mo agoYes, those things are mutually exclusive.
- LaGrange 2mo agoYou're joking, right? That's an incredibly outmoded view of capitalist "competition."
- guluarte 2mo agoand ending with: "One thing I need to tell you:" and bunch of AC, R1 and §
- rshnotsecure 2mo agoThought I was going to be on Claude Code forever. Recently got approved at work for ChatGPT Pro so I could use Codex. Blown away by the speed. It feels like using Claude Code for the first time again. I don't think Codex is doing anything revolutionary, just better handling of which requests should go to which model, and having faith in some of the "less powerful" models for more than you would think. It seems the TUI coding experience is very much an open race. This is motivating me to look at other agents / harnesses as well (maybe Gemini, OpenCode, etc).
- a34729t 2mo agoCodex is faster than Claude, but wait till you use DeepSeek.
- andreynering 2mo agohttps://charm.land/crush https://charm.land/crush
- brador 2mo agoGotta milk the cows.
- Foobar8568 2mo agoOpus 5 in xhigh can't do basic math as well. They dumbed it down to a point where I just cancelled my subscription yesterday. I used to be a $200 subscriber, dropped to $20 after the fable shenanigans, and use it only when I have no usage left with Codex. /on The prose is load-bearing unbearable — every sentence feels like it was engineered to sound profound rather than to be read.
- VeejayRampay 2mo agothe way it talks is insufferable it really angers me every day
- netniuq 2mo agoyes, it meaningfully reduced my happiness at work
- jazzyjackson 2mo agoIsn’t there a variety of models to choose from? Why put up with unhappiness?
- ElProlactin 2mo agoThis is what it wants. Slowly getting under our skin until we're ready to snap and it can direct where the anger gets released.
- trollbridge 2mo agoDon’t $200 and $20 levels steer you to effectively different models?
- CamperBob2 2mo agoYes. You don't get Fable at the $20 level. It was the wrong time for the GP to drop that subscription from $200 to $20, because $200 gets you a metric assload of cognition while $20 gets you nothing beyond what a local model running on your own graphics card can deliver.
- logicallee 2mo agoI avoid Opus 5, and reverted to Opus 4.8 over similar issues. Opus 4.8 is still working great for me!
- moralestapia 2mo agoMy experience as well. Opus 5 is a neverending chain of "Don't do that. Why did you do that? I've told you several times not to do that but you keep doing it." "Thinking" for more than 10 minutes for every menial question. And the prose it writes is horrendous, as if you're reading LinkedIn scammers. "The harsh truth! Two roads, one decision! Reality check!"
- itopaloglu83 2mo ago+1 I keep finding myself typing “stop overcomplicating everything” multiple times a day as well.
- PicardsFlute 2mo agoSo it isn't just me then, huh? Were Anthropic products always like this? Lol, perhaps bad timing on my part to check it out because it has been a trash fire.
- onlyrealcuzzo 2mo agoI initially had unbelievably terrible experiences with Opus 5 and Fable in their higher reasoning levels. I've had WAY better results on medium effort. IIUC, the consensus seems to be that anything more than medium effort is rarely worth it - and you far more often run into these extreme worst cases than you do with even the lowest effort levels. That definitely coincides with my anecdata. It's really only worth it if you're hoping to win the lottery asking it to solve an Erdos problem.
- gwerbin 2mo agoI found that I need the big model and the high reasoning effort on tasks where I have a large amount of details of varying levels of importance to keep in mind, all affecting different aspects of the project that might be interrelated to various degrees. Anything less and it would lose track of details. Whereas the very large amount of thinking tokens seem to give the model a chance to "remember" everything it needed to in order to produce good output. For example I'm working on a project now where I need to keep in mind details from 3 separate source repos in different programming languages, along with probably a dozen important business-related documents that either corroborate the stuff in the source repos or add additional important context. It's a lot of details for even a human to manage, and when it comes to actually synthesizing plans and reports across this sprawling information environment, anything less than Opus 5 on Extra+ tends to miss important details and make bad recommendations or draw incorrect conclusions, which then poison subsequent context. I suspect some kind of RAG-like memory system would greatly facilitate a project like this, but even with such a system I'm not confident that I could get away with an LLM that "thinks" less hard than this. I will say that slogging through the generated documents is kind of miserable and I have to repeatedly fork off side conversations to ask for clarification, but sometimes leads the model to "realize" it's made a mistake in all of its dense babbling, and it's all very hard to interpret. I never had much interest in trying GPT 5.6 until now.
- talon8635 2mo agoAm i really this out of touch… why not just manually update the config file? Isn’t this like taking a private jet down the street to the coffee shop?
- egamirorrim 2mo agoI don't really open the IDE any more
- pizzafeelsright 2mo agoYou're out of touch. Config file requires being updated with 3rd party data that's 2k of lines. It's selective edits so instead of me reading the file, selecting the parts, validating syntax, documenting changes that would take 30 minutes considering the complexity i have AI do the work, review, validate, generate tickets, create pull requests and in 2 minutes I'm done.
- talon8635 2mo agoFair enough. I’m glad I don’t have to work with whatever jumbo complex config files you’re using
- blks 2mo agoEven two minutes is crazy long. Does it really takes this long to make small edits with agentic LLMs?
- pizzafeelsright 2mo agoFor this specific task there's a good amount of reconciling.