5 ms·

If anyone from OpenAI is reading this -- a plea to not screw with the reasoning capabilities! Codex is so so good at finding bugs and little inconsistencies, i

• tgtweak 10mo ago
  Anecdotally I've found it very good in the exact same case for multi-agent workflows - as the "reviewer"
• ifwinterco 10mo ago
  I think the issue is for them "quality, not speed" means "expensive, not cheap" and they can't pass that extra cost on to customers
  • mccoyb 10mo ago
    I'm happy to pay the same right now for less (on the max plan, or whatever) -- because I'm never running into limits, and I'm running these models near all day every day (as a single user working on my own personal projects). I consistently run into limits with CC (Opus 4.5) -- but even though Codex seems to be spending significantly more tokens, it just seems like the quota limit is much higher?
    • Computer0 10mo ago
      I am on the $20 plan for CC and Codex, I feel like a session of usage on CC == ~20% Codex usage / 5 hours in terms of time spent inferencing. It has always seemed way more geneous than I would expect.
      • Aurornis 10mo ago
        Agreed. The $20 plans can go very far when you're using the coding agent as an additional tool in your development flow, not just trying to hammer it with prompts until you get output that works. Managing context goes a long way, too. I clear context for every new task and keep the local context files up to date with key info to get the LLM on target quickly
        • girvo 10mo ago
          > I clear context for every new task and keep the local context files up to date with key info to get the LLM on target quickly Aggressively recreating your context is still the best way to get the best results from these tools too, so it has a secondary benefit.
          • heliumtera 10mo ago
            It is ironic that in the gpt-4 era, when we couldn't see much value in this tools, all we could hear was "skill issues", "prompt engineering skills". Now they are actually quite capable for SOME tasks, specially for something that we don't really care about learning, and they, to a certain extent, can generalize. They perform much better than in gpt-4 era, objectively, across all domains. They perform much better with the absolute minimum input, objectively, across all domains. If someone skipped the whole "prompt engineering" and learned nothing during that time, this person is more equiped to perform well. Now I wonder how much I am leaving behind by ignoring this whole "skills, tools, MCP this and that, yada yada".
            • neom 10mo ago
              Any thoughts on your wondering? I too am wondering about the same mistake I might be making.
            • conradev 10mo ago
              Prompt engineering (communicating with models?) is a foundational skill. Skills, tools, MCPs, etc. are all built on prompts. My take is that the overlap is strongest with engineering management. If you can learn how to manage a team of human engineers well, that translates to managing a team of agents well.
    • andai 10mo ago
      If you look at benchmarks, the Claude models score significantly higher intelligence per token. I'm not sure how that works exactly, but they are offset from the entire rest of the chart on that metric. It seems they need less tokens to get the same result. (I can't speak for how that affects performance on very difficult tasks though, since most of mine are pretty straightforward.) So if you look at the total cost of running the benchmark, it's surprisingly similar to other models -- the higher price per token is offset by the significantly fewer tokens required to complete a task. See "Cost to Run Artificial Analysis Index" and "Intelligence vs Output Tokens" here https://artificialanalysis.ai/ https://artificialanalysis.ai/ ...With the obligatory caveat that benchmarks are largely irrelevant for actual real world tasks and you need to test the thing on your actual task to see how well it does!
    • hadlock 10mo ago
      I noticed I am not hitting limits either. My guess is OpenAI sees CC as a real competitor/serious threat. Had OAI not given me virtually unlimited use I probably would have jumped ship to CC by now. Burning tons of cash at this stage is likely Very Worth It to maintain "market leader" status if only in the eyes of the media/investors. It's going to be real hard to claw back current usage limits though.
  • golly_ned 10mo ago
    I wonder how much their revenue really ends up contributes towards covering their costs. In my mind, they're hardly making any money compared to how much they're spending, and are relying on future modeling and efficiency gains to be able to reduce their costs but are pursuing user growth and engagement almost fully -- the more queries they get, the more data they get, the bigger a data moat they can build.
    • nimchimpsky 10mo ago
      "In my mind, they're hardly making any money compared to how much they're spending" everyone seems to assume this, but its not like its a company run by dummies, or has dummy investors. They are obviously making awful lot of revenue.
      • Daneel_ 10mo ago
        To me it seems that they're banking on it becoming indispensable. Right now I could go back to pre-AI and be a little disappointed but otherwise fine. I figure all of these AI companies are in a race to make themselves part of everyone's core workflow in life, like clothing or a smart phone, such that we don't have much of a choice as to whether we use it or not - it just IS. That's what the investors are chasing, in my opinion.
        • zozbot234 10mo ago
          It'll never be literally indispensible, because open models exist - either served by third-party providers, or even ran locally in a homelab setup. A nice thing that's arguably unique about the latter is that you can trade scale for latency - you get to run much larger models on the same hardware if they can chug on the answer overnight (with offload to fast SSD for bulk storage of parameters and activations) instead of just answering on the spot. Large providers don't want to do this, because keeping your query's activations around is just too expensive when scaled to many users.
      • troupo 10mo ago
        Revenue != profit. They are drowning in debt and go into more and more ridiculous schemes to raise/get more money. --- start quote --- OpenAI has made $1.4 trillion in commitments to procure the energy and computing power it needs to fuel its operations in the future. But it has previously disclosed that it expects to make only $20 billion in revenues this year. And a recent analysis by HSBC concluded that even if the company is making more than $200 billion by 2030, it will still need to find a further $207 billion in funding to stay in business. https://finance.yahoo.com/news/openai-partners-carrying-96-billion-120300663.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAALipYaGPcnEz5DQxjQ3RY1XfzdvtMd1d8fkD2C4ug1Setj4Okzn-CPmxi9LNtZGQ3l5Ievandz_4tsnXvR8OCWC-217J4rAJA0Fs8EY7zVgAgDExeakiyLP7CYKOWB1EoP5Z4ziGlNiMMQ3v_9LBrlBhEmXyezXZMfWA5MiTHaJT https://finance.yahoo.com/news/openai-partners-carrying-96-b... --- end quote ---
  • zozbot234 10mo ago
    The "quality" model can cost $200/month. They'll be fine.
  • tejohnso 10mo ago
    > they can't pass that extra cost on to customers I don't understand why not. People pay for quality all the time, and often they're begging to pay for quality, it's just not an option. Of course, it depends on how much more quality is being offered, but it sounds like a significant amount here.
• kilroy123 10mo ago
  Interesting what I've seen is it spins and thinks forever. Then just breaks. Which is beyond frustrating.
  • mccoyb 10mo ago
    If by "just breaks" means "refuses to write code / gives up or reverts what it does" -- yes, I've experienced that. Experiencing that repeatedly motivated me to use it as a reviewer (which another commenter noted), a role which it is (from my experience) very good at. I basically use it to drive Claude Code, which will nuke the codebase with abandon.
    • kilroy123 10mo ago
      I've seen it think for a long time and then just timeout or something? It just stops and nothing happens.
      • JamesSwift 10mo ago
        Ive had the same but i only use it through zed so I wasnt sure if it was a codex issue or a zed issue
    • fragmede 10mo ago
      I've had codex rm -rf the git repo it's working in while running in yolo mode. Twice, even! (Play with fire, you're gonna get burnt.) I had it whip this up to try and avoid this, while still running it in yolo mode (which is still not recommended). https://gist.github.com/fragmede/96f35225c29cf8790f10b1668b849af7 https://gist.github.com/fragmede/96f35225c29cf8790f10b1668b8...
  • baq 10mo ago
    we're all senior continue engineers nowadays it seems
• apitman 10mo ago
  It's annoying though because it keeps (accurately) pointing out critical memory bugs that I clearly need to fix rather than pretending they aren't there. It's slowing me down.
  • gnatolf 10mo ago
    Love it when it circles around a minor issue that I clearly described as temporary hack instead of recognizing the tremendously large gaping hole in my implementation right next to it.
• echelon 10mo ago
  > If anyone from OpenAI is reading this (unrelated, but piggybacking on requests to reach the teams) If anyone from OpenAI or Google is reading this, please continue to make your image editing models work with the "previz-to-render" workflow. Image edits should strongly infer pose and blocking as an internal ControlNet, but should be able to upscale low-fidelity mannequins, cutouts, and plates/billboards. OpenAI kicks ass at this (but could do better with style controls - if I give a Midjourney style ref, use it) : https://imgur.com/gallery/previz-to-image-gpt-image-1-x8t1ijX https://imgur.com/gallery/previz-to-image-gpt-image-1-x8t1ij... https://imgur.com/a/previz-to-image-gpt-image-1-5-3fq042U https://imgur.com/a/previz-to-image-gpt-image-1-5-3fq042U Google fails the tests currently, but can probably easily catch up : https://imgur.com/a/previz-to-image-nano-banana-pro-Q2B8psd https://imgur.com/a/previz-to-image-nano-banana-pro-Q2B8psd
• baseonmars 10mo ago
  absolutely second this. I'm mainly a claude code user, but i have codex running in another tab and for code reviews and it's absolutely killer at analyzing flows and finding subtle bugs.
  • mkagenius 10mo ago
    Have you tried Claude Code in the second tab instead, that would be a fair comparison.
    • hugh-avherald 10mo ago
      Claude Code isn't as surgical as Codex at reviews
• garbagecoder 10mo ago
  Agree. Codex just read my source code for a toy lisp I wrote in ARM64 assembly and learned how to code in that lisp and wrote a few demo programs for me. The was impressive enough. Then it spent some time and effort to really hunt down some problems--there was a single bit mask error in my garbage collector that wasn't showing up until then. I was blown away. It's the kind of thing I would have spent forever trying to figure out before.
  • heliumtera 10mo ago
    Maybe you're a garbage programmer and that error was too obvious. Interesting observation, though. edit: username joke, don't get me banned
  • josephg 10mo ago
    I've been writing a little port of the SeL4 OS kernel to rust, mostly as a learning exercise. I ran into a weird bug yesterday where some of my code wasn't running - qemu was just exiting. And I couldn't figure out why. I asked codex to take a look. It took a couple minutes, but it managed to track the issue down using a bunch of tricks I've never seen before. I was blown away. In particular, it reran qemu with different flags to get more information about a CPU fault I couldn't see. Then got a hex code of the instruction pointer at the time of the fault, and used some tools I didn't know about to map that pointer to the lines of code which were causing the problem. Then took a read of that part of the code and guessed (correctly) what the issue was. I guess I haven't worked with operating systems much, so I haven't seen any of those tricks before. But, holy cow! Its tempting to just accept the help and move on, but today I want to go through what it did in detail, including all the tools it used, so I can learn to do the same thing myself next time.
  • varjag 10mo ago
    Interestingly it found a GC bug in my toy Lisp that I wrote in Z80 assembly almost 30 years ago. This kind of work appears to be more common than you'd think!
• johnnyfived 10mo ago
  Agreed, I'm surprised how much much care the "extra high" reasoning allows. It easily catches bugs in code other LLMs won't, using it to review Opus 4.5 is highly effective.
• smoe 10mo ago
  Do you think that for someone who only needs careful, methodical identification of “problems” occasionally, like a couple of times per day, the $20/month plan gets you anywhere, or do you need the $200 plan just to get access to this?
  • nl 10mo ago
    The $20 does this fine. The OpenAI token limits seem more generous than the Anthropic ones too.
    • rbancroft 10mo ago
      Listening to Dario at the NYT DealBook summit, and reading between the lines a bit, it seems like he is basically saying Anthropic is trying to be a reponsible, sustainable business and charging customers accordingly, and insinuating that OpenAI is being much more reckless, financially.
      • nl 10mo ago
        I think it's difficult to estimate how profitable both are - depends too much on usage and that varies so much. I think it is widely accepted that Anthropic is doing very well in enterprise adoption of Claude Code. In most of those cases that is paid via API key not by subscription so the business model works differently - it doesn't rely on low usage users subsidizing high usage users. OTOH OpenAI is way ahead on consumer usage - which also includes Codex even if most consumers don't use it. I don't think it matters - just make use of the best model at the best price. At the moment Codex 5.2 seems best at the mid-price range, while Opus seems slightly stronger than Codex Max (but too expensive to use for many things).
        • grim_io 10mo ago
          Consumer subscriptions are an impossible business, that's why openai NEEDS ads to have any chance of sustainability.
  • hatefulmoron 10mo ago
• sinatra 10mo ago
  Piggybacking on this post. Codex is not only finding much higher quality issues, it’s also writing code that usually doesn’t leave quality issues behind. Claude is much faster but it definitely leaves serious quality issues behind. So much so that now I rely completely on Codex for code reviews and actual coding. I will pick higher quality over speed every day. Please don’t change it, OpenAI team!
  • F7F7F7 10mo ago
    Every plan Opus creates in Planning mode gets run through ChatGPT 5.2. It catches at least 3 or 4 serious issues that Claude didn’t think of. It typically takes 2 or 3 back and fourths for Claude to ultimately get it right. I’m in Claude Code so often (x20 Max) and I’m so comfortable with my environment setup with hooks (for guardrails and context) that I haven’t given Codex a serious shot yet.
    • SkyPuncher 10mo ago
      The same thing can be said about Opus running through Opus. It's often not that a different model is better (well, it still has to be a good model). It's that the different chat has a different objective - and will identify different things.
      • sinatra 10mo ago
        My (admittedly one person's anecdotal) experience has been that when I ask Codex and Claude to make a plan/fix and then ask them both to review it, they both agree that Codex's version is better quality. This is on a 140K LOC codebase with an unreasonable amount of time spent on rules (lint, format, commit, etc), on specifying coding patterns, on documenting per workspace README.md, etc.
      • shinycode 10mo ago
        Every time Claude Code finishes a task, I plan a full review of its own task with a very detailed plan and it catches itself many things it didn’t see before. It works well and it’s part of the process of refinement. We all know it’s almost never 100% hit of the first try on big chunks of code generated.
• energy123 10mo ago
  Second this but for the chat subscription. Whatever they did with 5.2 compared to 5.0 in ChatGPT increased the test-time compute and the quality shows. If only they would allow more tokens to be submitted in one prompt (it's currently capped at 46k for Plus). I don't touch Gemini 3.0 Pro now (am also subbed there) unless I need the context length.
• jvermillard 10mo ago
  I use it mainly for embedded programming and I find codex way better than claude. I don't my the delay anyway I'm slower to code carefully crafted C
• rane 10mo ago
  Exactly. This is why the workflow of consulting Gemini/Codex for architecture and overall plan, and then have Claude implement the changes is so powerful.
• stared 10mo ago
  If you want to combine Claude Code coding with reasoning, it is easy to do it with a plugin - https://github.com/stared/gemini-claude-skills https://github.com/stared/gemini-claude-skills, wrote for myself, but shared in case anyone wants. Somehow bigger context here: https://quesma.com/blog/claude-skills-not-antigravity/ https://quesma.com/blog/claude-skills-not-antigravity/.
• mattio 10mo ago
  Completely agreed. Used claude and codex both on highest tier next to each other for a month. On complex tasks where Claude would get stuck and not be able to fix it at all, codex would fix the issue in one go. Codex is amazing. I did found some slip ups in 5.2 where I did a refactor of a client header where I removed two header properties, but 5.2 forgot to remove those from the toArray method of the class. Was using 5.2 on medium (default).