3 ms·

GPT is shit at writing code. It's not dumb - extra high thinking is really good at catching stuff - but it's like letting a smart junior into your codebase - ig

GPT is shit at writing code. It's not dumb - extra high thinking is really good at catching stuff - but it's like letting a smart junior into your codebase - ignore all the conventions, surrounding context, just slop all over the place to get it working. Claude is just a level above in terms of editing code.

whalesalad 6mo ago
This has been my experience. With very very rigid constraints it does ok, but without them it will optimize expediency and getting it done at the expense of integrating with the broader system.
- ctoth 6mo ago
  My favorite example of this from last night: Me: Let's figure out how to clone our company Wordpress theme in Hugo. Here're some tools you can use, here's a way to compare screenshots, iterate until 0% difference. Codex: Okay Boss! I did the thing! I couldn't get the CSS to match so I just took PNGs of the original site and put them in place! Matches 100%!
leobuskin 6mo ago
And as a bonus: GPT is slow. I’m doing a lot of RE (IDA Pro + MCP), even when 5.4 gives a little bit better guesses (rarely, but happens) - it takes x2-x4 longer. So, it’s just easier to reiterate with Opus
- blazespin 6mo ago
  Yeah, need some good RE benchmarks for the LLMs. :) RE is very interesting problem. A lot more that SWE can be RE'd. I've found the LLMs are reluctant to assist, though you can workaround.
  - porker 6mo ago
    What is RE in this context?
    - astrange 6mo ago
      Reverse engineering
- 19h 6mo ago
  Mind sharing the use cases you're using IDA via MCP for?
- aizk 6mo ago
  I've been messing with using Claude, Codex, and Kimi even for reverse engineering at https://decomp.dev/ https://decomp.dev/ it's a ton of fun. Great because matching bytes is a scoring function that's easy for the models to understand and make progress on.
  - adamgoodapp 6mo ago
    I want to get into RE with AI. Which model you liking the most?
- rustystump 6mo ago
  This. People drastically underestimate how much more useful a lightning fast slightly dumb model is compared to a super smart but mega slow model is. Sure, u may need to bust out the beef now and then. However, the overwhelming majority of work the fast stupid model is a better fit.
Jcampuzano2 6mo ago
Not my experience. GPT 5.4 walks all over Claude from what I've worked with and its Claude that is the one willing to just go do unnecessary stuff that was never asked for or implement the more hacky solutions to things without a care for maintainability/readability. But I do not use extra high thinking unless its for code review. I sit at GPT 5.4 high 95% of the time.
zarzavat 6mo ago
Yes, it's becoming clear that OpenAI kinda sucks at alignment. GPT-5 can pass all the benchmarks but it just doesn't "feel good" like Claude or Gemini.
- lilytweed 6mo ago
  Whenever I come back to ChatGPT after using Claude or Gemini for an extended period, I’m really struck by the “AI-ness.” All the verbal tics and, truly, sloppishness, have been trained away by the other, more human-feeling models at this point.
  - kranke155 6mo ago
    GPT was clearly changed after its sycophantic models lead to the lawsuits.
    - josephg 6mo ago
      It still has a very ... plastic feeling. The way it writes feels cheap somehow. I don't know why, but Claude seems much more natural to me. I enjoy reading its writing a lot more. That said, I'll often throw a prompt into both claude and chatgpt and read both answers. GPT is frequently smarter.
      - kranke155 6mo ago
        GPT is more accurate. But Claude has this way of association between things that seems smarter and more human to me.
- chaos_emergent 6mo ago
  An alternative but similar formulation of that statement is that Anthropic has spent more training effort in getting the model to “feel good” rather than being correct on verifiable tasks. Which more or less tracks with my experience of using the model.
  - 6mo ago
sho_hn 6mo ago
Very different experience for me. Codex 5.3+ on xhigh are the only models I've tried so far that write reasonably decent C++ (domains: desktop GUI, robotics, game engine dev, embedded stuff, general systems engineering-type codebases), and idiomatic code in languages not well-represented in training data, e.g. QML. One thing I like is explicitly that it knows better when to stop, instead of brute-forcing a solution by spamming bespoke helpers everywhere no rational dev would write that way. Not always, no, and it takes investment in good prompting/guardrails/plans/explicit test recipes for sure. I'm still on average better at programming in context than Codex 5.4, even if slower. But in terms of "task complexity I can entrust to a model and not be completely disappointed and annoyed", it scores the best so far. Saves a lot on review/iteration overhead. It's annoying, too, because I don't much like OpenAI as a company. (Background: 25 years of C++ etc.)
- boring-human 6mo ago
  Same background as you, and same exact experience as you. Opus and Gemini have not come close to Codex for C++ work. I also run exclusively on xhigh. Its handling of complexity is unmatched. At least until next week when Mythos and GPT 6 throw it all up in the air again.
camdenreslink 6mo ago
ChatGPT 5.4 with extra high reasoning has worked really well for me, and I don't notice a huge difference with Opus 4.6 with high reasoning (those are the 2 models/thinking modes I've used the most in the last month or so).

whalesalad 6mo ago

This has been my experience. With very very rigid constraints it does ok, but without them it will optimize expediency and getting it done at the expense of integrating with the broader system.

ctoth 6mo ago
My favorite example of this from last night: Me: Let's figure out how to clone our company Wordpress theme in Hugo. Here're some tools you can use, here's a way to compare screenshots, iterate until 0% difference. Codex: Okay Boss! I did the thing! I couldn't get the CSS to match so I just took PNGs of the original site and put them in place! Matches 100%!

leobuskin 6mo ago

And as a bonus: GPT is slow. I’m doing a lot of RE (IDA Pro + MCP), even when 5.4 gives a little bit better guesses (rarely, but happens) - it takes x2-x4 longer. So, it’s just easier to reiterate with Opus

blazespin 6mo ago
Yeah, need some good RE benchmarks for the LLMs. :) RE is very interesting problem. A lot more that SWE can be RE'd. I've found the LLMs are reluctant to assist, though you can workaround.
- porker 6mo ago
  What is RE in this context?
  - astrange 6mo ago
    Reverse engineering
19h 6mo ago
Mind sharing the use cases you're using IDA via MCP for?
aizk 6mo ago
I've been messing with using Claude, Codex, and Kimi even for reverse engineering at https://decomp.dev/ https://decomp.dev/ it's a ton of fun. Great because matching bytes is a scoring function that's easy for the models to understand and make progress on.
- adamgoodapp 6mo ago
  I want to get into RE with AI. Which model you liking the most?
rustystump 6mo ago
This. People drastically underestimate how much more useful a lightning fast slightly dumb model is compared to a super smart but mega slow model is. Sure, u may need to bust out the beef now and then. However, the overwhelming majority of work the fast stupid model is a better fit.

Jcampuzano2 6mo ago

Not my experience. GPT 5.4 walks all over Claude from what I've worked with and its Claude that is the one willing to just go do unnecessary stuff that was never asked for or implement the more hacky solutions to things without a care for maintainability/readability. But I do not use extra high thinking unless its for code review. I sit at GPT 5.4 high 95% of the time.

zarzavat 6mo ago

Yes, it's becoming clear that OpenAI kinda sucks at alignment. GPT-5 can pass all the benchmarks but it just doesn't "feel good" like Claude or Gemini.

lilytweed 6mo ago
Whenever I come back to ChatGPT after using Claude or Gemini for an extended period, I’m really struck by the “AI-ness.” All the verbal tics and, truly, sloppishness, have been trained away by the other, more human-feeling models at this point.
- kranke155 6mo ago
  GPT was clearly changed after its sycophantic models lead to the lawsuits.
  - josephg 6mo ago
    It still has a very ... plastic feeling. The way it writes feels cheap somehow. I don't know why, but Claude seems much more natural to me. I enjoy reading its writing a lot more. That said, I'll often throw a prompt into both claude and chatgpt and read both answers. GPT is frequently smarter.
    - kranke155 6mo ago
      GPT is more accurate. But Claude has this way of association between things that seems smarter and more human to me.
chaos_emergent 6mo ago
An alternative but similar formulation of that statement is that Anthropic has spent more training effort in getting the model to “feel good” rather than being correct on verifiable tasks. Which more or less tracks with my experience of using the model.
- 6mo ago

sho_hn 6mo ago

Very different experience for me. Codex 5.3+ on xhigh are the only models I've tried so far that write reasonably decent C++ (domains: desktop GUI, robotics, game engine dev, embedded stuff, general systems engineering-type codebases), and idiomatic code in languages not well-represented in training data, e.g. QML. One thing I like is explicitly that it knows better when to stop, instead of brute-forcing a solution by spamming bespoke helpers everywhere no rational dev would write that way. Not always, no, and it takes investment in good prompting/guardrails/plans/explicit test recipes for sure. I'm still on average better at programming in context than Codex 5.4, even if slower. But in terms of "task complexity I can entrust to a model and not be completely disappointed and annoyed", it scores the best so far. Saves a lot on review/iteration overhead. It's annoying, too, because I don't much like OpenAI as a company. (Background: 25 years of C++ etc.)

boring-human 6mo ago
Same background as you, and same exact experience as you. Opus and Gemini have not come close to Codex for C++ work. I also run exclusively on xhigh. Its handling of complexity is unmatched. At least until next week when Mythos and GPT 6 throw it all up in the air again.

camdenreslink 6mo ago

ChatGPT 5.4 with extra high reasoning has worked really well for me, and I don't notice a huge difference with Opus 4.6 with high reasoning (those are the 2 models/thinking modes I've used the most in the last month or so).