5 ms·

o3 is still my favorite over even Opus 4 in most cases. I've spent hundreds of dollars on AI code gen tools in the last month alone and my ranking is: 1. o3 -

o3 is still my favorite over even Opus 4 in most cases. I've spent hundreds of dollars on AI code gen tools in the last month alone and my ranking is: 1. o3 - it's just really damn good at nuance, getting to the core of the goal, and writing the closest thing to quality production level code. The only negative is it's cutoff window and cost, especially with it's love of tools. That's not usually a big deal for the Rails projects I work on but sometimes it is. 2. Opus 4 via Claude Code - also really good and is my daily driver because o3 is so expensive. I will often have Opus 4 come up with the plan and first pass and then let o3 critique and make a list of feedback to make it really good. 3. Gemini 2.5 Pro - haven't tested this latest release but this was my prior #2 before last week. Now I'd say it's tied or slightly better than Sonnet 4. Depends on the situation. 4. Sonnet 4 via claude Code - it's not bad but needs a lot of coaching and oversight to produce really good code. It will definitely produce a lot of code if you just let it go do it's thing but it's not the quality, concise, and thoughtful code without more specific prompting and revisions. I'm also extremely picky and a bit OCD with code quality and organization in projects down to little details with naming, reusability, etc. I accept only 33% of suggested code based on my Cursor stats from last month. I will often revert and go back to refine the prompt before accepting and going down a less than optimal path.

throwaway314155 1y ago
It's interesting you say that because o3, while being a considerable improvement over OpenAI's other models, still doesn't match the performance of Opus 4 and Gemini 2.5 Pro by a long shot for me. However, o3 resides in the ChatGPT app, which is still superior to the other chat apps in many ways, particularly the internet search implementation works very well.
- joshmlewis 1y ago
  What languages do you use it with and IDE? I use it in Cursor mainly with Max reasoning on. I spent around $300 on token based usage for o3 alone in May still only accepting around 33% of suggestions though. I made a post on X about this the other day but I expect that amount of rejections will go down significantly by the end of this year at the rate things are going.
  - drawnwren 1y ago
    Very strange. I find reasoning has very narrow usefulness for me. It's great to get a project in context or to get oriented in the conversation, but on long conversations I find reasoning starts to add way too much extraneous stuff and get distracted from the task at hand. I think my coding model ranking is something like Claude Code > Claude 4 raw > Gemini > big gap > o4-mini > o3
    - joshmlewis 1y ago
      Claude Code isn't a model in itself. By default it routes some to Opus 4 or Sonnet 4 but mostly Sonnet 4 unless you explicitly set it.
      - drawnwren 1y ago
        I am aware
  - throwaway314155 1y ago
    i'm using with python, VS Code (not integrated with claude just basic copilot) and Claude Code. For Gemini i'm using AI studio with repomix to package my code into a single file. I copy files over manually in that workflow. All subscription based, not per token pricing. I'm currently using Claude Max. Can't see myself exhausting its usage at this rate but who knows.
- svachalek 1y ago
  If you're coding through chat apps you're really behind the times. Try an agent IDE or plugin.
  - joshmlewis 1y ago
    Yeah, exactly. For everyone who might not know, the chat apps add lots of complex system prompting to handle and shape personality, tone, general usability, etc. IDE's also do this (with Claude Code being one of the ones that are closest to "bare" model that you can get) but at they are at least guiding it's behavior to be really good at coding tasks. Another reason is using the Agent feature that IDE's have had for a few months now which gives it the ability to search/read/edit files across your codebase. You may not like the idea of this and it feels like losing control, but it's the future. After months of using it I've learned how to get it to do what I want but I think a lot of people who try it once and stop get frustrated that it does something dumb and just assume it's not good. That's a practice and skill problem not a model problem.
    - Workaccount2 1y ago
      IDE's are intimidating to non-tech people. I'm surprised there isn't a VibeIDE yet that is purpose build to make it possible for your grandmother to execute code output by an LLM.
      - dragonwriter 1y ago
        > I'm surprised there isn't a VibeIDE yet that is purpose build to make it possible for your grandmother to execute code output by an LLM. The major LLM chat interfaces often have code execution built in, so there kind of is, it just doesn't look like what an SWE thinks of as an IDE.
      - joshmlewis 1y ago
        I have not used them but I feel like there are tools like Replit, Lovable, etc that are for that audience. I totally agree IDE's are intimidating for non-technical people though. Claude Code is pretty cool in that way where it's one command to install and pretty easy to get started with.
- jorvi 1y ago
  What's most annoying about Gemini 2.5 is that it is obnoxiously verbose compared to Opus 4. Both in explaining the code it wrote and the amount of lines it writes and comments it adds, to the point where the output is often 2-3x more than Opus 4. You can obviously alleviate this by asking it to be more concise but even then it bleeds through sometimes.
  - joshmlewis 1y ago
    Yes this is what I mean by conciseness with o3. If prompted well it can produce extremely high level quality code that blows me away at times. I've also had several instances now where I gave it slightly wrong context and other models just butchered a solution with dozens of lines for the proposed fix which I could tell wasn't right and then after reverting and asking o3, it immediately went searching for another file I hadn't included and fixed it in one line. That kind of, dare I say independent thinking, is worth a lot when dealing with complex codebases.
    - jorvi 1y ago
      Personally I still am of the opinion current LLMs are more of a very advanced autocomplete. I have to think of the guy posting that he fed his entire project codebase to an AI, it refactored everything, modularizing it but still reducing the file count from 20 to 12. "It was glorious to see. Nothing worked of course, but glorious nonetheless". In the future I can certainly see it get better and better, especially because code is a hard science that reduces down to control flow logic which reduces down to math. It's a much more narrow problem space than, say, poetry or visuals.
monkpit 1y ago
Have you used Cline with opus+sonnet? Do you have opinions about Claude code vs cline+api? Curious to hear your thoughts!
spaceman_2020 1y ago
I use o3 a lot for basic research and analysis. I also find the deep research tool really useful for even basic shopping research Like just today, it made a list of toys for my toddler that fit her developmental stage and play style. Would have taken me 1-2 hrs of browsing multiple websites otherwise
- jml78 1y ago
  Gemini deep research runs circles around OpenAI deep research. It goes way deeper and uses way more sources.
pqdbr 1y ago
How do you choose which model to use with Claude Code?
- joshmlewis 1y ago
  I have the Max $200 plan so I set it to Opus until it limits me to Sonnet 4 which has only happened in two out of a few dozen sessions so far. My rule of thumb in Cursor is it's worth paying for the Max reasoning models for pretty much every request unless it's stupid simple because it produces the best code each time without any funny business you get with cheaper models.
  - sunshinerag 1y ago
    You can use the max plan in cursor? I thought it didn’t support calls via api and only worked in Claude code?
    - symbolicAGI 1y ago
      I launch Claude Code in VS Studio (similar to Cursor): > claude Then I use the /login command that opens a browser window to log into Claude Max. You can confirm Claude Max billing going forward in VS Studio/Claude Code: /cost "With your Claude Max subscription, no need to monitor cost — your subscription includes Claude Code usage"
- jasonjmcghee 1y ago
  In case you're asking for the literal command... /model
VeejayRampay 1y ago
we need to stop it with the anecdotal evidence presented by one random dude
- seeEllArr 1y ago
  [dead]
vendiddy 1y ago
I find o3 to be the clearest thinker as well. If I'm working on a complex problem and want to go back and forth on software architecture, I like having o3 research prior art and have a back and forth on trade-offs. If o3 was faster and cheaper I'd use it a lot more. I'm curious what your workflows are !
jonplackett 1y ago
How do you find o3 vs o4-mini?
- joshmlewis 1y ago
  For coding at least, I don't bother with anything less than the top thinking models. They do have their place for some tasks in agentic systems but time is money and I don't want to waste time trying to coral less skilled models when there are more powerful ones available.
  - jonplackett 1y ago
    I have the same logic but opposite conclusion - o3 just takes SO LONG to respond that I often just use o4-mini

throwaway314155 1y ago

It's interesting you say that because o3, while being a considerable improvement over OpenAI's other models, still doesn't match the performance of Opus 4 and Gemini 2.5 Pro by a long shot for me. However, o3 resides in the ChatGPT app, which is still superior to the other chat apps in many ways, particularly the internet search implementation works very well.

joshmlewis 1y ago
What languages do you use it with and IDE? I use it in Cursor mainly with Max reasoning on. I spent around $300 on token based usage for o3 alone in May still only accepting around 33% of suggestions though. I made a post on X about this the other day but I expect that amount of rejections will go down significantly by the end of this year at the rate things are going.
- drawnwren 1y ago
  Very strange. I find reasoning has very narrow usefulness for me. It's great to get a project in context or to get oriented in the conversation, but on long conversations I find reasoning starts to add way too much extraneous stuff and get distracted from the task at hand. I think my coding model ranking is something like Claude Code > Claude 4 raw > Gemini > big gap > o4-mini > o3
  - joshmlewis 1y ago
    Claude Code isn't a model in itself. By default it routes some to Opus 4 or Sonnet 4 but mostly Sonnet 4 unless you explicitly set it.
    - drawnwren 1y ago
      I am aware
- throwaway314155 1y ago
  i'm using with python, VS Code (not integrated with claude just basic copilot) and Claude Code. For Gemini i'm using AI studio with repomix to package my code into a single file. I copy files over manually in that workflow. All subscription based, not per token pricing. I'm currently using Claude Max. Can't see myself exhausting its usage at this rate but who knows.
svachalek 1y ago
If you're coding through chat apps you're really behind the times. Try an agent IDE or plugin.
- joshmlewis 1y ago
  Yeah, exactly. For everyone who might not know, the chat apps add lots of complex system prompting to handle and shape personality, tone, general usability, etc. IDE's also do this (with Claude Code being one of the ones that are closest to "bare" model that you can get) but at they are at least guiding it's behavior to be really good at coding tasks. Another reason is using the Agent feature that IDE's have had for a few months now which gives it the ability to search/read/edit files across your codebase. You may not like the idea of this and it feels like losing control, but it's the future. After months of using it I've learned how to get it to do what I want but I think a lot of people who try it once and stop get frustrated that it does something dumb and just assume it's not good. That's a practice and skill problem not a model problem.
  - Workaccount2 1y ago
    IDE's are intimidating to non-tech people. I'm surprised there isn't a VibeIDE yet that is purpose build to make it possible for your grandmother to execute code output by an LLM.
    - dragonwriter 1y ago
      > I'm surprised there isn't a VibeIDE yet that is purpose build to make it possible for your grandmother to execute code output by an LLM. The major LLM chat interfaces often have code execution built in, so there kind of is, it just doesn't look like what an SWE thinks of as an IDE.
    - joshmlewis 1y ago
      I have not used them but I feel like there are tools like Replit, Lovable, etc that are for that audience. I totally agree IDE's are intimidating for non-technical people though. Claude Code is pretty cool in that way where it's one command to install and pretty easy to get started with.
jorvi 1y ago
What's most annoying about Gemini 2.5 is that it is obnoxiously verbose compared to Opus 4. Both in explaining the code it wrote and the amount of lines it writes and comments it adds, to the point where the output is often 2-3x more than Opus 4. You can obviously alleviate this by asking it to be more concise but even then it bleeds through sometimes.
- joshmlewis 1y ago
  Yes this is what I mean by conciseness with o3. If prompted well it can produce extremely high level quality code that blows me away at times. I've also had several instances now where I gave it slightly wrong context and other models just butchered a solution with dozens of lines for the proposed fix which I could tell wasn't right and then after reverting and asking o3, it immediately went searching for another file I hadn't included and fixed it in one line. That kind of, dare I say independent thinking, is worth a lot when dealing with complex codebases.
  - jorvi 1y ago
    Personally I still am of the opinion current LLMs are more of a very advanced autocomplete. I have to think of the guy posting that he fed his entire project codebase to an AI, it refactored everything, modularizing it but still reducing the file count from 20 to 12. "It was glorious to see. Nothing worked of course, but glorious nonetheless". In the future I can certainly see it get better and better, especially because code is a hard science that reduces down to control flow logic which reduces down to math. It's a much more narrow problem space than, say, poetry or visuals.

monkpit 1y ago

Have you used Cline with opus+sonnet? Do you have opinions about Claude code vs cline+api? Curious to hear your thoughts!

spaceman_2020 1y ago

I use o3 a lot for basic research and analysis. I also find the deep research tool really useful for even basic shopping research Like just today, it made a list of toys for my toddler that fit her developmental stage and play style. Would have taken me 1-2 hrs of browsing multiple websites otherwise

jml78 1y ago
Gemini deep research runs circles around OpenAI deep research. It goes way deeper and uses way more sources.

pqdbr 1y ago

How do you choose which model to use with Claude Code?

joshmlewis 1y ago
I have the Max $200 plan so I set it to Opus until it limits me to Sonnet 4 which has only happened in two out of a few dozen sessions so far. My rule of thumb in Cursor is it's worth paying for the Max reasoning models for pretty much every request unless it's stupid simple because it produces the best code each time without any funny business you get with cheaper models.
- sunshinerag 1y ago
  You can use the max plan in cursor? I thought it didn’t support calls via api and only worked in Claude code?
  - symbolicAGI 1y ago
    I launch Claude Code in VS Studio (similar to Cursor): > claude Then I use the /login command that opens a browser window to log into Claude Max. You can confirm Claude Max billing going forward in VS Studio/Claude Code: /cost "With your Claude Max subscription, no need to monitor cost — your subscription includes Claude Code usage"
jasonjmcghee 1y ago
In case you're asking for the literal command... /model

VeejayRampay 1y ago

we need to stop it with the anecdotal evidence presented by one random dude

seeEllArr 1y ago
[dead]

vendiddy 1y ago

I find o3 to be the clearest thinker as well. If I'm working on a complex problem and want to go back and forth on software architecture, I like having o3 research prior art and have a back and forth on trade-offs. If o3 was faster and cheaper I'd use it a lot more. I'm curious what your workflows are !

jonplackett 1y ago

How do you find o3 vs o4-mini?

joshmlewis 1y ago
For coding at least, I don't bother with anything less than the top thinking models. They do have their place for some tasks in agentic systems but time is money and I don't want to waste time trying to coral less skilled models when there are more powerful ones available.
- jonplackett 1y ago
  I have the same logic but opposite conclusion - o3 just takes SO LONG to respond that I often just use o4-mini