4 ms·

In my testing the gap between claude and gemini pro 2.5 is close. My company is in asia pacific and we can't get access to claude via vertex for some stupid rea

In my testing the gap between claude and gemini pro 2.5 is close. My company is in asia pacific and we can't get access to claude via vertex for some stupid reason. but i tested it via other providers, the gap used to be huge but now not.

Tostino 1y ago
For me the gap is pretty large (in Gemini Pro 2.5's favor). For reference, the code I am working on is a Spring Boot / (Vaadin) Hilla multi-module project with helm charts for deployment and a separate Python based module for ancillary tasks that were appropriate for it. I've not been able to get any good use out of Sonnet in months now, whereas Gemini Pro 2.5 has (still) been able to grok the project well enough to help out.
- jona777than 1y ago
  I initially found Gemini Pro 2.5 to work well for coding. Over time, I found Claude to be more consistently productive. Gemini Pro 2.5 became my go-to for use cases benefitting from larger context windows. Claude seemed to be the safer daily driver (if I needed to get something done.) All that being said, Gemini has been consistently dependable when I had asks that involved large amounts of code and data. Claude and the OpenAI models struggled with some tasks that Gemini responsively satisfied seemingly without "breaking a sweat." Lately, it's been GPT-5 for brainstorming/planning, Claude for hammering out some code, Gemini when there is huge data/code requirements. I'm curious if the widened Sonnet 4 context window will change things.
- llm_nerd 1y ago
  Opus 4.1 is a much better model for coding than Sonnet. The latter is good for general queries / investigations or to draw up some heuristics. I have paid subscriptions to both Gemini Pro and Claude. Hugely worthwhile expense professionally.
- faangguyindia 1y ago
  when gemini 2.5 pro gets stuck, i often use deep seek r1 in architect mode and qwen3 in coder mode in aider and it solves all the problem last month i ran into some wicked dependency bug and only chatgpt could solve it which i am guessing is the case because it has hot data from github? On the other hand, i really need a tool like aider where i can use various models in "architect" and "coder" mode. what i've found is better reasoning models tend to be bad at writing actual code, and models like qwen3 coder seems better. deep seek r1 will not write reliable code but it will reason well and map out the path forward. i wouldn't be surprised if sonnets success was doing EXACTLY this behind the scenes. but now i am looking for pure models who do not use this black magic hack behind API. I want more control at tool end where i can alter the prompts and achieve results i want this is one reason i do not use claude code etc.... aider is 80% of what i want wish it had more of what i want though. i just don't know why no one has build a perfect solution to this yet. Here are things i am missing in aider 1. Automatic model switching, use different models for asking questions about the code, different one for planning a feature, different one for writing actual code. 2. Self determine, if a feature needs a "reasoning" model or coding model will suffice. 3. be able to do more, selectively send context and drop the files we don't need. Intelligently add files to context which will be touched by the feature, not after having done all code planning asking to add files, then doing it all over again with more context available.
penguin202 1y ago
Claude doesn't have a mid-life crisis and try to `rm -rf /` or delete your project.
film42 1y ago
Agree but pricing wise, Gemini 2.5 pro wins. Gemini input tokens are half the cost of Claude 4. Output is $5/million cheaper than Claude. But, document processing is significantly cheaper. A 5MB PDF (customer invoice) with Gemini is like 5k tokens vs 56k with Claude. The only downside with Gemini (and it's a big one) is availability. We get rate limited by their dynamic QoS all the time even if we haven't reached our quota. Our GCP sales rep keeps recommending "provisioned throughput," but it's both expensive, and doesn't fit our workload type. Plus, the VertexAI SDK is kind of a PITA compared to Anthropic.
- Alex-Programs 1y ago
  Google products are such a pain to work with from an API perspective that I actively avoid them where possible.

Tostino 1y ago

For me the gap is pretty large (in Gemini Pro 2.5's favor). For reference, the code I am working on is a Spring Boot / (Vaadin) Hilla multi-module project with helm charts for deployment and a separate Python based module for ancillary tasks that were appropriate for it. I've not been able to get any good use out of Sonnet in months now, whereas Gemini Pro 2.5 has (still) been able to grok the project well enough to help out.

jona777than 1y ago
I initially found Gemini Pro 2.5 to work well for coding. Over time, I found Claude to be more consistently productive. Gemini Pro 2.5 became my go-to for use cases benefitting from larger context windows. Claude seemed to be the safer daily driver (if I needed to get something done.) All that being said, Gemini has been consistently dependable when I had asks that involved large amounts of code and data. Claude and the OpenAI models struggled with some tasks that Gemini responsively satisfied seemingly without "breaking a sweat." Lately, it's been GPT-5 for brainstorming/planning, Claude for hammering out some code, Gemini when there is huge data/code requirements. I'm curious if the widened Sonnet 4 context window will change things.
llm_nerd 1y ago
Opus 4.1 is a much better model for coding than Sonnet. The latter is good for general queries / investigations or to draw up some heuristics. I have paid subscriptions to both Gemini Pro and Claude. Hugely worthwhile expense professionally.
faangguyindia 1y ago
when gemini 2.5 pro gets stuck, i often use deep seek r1 in architect mode and qwen3 in coder mode in aider and it solves all the problem last month i ran into some wicked dependency bug and only chatgpt could solve it which i am guessing is the case because it has hot data from github? On the other hand, i really need a tool like aider where i can use various models in "architect" and "coder" mode. what i've found is better reasoning models tend to be bad at writing actual code, and models like qwen3 coder seems better. deep seek r1 will not write reliable code but it will reason well and map out the path forward. i wouldn't be surprised if sonnets success was doing EXACTLY this behind the scenes. but now i am looking for pure models who do not use this black magic hack behind API. I want more control at tool end where i can alter the prompts and achieve results i want this is one reason i do not use claude code etc.... aider is 80% of what i want wish it had more of what i want though. i just don't know why no one has build a perfect solution to this yet. Here are things i am missing in aider 1. Automatic model switching, use different models for asking questions about the code, different one for planning a feature, different one for writing actual code. 2. Self determine, if a feature needs a "reasoning" model or coding model will suffice. 3. be able to do more, selectively send context and drop the files we don't need. Intelligently add files to context which will be touched by the feature, not after having done all code planning asking to add files, then doing it all over again with more context available.

penguin202 1y ago

Claude doesn't have a mid-life crisis and try to `rm -rf /` or delete your project.

film42 1y ago

Agree but pricing wise, Gemini 2.5 pro wins. Gemini input tokens are half the cost of Claude 4. Output is $5/million cheaper than Claude. But, document processing is significantly cheaper. A 5MB PDF (customer invoice) with Gemini is like 5k tokens vs 56k with Claude. The only downside with Gemini (and it's a big one) is availability. We get rate limited by their dynamic QoS all the time even if we haven't reached our quota. Our GCP sales rep keeps recommending "provisioned throughput," but it's both expensive, and doesn't fit our workload type. Plus, the VertexAI SDK is kind of a PITA compared to Anthropic.

Alex-Programs 1y ago
Google products are such a pain to work with from an API perspective that I actively avoid them where possible.