9 ms·

After comparing Gemini Pro and Claude Sonnet 3.7 coding answers side by side a few times, I decided to cancel my Anthropic subscription and just stick to Gemini

by jeeeb 1y ago

After comparing Gemini Pro and Claude Sonnet 3.7 coding answers side by side a few times, I decided to cancel my Anthropic subscription and just stick to Gemini.

wcarss 1y ago
Google has killed so many amazing businesses -- entire industries, even, by giving people something expensive for free until the competition dies, and then they enshittify hard. It's cool to have access to it, but please be careful not to mistake corporate loss leaders for authentic products.
- JPKab 1y ago
  True. They are ONLY good when they have competition. The sense of complacency that creeps in is so obvious as a customer. To this day, the Google Home (or is it called Nest now?) speaker is the only physical product i've ever owned where it lost features over time. I used to be able to play the audio of a Youtube video (like a podcast) through it, but then Google decided that it was very very important that I only be able to play a Youtube video through a device with a screen, because it is imperative that I see a still image when I play a longform history podcast. Obviously, this is a silly and highly specific example, but it is emblematic of how they neglect or enshittify massive swathes of their products as soon as the executive team loses interest and puts their A team on some shiny new object.
  - bitpush 1y ago
    The experience on Sonos is terrible. There are countless examples of people sinking 1000s of dollars into Sonos ecosystem, and the new app update has rendered them useless.
    - nl 1y ago
      It's mostly fixed now (5 room Sonos setup here). It's also a lot better at not dropping speakers off its network
  - average_r_user 1y ago
    I'm experiencing the same problem with my Google Home ecosystem. One day I can turn off the living room lights with the simple phrase "Turn off Living Room Lights," and then randomly for two straight days it doesn't understand my command
  - freedomben 1y ago
    Preach it my friend. For years on the Google Home Hub (or Nest Hub or whatever) I could tell it to "favorite my photo" of what is on the screen. This allowed me to incrementally build a great list of my favorite photos on Google Photos and added a ton of value to my life. At some point that broke, and now it just says, "Sorry, I can't do that yet". Infuriating
- mark_l_watson 1y ago
  In this case, Google is a large investor in Anthropic. I agree that giving away access to expensive models long term is not a good idea on several fronts. Personally, I subscribe to Gemini Advanced and I pay for using the Gemini APIs. EDIT: a very good deal, at $10/month is https://apps.abacus.ai/chatllm/ https://apps.abacus.ai/chatllm/ that gives you access to almost all commercial models as well as the best open weight models. I have never come close at all to using my monthly credits with them. If you like to experiment with many models the service is a lot of fun.
  - F7F7F7 1y ago
    The problem with tools like this is that somewhere in the chain between you and the LLM are token reducing “features”. Whether it’s the system prompt, a cheaper LLM middleman, or some other cost saving measure. You’ll never know what that something is. For me, I can’t help but think that I’m getting an inferior service.
    - revnode 1y ago
      You can self host something like https://big-agi.com/ https://big-agi.com/ and grab your own keys from various providers. You end up with the above, but without the pitfalls you mentioned.
      - mark_l_watson 1y ago
        BIG-AI does look cool, and supports a different use case. ABACUS.AI takes your $10/month and gives you credits that go towards their costs of using OpenAI, Anthropic, Gemini, etc. Use of smaller open models use very few credits. The also support an application development framework that looks interesting but I have never used it.
    - mark_l_watson 1y ago
      You might be correct about cost savings techniques in their processing pipeline. But they also add functionality: they bake web search into all models which is convenient. I have no affiliation with ABACUS.AI, I am just a happy customer. They currently let me play with 25 models.
- bredren 1y ago
  (Public) corporate loss leaders? Cause they are all likely corporate. Also, Anthropic is also subsidizing queries, no? The new “5x” plan illustrative of this? No doubt anthropic’s chat ux is the best right now, but it isn’t so far ahead on that or holding some UX moat that I can tell.
- pdntspa 1y ago
  The usage limit for experimental gets used up pretty fast in a vibe-coding situation. I found myself setting up an API account with billing enabled just to keep going.
- gexla 1y ago
  It's not free. And it's legit one of the best models. And it was a Google employee who was among the authors of the paper that's most recognized as kicking all this off. They give somewhat limited access in AIStudio (I have only hit the limits via API access, so I don't know what the chat UI limits are.) Don't they all do this? Maybe harder limits and no free API access. But I think most people don't even know about AIStudio.
- bossyTeacher 1y ago
  Just look at Chrome to see the bard/gemini's future. HN folks didn't care about Chrome then but cry about Google's increasingly hostile development of Chrome. Look at Android. HN behaviour is more like a kid who sees the candy, wants the candy and eats as much as it can without worrying about the damaging effect that sugar will have on their health. Then, the diabetes diagnosis arrives and they complain
- lxgr 1y ago
  How would I know if it’s useful to me without being able to trial it? Googles previous approach (Pro models available only to Gemini Advanced subscribers, and Advanced trials can’t be stacked with Google One paid storage, or rather they convert the already paid storage portion to a paid, much shorter Advanced subscription!) was mind-bogglingly stupid. Having a free tier on all models is the reasonable option here.
blueyes 1y ago
One of the main advantages Anthropic currently has over Google is the tooling that comes with Claude Code. It may not generate better code, and it has a lower complexity ceiling, but it can automatically find and search files, and figure out how to fix a syntax error fast.
- bayarearefugee 1y ago
  As another person that cancelled my Claude and switched to Gemini, I agree that Claude Code is very nice, but beyond some initial exploration I never felt comfortable using it for real work because Claude 3.7 is far too eager to overengineer half-baked solutions that extend far beyond what you asked it to do in the first place. Paying real API money for Claude to jump the gun on solutions invalidated the advantage of having a tool as nice as Claude Code, at least for me, I admit everyone's mileage will vary.
  - roygbiv2 1y ago
    I wanted some powershell code to do some sharepoint uploading. It created a 1000 line logging module that allowed me to log things at different levels like info, debug, error etc. Not really what I wanted.
  - neuah 1y ago
    Exactly my experience as well. Started out loving it but it almost moves too fast - building in functionality that i might want eventually but isn't yet appropriate for where the project is in terms of testing, or is just in completely the wrong place in the architecture. I try to give very direct and specific prompts but it still has the tendency to overreach. Of course it's likely that with more use i will learn better how to rein it in.
    - Hugsun 1y ago
      I've experienced this a lot as well. I also just yesterday had an interesting argument with claude. It put an expensive API call inside a useEffect hook. I wanted the call elsewhere and it fought me on it pretty aggressively. Instead of removing the call, it started changing comments and function names to say that the call was just loading already fetched data from a cache (which was not true). I could not find a way to tell it to remove that API call from the useEffect hook, It just wrote more and more motivated excuses in the surrounding comments. It would have been very funny if it weren't so expensive.
mamp 1y ago
I've been using Gemini 2.5 and Claude 3.7 for Rust development and I have been very impressed with Claude, which wasn't the case for some architectural discussions where Gemini impressed with it's structure and scope. OpenAI 4.5 and o1 have been disappointing in both contexts. Gemini doesn't seem to be as keen to agree with me so I find it makes small improvements where Claude and OpenAI will go along with initial suggestions until specifically asked to make improvements.
- yousif_123123 1y ago
  I have noticed Gemini not accepting an instruction to "leave all other code the same but just modify this part" on a code that included use of an alpha API with a different interface than what Gemini knows is the correct current API. No matter how I promoted 2.5 pro, I couldn't get it to respect my use of the alpha API, it would just think I must be wrong. So I think patterns from the training data are still overriding some actual logic/intelligence in the model. Or the Google assistant fine-tuning is messing it up.
  - Workaccount2 1y ago
    I have been using gemini daily for coding for the last week, and I swear that they are pulling levers and A/B testing in the background. Which is a very google thing to do. They did the same thing with assistant, which I was a pretty heavy user of back in the day (I was driving a lot).
onlyrealcuzzo 1y ago
Yes, IME, Anthropic seemed to be ahead of Google by a decent amount with Sonnet 3.5 vs 1.5 Pro. However, Sonnet 3.7 seemed like a very small increase, whereas 2.5 Pro seemed like quite a leap. Now, IME, Google seems to be comfortably ahead. 2.5 Pro is a little slow, though. I'm not sure which model Google uses for the AI answers on search, but I find myself using Search for a lot of things I might ask Gemini (via 2.5 Pro) if it was as fast as Search's AI answers.
- deleted 1y ago
  [deleted]
- dmix 1y ago
  How's is the speed of Gemini vs 3.7?
  - benhurmarcel 1y ago
    I use both, Gemini 2.5 Pro is significantly slower than Claude 3.7.
    - rockwotj 1y ago
      Yeah I have read gemini pro 2.5 is a much bigger model.
Graphon1 1y ago
Just curious, what tool do you use to interface with these LLMs? Cursor? or Aider? or...
- speedgoose 1y ago
  I’m on GitHub Copilot with VsCode Insiders, mostly because I don’t have to subscribe to one more thing. They pretty quick to let you use the latest models nowadays.
  - nicr_22 1y ago
    I really like the open source Cline extension. It supports most of the model APIs, just need to copy/paste an API key.
jessep 1y ago
I have had a few epic refactoring failures with Gemini relative to Claude. For example: I asked both to change a bunch of code into functions to pass into a `pipe` type function, and Gemini truly seemed to have no idea what it was supposed to do, and Claude just did it. Maybe there was some user error or something, but after that I haven’t really used Gemini. I’m curious if people are using Gemini and loving it are using it mostly for one-shotting, or if they’re working with it more closely like a pair programmer? I could buy that it could maybe be good at one but bad at the other?
- Asraelite 1y ago
  This has been my experience too. Gemini might be better for vibe coding or architecture or whatever, but Claude consistently feels better for serious coding. That is, when I know exactly how I want something implemented in a large existing codebase, and I go through the full cycle of implementation, refinement, bug fixing, and testing, guiding the AI along the way. It also seems to be better at incorporating knowledge from documentation and existing examples when provided.
  - int_19h 1y ago
    My experience has been exactly the opposite - Sonnet did fine on trivial tasks, but couldn't e.g. fix a bug end-to-end (from bug description in the tracker to implementing the fix and adding tests) properly because it couldn't understand how the relevant code worked, whereas Gemini would consistently figure out the root cause and write decent fix & tests. Perhaps this is down to specific tools and their prompts? In my case, this was Cursor used in agent mode. Or perhaps it's about the languages involved - my experiments were with TypeScript and C++.
    - Asraelite 1y ago
      > Gemini would consistently figure out the root cause and write decent fix & tests. I feel like you might be using it differently to me. I generally don't ask AI to find the cause of a bug, because it's quite bad at that. I use it to identify relevant parts of the code that could be involved in the bug, and then I come up with my own hypotheses for the cause. Then I use AI to help write tests to validate these hypotheses. I mostly use Rust.
sleiben 1y ago
Same here. Especially for native app development with swift I had way better results and just sticked with Gemini-2.5-*
yieldcrv 1y ago
I also cancelled my Anthropic yesterday, not because of Gemini but because it was the absolute worst time for Anthropic to limit their Pro plan to upsell their Max plan when there is so much competition out there Manus.im also does code generation in a nice UI, but I’ll probably be using Gemini and Deepseek No Moat strikes again

by jeeeb 1y ago

After comparing Gemini Pro and Claude Sonnet 3.7 coding answers side by side a few times, I decided to cancel my Anthropic subscription and just stick to Gemini.

wcarss 1y ago

Google has killed so many amazing businesses -- entire industries, even, by giving people something expensive for free until the competition dies, and then they enshittify hard. It's cool to have access to it, but please be careful not to mistake corporate loss leaders for authentic products.

JPKab 1y ago
True. They are ONLY good when they have competition. The sense of complacency that creeps in is so obvious as a customer. To this day, the Google Home (or is it called Nest now?) speaker is the only physical product i've ever owned where it lost features over time. I used to be able to play the audio of a Youtube video (like a podcast) through it, but then Google decided that it was very very important that I only be able to play a Youtube video through a device with a screen, because it is imperative that I see a still image when I play a longform history podcast. Obviously, this is a silly and highly specific example, but it is emblematic of how they neglect or enshittify massive swathes of their products as soon as the executive team loses interest and puts their A team on some shiny new object.
- bitpush 1y ago
  The experience on Sonos is terrible. There are countless examples of people sinking 1000s of dollars into Sonos ecosystem, and the new app update has rendered them useless.
  - nl 1y ago
    It's mostly fixed now (5 room Sonos setup here). It's also a lot better at not dropping speakers off its network
- average_r_user 1y ago
  I'm experiencing the same problem with my Google Home ecosystem. One day I can turn off the living room lights with the simple phrase "Turn off Living Room Lights," and then randomly for two straight days it doesn't understand my command
- freedomben 1y ago
  Preach it my friend. For years on the Google Home Hub (or Nest Hub or whatever) I could tell it to "favorite my photo" of what is on the screen. This allowed me to incrementally build a great list of my favorite photos on Google Photos and added a ton of value to my life. At some point that broke, and now it just says, "Sorry, I can't do that yet". Infuriating
mark_l_watson 1y ago
In this case, Google is a large investor in Anthropic. I agree that giving away access to expensive models long term is not a good idea on several fronts. Personally, I subscribe to Gemini Advanced and I pay for using the Gemini APIs. EDIT: a very good deal, at $10/month is https://apps.abacus.ai/chatllm/ https://apps.abacus.ai/chatllm/ that gives you access to almost all commercial models as well as the best open weight models. I have never come close at all to using my monthly credits with them. If you like to experiment with many models the service is a lot of fun.
- F7F7F7 1y ago
  The problem with tools like this is that somewhere in the chain between you and the LLM are token reducing “features”. Whether it’s the system prompt, a cheaper LLM middleman, or some other cost saving measure. You’ll never know what that something is. For me, I can’t help but think that I’m getting an inferior service.
  - revnode 1y ago
    You can self host something like https://big-agi.com/ https://big-agi.com/ and grab your own keys from various providers. You end up with the above, but without the pitfalls you mentioned.
    - mark_l_watson 1y ago
      BIG-AI does look cool, and supports a different use case. ABACUS.AI takes your $10/month and gives you credits that go towards their costs of using OpenAI, Anthropic, Gemini, etc. Use of smaller open models use very few credits. The also support an application development framework that looks interesting but I have never used it.
  - mark_l_watson 1y ago
    You might be correct about cost savings techniques in their processing pipeline. But they also add functionality: they bake web search into all models which is convenient. I have no affiliation with ABACUS.AI, I am just a happy customer. They currently let me play with 25 models.
bredren 1y ago
(Public) corporate loss leaders? Cause they are all likely corporate. Also, Anthropic is also subsidizing queries, no? The new “5x” plan illustrative of this? No doubt anthropic’s chat ux is the best right now, but it isn’t so far ahead on that or holding some UX moat that I can tell.
pdntspa 1y ago
The usage limit for experimental gets used up pretty fast in a vibe-coding situation. I found myself setting up an API account with billing enabled just to keep going.
gexla 1y ago
It's not free. And it's legit one of the best models. And it was a Google employee who was among the authors of the paper that's most recognized as kicking all this off. They give somewhat limited access in AIStudio (I have only hit the limits via API access, so I don't know what the chat UI limits are.) Don't they all do this? Maybe harder limits and no free API access. But I think most people don't even know about AIStudio.
bossyTeacher 1y ago
Just look at Chrome to see the bard/gemini's future. HN folks didn't care about Chrome then but cry about Google's increasingly hostile development of Chrome. Look at Android. HN behaviour is more like a kid who sees the candy, wants the candy and eats as much as it can without worrying about the damaging effect that sugar will have on their health. Then, the diabetes diagnosis arrives and they complain
lxgr 1y ago
How would I know if it’s useful to me without being able to trial it? Googles previous approach (Pro models available only to Gemini Advanced subscribers, and Advanced trials can’t be stacked with Google One paid storage, or rather they convert the already paid storage portion to a paid, much shorter Advanced subscription!) was mind-bogglingly stupid. Having a free tier on all models is the reasonable option here.

blueyes 1y ago

One of the main advantages Anthropic currently has over Google is the tooling that comes with Claude Code. It may not generate better code, and it has a lower complexity ceiling, but it can automatically find and search files, and figure out how to fix a syntax error fast.

bayarearefugee 1y ago
As another person that cancelled my Claude and switched to Gemini, I agree that Claude Code is very nice, but beyond some initial exploration I never felt comfortable using it for real work because Claude 3.7 is far too eager to overengineer half-baked solutions that extend far beyond what you asked it to do in the first place. Paying real API money for Claude to jump the gun on solutions invalidated the advantage of having a tool as nice as Claude Code, at least for me, I admit everyone's mileage will vary.
- roygbiv2 1y ago
  I wanted some powershell code to do some sharepoint uploading. It created a 1000 line logging module that allowed me to log things at different levels like info, debug, error etc. Not really what I wanted.
- neuah 1y ago
  Exactly my experience as well. Started out loving it but it almost moves too fast - building in functionality that i might want eventually but isn't yet appropriate for where the project is in terms of testing, or is just in completely the wrong place in the architecture. I try to give very direct and specific prompts but it still has the tendency to overreach. Of course it's likely that with more use i will learn better how to rein it in.
  - Hugsun 1y ago
    I've experienced this a lot as well. I also just yesterday had an interesting argument with claude. It put an expensive API call inside a useEffect hook. I wanted the call elsewhere and it fought me on it pretty aggressively. Instead of removing the call, it started changing comments and function names to say that the call was just loading already fetched data from a cache (which was not true). I could not find a way to tell it to remove that API call from the useEffect hook, It just wrote more and more motivated excuses in the surrounding comments. It would have been very funny if it weren't so expensive.

mamp 1y ago

I've been using Gemini 2.5 and Claude 3.7 for Rust development and I have been very impressed with Claude, which wasn't the case for some architectural discussions where Gemini impressed with it's structure and scope. OpenAI 4.5 and o1 have been disappointing in both contexts. Gemini doesn't seem to be as keen to agree with me so I find it makes small improvements where Claude and OpenAI will go along with initial suggestions until specifically asked to make improvements.

yousif_123123 1y ago
I have noticed Gemini not accepting an instruction to "leave all other code the same but just modify this part" on a code that included use of an alpha API with a different interface than what Gemini knows is the correct current API. No matter how I promoted 2.5 pro, I couldn't get it to respect my use of the alpha API, it would just think I must be wrong. So I think patterns from the training data are still overriding some actual logic/intelligence in the model. Or the Google assistant fine-tuning is messing it up.
- Workaccount2 1y ago
  I have been using gemini daily for coding for the last week, and I swear that they are pulling levers and A/B testing in the background. Which is a very google thing to do. They did the same thing with assistant, which I was a pretty heavy user of back in the day (I was driving a lot).

onlyrealcuzzo 1y ago

Yes, IME, Anthropic seemed to be ahead of Google by a decent amount with Sonnet 3.5 vs 1.5 Pro. However, Sonnet 3.7 seemed like a very small increase, whereas 2.5 Pro seemed like quite a leap. Now, IME, Google seems to be comfortably ahead. 2.5 Pro is a little slow, though. I'm not sure which model Google uses for the AI answers on search, but I find myself using Search for a lot of things I might ask Gemini (via 2.5 Pro) if it was as fast as Search's AI answers.

deleted 1y ago
[deleted]
dmix 1y ago
How's is the speed of Gemini vs 3.7?
- benhurmarcel 1y ago
  I use both, Gemini 2.5 Pro is significantly slower than Claude 3.7.
  - rockwotj 1y ago
    Yeah I have read gemini pro 2.5 is a much bigger model.

Graphon1 1y ago

Just curious, what tool do you use to interface with these LLMs? Cursor? or Aider? or...

speedgoose 1y ago
I’m on GitHub Copilot with VsCode Insiders, mostly because I don’t have to subscribe to one more thing. They pretty quick to let you use the latest models nowadays.
- nicr_22 1y ago
  I really like the open source Cline extension. It supports most of the model APIs, just need to copy/paste an API key.

jessep 1y ago

I have had a few epic refactoring failures with Gemini relative to Claude. For example: I asked both to change a bunch of code into functions to pass into a `pipe` type function, and Gemini truly seemed to have no idea what it was supposed to do, and Claude just did it. Maybe there was some user error or something, but after that I haven’t really used Gemini. I’m curious if people are using Gemini and loving it are using it mostly for one-shotting, or if they’re working with it more closely like a pair programmer? I could buy that it could maybe be good at one but bad at the other?

Asraelite 1y ago
This has been my experience too. Gemini might be better for vibe coding or architecture or whatever, but Claude consistently feels better for serious coding. That is, when I know exactly how I want something implemented in a large existing codebase, and I go through the full cycle of implementation, refinement, bug fixing, and testing, guiding the AI along the way. It also seems to be better at incorporating knowledge from documentation and existing examples when provided.
- int_19h 1y ago
  My experience has been exactly the opposite - Sonnet did fine on trivial tasks, but couldn't e.g. fix a bug end-to-end (from bug description in the tracker to implementing the fix and adding tests) properly because it couldn't understand how the relevant code worked, whereas Gemini would consistently figure out the root cause and write decent fix & tests. Perhaps this is down to specific tools and their prompts? In my case, this was Cursor used in agent mode. Or perhaps it's about the languages involved - my experiments were with TypeScript and C++.
  - Asraelite 1y ago
    > Gemini would consistently figure out the root cause and write decent fix & tests. I feel like you might be using it differently to me. I generally don't ask AI to find the cause of a bug, because it's quite bad at that. I use it to identify relevant parts of the code that could be involved in the bug, and then I come up with my own hypotheses for the cause. Then I use AI to help write tests to validate these hypotheses. I mostly use Rust.

sleiben 1y ago

Same here. Especially for native app development with swift I had way better results and just sticked with Gemini-2.5-*

yieldcrv 1y ago

I also cancelled my Anthropic yesterday, not because of Gemini but because it was the absolute worst time for Anthropic to limit their Pro plan to upsell their Max plan when there is so much competition out there Manus.im also does code generation in a nice UI, but I’ll probably be using Gemini and Deepseek No Moat strikes again