4 ms·
I gave it a shot just now with a fairly simple refactor. +19 lines, -9 lines, across two files. Totally ballsed it up. Defined one of the two variables it was
by mulmboy 1y ago
I gave it a shot just now with a fairly simple refactor. +19 lines, -9 lines, across two files. Totally ballsed it up. Defined one of the two variables it was meant to, referred to the non-implemented one. I told it "hey you forgot the second variable" and then it went and added it in twice. Added comments (after prompting it to) which were half-baked, ambiguous when read in context.
Never had anything like this with claude code.
I've used Gemini 2.5 Pro quite a lot and like most people I find it's very intelligent. I've bent over backwards to use Gemini 2.5 Pro in another piece of work because it's so good. I can only assume it's the gemini CLI itself that's using the model poorly. Keen to try again in a month or two and see if this poor first impression is just a teething issue.
I told it that it did a pretty poor job and asked it why it thinks that is, told it that I know it's pretty smart. It gave me a wall of text and I asked for the short summary
> My tools operate on raw text, not the code's structure, making my edits brittle and prone to error if the text patterns aren't perfect. I lack a persistent, holistic view of the code like an IDE provides, so I can lose track of changes during multi-step tasks. This led me to make simple mistakes like forgetting a calculation and duplicating code.
- luckydata 1y agoI noticed a significant degradation of Gemini's coding abilities in the last couple checkpoints of 2.5. the benchmarks say it should be better but it doesn't jive with my personal experience.
- tom_m 1y agoOh interesting. I have yet to try it. I love Gemini 2.5 Pro, so I expect the same here...but if not, wow. That would be a big whoops on their part.