3 ms·

My guess is that they've done a lot of tuning to improve diff based code editing. Gemini 2.5 is fantastic at agentic work, but it still is pretty rough around t

My guess is that they've done a lot of tuning to improve diff based code editing. Gemini 2.5 is fantastic at agentic work, but it still is pretty rough around the edges in terms of generating perfectly matching diffs to edit code. It's probably one of the very few issues with the model. Luckily, aider tracks this. They measure the old gemini 2.5 generating proper diffs 92% of the time. I bet this goes up to ~95-98% https://aider.chat/docs/leaderboards/ https://aider.chat/docs/leaderboards/ Question for the google peeps who monitor these threads: Is gemini-2.5-pro-exp (free tier) updated as well, or will it go away? Also, in the blog post, it says: > The previous iteration (03-25) now points to the most recent version (05-06), so no action is required to use the improved model, and it continues to be available at the same price. Does this mean gemini-2.5-pro-preview-03-25 now uses 05-06? Does the same apply to gemini-2.5-pro-exp-03-25? update: I just tried updating the date in the exp model (gemini-2.5-pro-exp-05-06) and that doesnt work.

deleted 1y ago
[deleted]
okdood64 1y ago
What do you mean by agentic work in this context?
- laborcontract 1y ago
  Knowing when to call functions, generating the proper function calling text structure, properly executing functions in sequence, knowing when it's completed its objective, and doing that over an extended context window.
laborcontract 1y ago
Update 2: I've been using this model in both aider and cline and I've haven't gotten a diff matching error yet, even with some pretty difficult substitutions across different places in multiple files. The overall feel of this model is nice. I don't have a formal benchmark but there's a notable improvement in code generation due to this alone. I've had gemini chug away on plans that have taken ~1 hour to implement. (~80mln tokens spent) A good portion of that energy was spent fixing mistakes made by cline/aider/roo due to search/replace mistakes. If this model gets anywhere close to 100% on diffs then this is a BFD. I estimate this will translate to a 50-75% productivity boost on long context coding tasks. I hope the initial results i'm seeing hold up! I'm surprised by the reaction in the rest of the thread. A lot unproductive complaining, a lot of off topic stuff, nothing talking about the model itself. Any thoughts from anyone else using the updated model?
- esperent 1y ago
  Have you been using the Gemini 2.5 pro "Experimental" or "3-25" models in Cline? I've been using both over the last week and got quite a few diff errors, maybe 1/10 of edit so that 92% tracks for me. Does this 2.5 pro "Preview" feel like an improvement if you had used the others?
  - laborcontract 1y ago
    Yep i’ve been using the old and new models in cline. I can’t tell any difference outside of the improvement with diffs, but that’s good enough for me.
- vessenes 1y ago
  Question; are you calling it with “aider -model gemini”? And if so do you see 05-04 listed or the old one?
deleted 1y ago
[deleted]

deleted 1y ago

[deleted]

okdood64 1y ago

What do you mean by agentic work in this context?

laborcontract 1y ago
Knowing when to call functions, generating the proper function calling text structure, properly executing functions in sequence, knowing when it's completed its objective, and doing that over an extended context window.

laborcontract 1y ago

Update 2: I've been using this model in both aider and cline and I've haven't gotten a diff matching error yet, even with some pretty difficult substitutions across different places in multiple files. The overall feel of this model is nice. I don't have a formal benchmark but there's a notable improvement in code generation due to this alone. I've had gemini chug away on plans that have taken ~1 hour to implement. (~80mln tokens spent) A good portion of that energy was spent fixing mistakes made by cline/aider/roo due to search/replace mistakes. If this model gets anywhere close to 100% on diffs then this is a BFD. I estimate this will translate to a 50-75% productivity boost on long context coding tasks. I hope the initial results i'm seeing hold up! I'm surprised by the reaction in the rest of the thread. A lot unproductive complaining, a lot of off topic stuff, nothing talking about the model itself. Any thoughts from anyone else using the updated model?

esperent 1y ago
Have you been using the Gemini 2.5 pro "Experimental" or "3-25" models in Cline? I've been using both over the last week and got quite a few diff errors, maybe 1/10 of edit so that 92% tracks for me. Does this 2.5 pro "Preview" feel like an improvement if you had used the others?
- laborcontract 1y ago
  Yep i’ve been using the old and new models in cline. I can’t tell any difference outside of the improvement with diffs, but that’s good enough for me.
vessenes 1y ago
Question; are you calling it with “aider -model gemini”? And if so do you see 05-04 listed or the old one?

deleted 1y ago

[deleted]