4 ms·
They nerfed Pro 2.5 significantly in the last few months. Early this year, I had genuinely insightful conversations with Gemini 2.5 Pro. Now they are mostly fru
by dr_kiszonka 1y ago
They nerfed Pro 2.5 significantly in the last few months. Early this year, I had genuinely insightful conversations with Gemini 2.5 Pro. Now they are mostly frustrating.
I also have a personal conspiracy theory, i.e., that once a user exceeds a certain use threshold of 2.5 Pro in the Google Gemini app, they start serving a quantized version. Of course, I have no proof, but it certainly feels that way.
- esafak 1y agoI wonder how smart they are about quantizing. Do they look at feedback to decide which users won't mind?
- conradkay 1y agoMaybe they've been focusing so much on improving coding performance with RL for the new versions/previews that other areas degraded in performance
- dr_kiszonka 1y agoI think you are right and this is probably the case. Although, given that I rapidly went from +4 to 0 karma, a few other comments in this topic are grey, and at least one is missing, I am getting suspicious. (Or maybe it is just lunch time in MTV.)
- cma 1y agoOne of the early updates improved agentic coding scores while lowering other general benchmark scores, which may have impacted those kind of conversations.
- SirensOfTitan 1y agoThere was a significant nerf of Gemini 3-25 a little while ago, so much so that I detected it without knowing there was even a new release. Totally convinced they quantized the model quietly and improved on the coding benchmark to hide that fact. I’m frankly quite tired of LLM providers changing the model I’m paying for access to behind the scenes, often without informing me, and in Gemini’s case on the API too—at least last time I checked they updated the 3-25 checkpoint to the May update.