I find gemini 2.5 pro starts losing its shit around 50K tokens, using Roo Code. Between Roo's system prompt and my AGENTS.md there's probably about 10k used off

I find gemini 2.5 pro starts losing its shit around 50K tokens, using Roo Code. Between Roo's system prompt and my AGENTS.md there's probably about 10k used off the bat. So I have about 30-40k tokens to complete whatever task I assign it. It's a workable limit but I really wish I could get more out of a single thread before it goes crazy. Does this match others' experience?