4 ms·
You should be doing everything you can to keep context under 200k, ideally even 100k. All the models unwind so badly as context grows.
by Workaccount2 10mo ago
You should be doing everything you can to keep context under 200k, ideally even 100k. All the models unwind so badly as context grows.
- patates 10mo agoI don't have that experience with gemini. Up to 90% full, it's just fine.
- tgtweak 10mo agoIf the models are designed around it, and not resorting to compression to get to higher input token lengths, they don't 'fall off' as they get near the context window limit. When working with large codebases, exhausting or compressing the context actually causes more issues since the agent forgets what was in the other libraries and files. Google has realized this internally and were among the first to get to 2M token context length (internally then later released publicly).