3 ms·
How do you handle context limits? With more thinking tokens you fill it up earlier. Compaction degrades performance too. What's your strategy?
by FooBarWidget 2mo ago
How do you handle context limits? With more thinking tokens you fill it up earlier. Compaction degrades performance too. What's your strategy?
- edg5000 2mo agoInitially I was planning heavily around context limits, but I've learned to just ignore it completely. Compaction is seamless for me. If details are lost in compaction, the model just re-reads what's needed. My conclusion is that at least for Sol, the summaries (which I've never seen) must be amazing. Every now and then a detail gets lost and I have to repeat it. I don't think there is performance degration, because the model is smart enough to re-read relevant files as needed.