4 ms·
Thanks for the hint, but is a large context window actually that useful? I tend to get garbage too often with a normal big context window.
by lukan 6mo ago
Thanks for the hint, but is a large context window actually that useful?
I tend to get garbage too often with a normal big context window.
- stldev 6mo agoIME, based on an in-house bench it's still good to about 20% on the 1M for 4.6 and 4.7 with a code base >50k loc. The trick I used before switching providers was to have it write a handoff when it hit ~18% of context and reset. There are also many people running 4.5 with specific parameters that claim to be having luck.