2 ms·
The issue isn't the nominal size of the context window which is easy to objectively measure, but effective use of the context window, which is harder to measure
by dragonwriter 2y ago
The issue isn't the nominal size of the context window which is easy to objectively measure, but effective use of the context window, which is harder to measure precisely, but a big issue: it doesn't matter how much stuff you can throw at the model if it effectively ignores most of it past a certain point.