2 ms·
From the anecdotes there, it sounds a whole lot like something is going on where it's internally doing a summarizing step to fit into a much smaller actual cont
by crooked-v 3y ago
From the anecdotes there, it sounds a whole lot like something is going on where it's internally doing a summarizing step to fit into a much smaller actual context window than 128K.