12 ms·
Right- "forgetting" assuming a rolling context window maxed at the models max token count. "If it doesn't fit then it's game over" - assuming 8k tokens with eac
by jasonjmcghee 4y ago
Right- "forgetting" assuming a rolling context window maxed at the models max token count. "If it doesn't fit then it's game over" - assuming 8k tokens with each token being ~4 characters, that's a pretty small repo. And that's a motivator behind the similarity search approach.