3 ms·
For chat, its common to use gpt to summarize previous turns so they consume fewer tokens, while not losing the context completely. Seems like that could work e
by panarky 3y ago
For chat, its common to use gpt to summarize previous turns so they consume fewer tokens, while not losing the context completely.
Seems like that could work even better for code, where gpt could compactly summarize what functions do without memorizing all the code.
- anotherpaulg 3y agoThanks for the thoughts on this. Ya, I am familiar with the technique of summarizing the chat history. It's on my todo list for aider. In the [2] link I included above, I discuss the approach you suggest of summarizing the meat of most code. It didn't work well with gpt-3.5-turbo, but I do plan to revisit it again with gpt-4 at some point. Again, I haven't actually exhausted the context window often in my own use of aider. So it hasn't been a problem in practice. But no doubt it will be an issue for certain code bases and source files. I am quite interested in finding an effective solution, and plan to put in more effort here in the future.