4 ms·
Thanks for checking it out. Right now aider doesn't even try and deal with files larger than the context window. For gpt-4, that's 8k tokens or about 32kbytes.
by anotherpaulg 3y ago
Thanks for checking it out.
Right now aider doesn't even try and deal with files larger than the context window. For gpt-4, that's 8k tokens or about 32kbytes. Which is pretty reasonable.
According to the data in [1], the average source file in github is less than 14KB. It's worth noting that they explicitly discarded small files with less than 10 lines. So the true average is probably much lower.
Regardless, I haven't found the gpt-4 context window issue to be problematic in practice yet. You do need to be careful about how many files you "add to the chat" at once. But that's not hard. For sure there are cases where you would need to refactor a large file before you could use it with aider.
I am certainly interested added context window management features to aider. I have previously explored a bunch of approaches for this with gpt-3.5-turbo. I shared some notes about these experiments previously on HN [2].
[1] https://hoffa.medium.com/400-000-github-repositories-1-billion-files-14-terabytes-of-code-spaces-or-tabs-7cfe0b5dd7fd https://hoffa.medium.com/400-000-github-repositories-1-billi...
[2] https://news.ycombinator.com/item?id=35441666 https://news.ycombinator.com/item?id=35441666
- panarky 3y agoFor chat, its common to use gpt to summarize previous turns so they consume fewer tokens, while not losing the context completely. Seems like that could work even better for code, where gpt could compactly summarize what functions do without memorizing all the code.
- anotherpaulg 3y agoThanks for the thoughts on this. Ya, I am familiar with the technique of summarizing the chat history. It's on my todo list for aider. In the [2] link I included above, I discuss the approach you suggest of summarizing the meat of most code. It didn't work well with gpt-3.5-turbo, but I do plan to revisit it again with gpt-4 at some point. Again, I haven't actually exhausted the context window often in my own use of aider. So it hasn't been a problem in practice. But no doubt it will be an issue for certain code bases and source files. I am quite interested in finding an effective solution, and plan to put in more effort here in the future.