4 ms·
The legend on the graph lists edit format diff/whole, but I assume that's an error? Which method was used for these tests? Either way, the new model is looking
by reitzensteinm 3y ago
The legend on the graph lists edit format diff/whole, but I assume that's an error? Which method was used for these tests?
Either way, the new model is looking very promising.
- anotherpaulg 3y agoAider uses different prompting strategies for GPT-3.5 and 4. GPT-3.5 is only able to reliably edit a file by returning a whole new copy of the file with the edits included. This is the "whole" edit format. GPT-4 is able to use a more efficient "diff" edit format, where it species blocks of code to search and replace. All of this is described and quantified in more detail in the original aider benchmarking article. https://aider.chat/docs/benchmarks.html https://aider.chat/docs/benchmarks.html
- reitzensteinm 3y agoMakes sense, thanks. I've so far stuck with "whole" for my own experiments, which is slower and more expensive, but it's a more asynchronous use case compared to Aider.