4 ms·
Five noteworthy models have been released in the last few days, with a wide range of code editing capabilities. Here are their results from aider’s code editing
by anotherpaulg 2y ago
Five noteworthy models have been released in the last few days, with a wide range of code editing capabilities. Here are their results from aider’s code editing leaderboard with Sonnet and GPT-3.5 included for scale.
77% claude-3.5-sonnet
73% DeepSeek Coder V2 0724
66% llama-3.1-405b-instruct
60% Mistral Large 2 (2407)
59% llama-3.1-70b-instruct
58% gpt-3.5-turbo-0301
38% llama-3.1-8b-instruct
- WiSaGaN 2y agoExciting! So many models dropped on the 24th. Really to know whether the numerical suffix means YYMM or MMDD!
- tarasglek 2y agoBeen experimenting with aider for a while. Really fun using it for basically free with deepseek. Thanks for getting that to work. My question is re your whole file fallback approach. Since you already have everything mapped out with treesitter, have you considered doing function at a time instead? It's just as easy for a model to ask for a function as it is for a file