2 ms·

Aider Is SOTA for Both SWE Bench and SWE Bench Lite

• anotherpaulg 2y ago
  Aider scored 18.9% on the main SWE Bench benchmark, achieving a state-of-the-art result. The current top leaderboard entry is 13.8% from Amazon Q Developer Agent. The best result reported elsewhere seems to be 13.9% from Devin. This result on the main SWE Bench builds on aider’s recent SOTA result on the easier SWE Bench Lite.