3 ms·
Gartner "Magic Quadrant for AI Code Assistants" (2024) https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819&st=sb https://www.gartner.com/doc/reprints?i
by westurner 2y ago
Gartner "Magic Quadrant for AI Code Assistants" (2024)
https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819&st=sb https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819...
Additional criteria for assessing AI code assistants from https://news.ycombinator.com/item?id=40478539 https://news.ycombinator.com/item?id=40478539 re: Text-to-SQL bemchmarks :
codefuse-ai/Awesome-Code-LLM > Analysis of AI-Generated Code, Benchmarks:
https://github.com/codefuse-ai/Awesome-Code-LLM https://github.com/codefuse-ai/Awesome-Code-LLM :
> 8.2. Benchmarks: *
Integrated Benchmarks,
Program Synthesis,
Visually Grounded Program Synthesis,
Code Reasoning and QA,
Text-to-SQL,
Code Translation,
Program Repair,
Code Summarization,
Defect/Vulnerability Detection,
Code Retrieval,
Type Inference,
Commit Message Generation,
Repo-Level Coding*
OT did not assess:
Aider:
https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider :
> Aider works best with GPT-4o & Claude 3.5 Sonnet and can connect to almost any LLM.
https://aider.chat/ https://aider.chat/ :
> Aider has one of the top scores on SWE Bench. SWE Bench is a challenging software engineering benchmark where aider solved real GitHub issues from popular open source projects like django, scikitlearn, matplotlib, etc
SWE Bench benchmark:
https://www.swebench.com/ https://www.swebench.com/