4 ms·
GitHub Named a Leader in the Gartner First Magic Quadrant for AI Code Assistants
- westurner 2y agoGartner "Magic Quadrant for AI Code Assistants" (2024) https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819&st=sb https://www.gartner.com/doc/reprints?id=1-2IKO4MPE&ct=240819... Additional criteria for assessing AI code assistants from https://news.ycombinator.com/item?id=40478539 https://news.ycombinator.com/item?id=40478539 re: Text-to-SQL bemchmarks : codefuse-ai/Awesome-Code-LLM > Analysis of AI-Generated Code, Benchmarks: https://github.com/codefuse-ai/Awesome-Code-LLM https://github.com/codefuse-ai/Awesome-Code-LLM : > 8.2. Benchmarks: * Integrated Benchmarks, Program Synthesis, Visually Grounded Program Synthesis, Code Reasoning and QA, Text-to-SQL, Code Translation, Program Repair, Code Summarization, Defect/Vulnerability Detection, Code Retrieval, Type Inference, Commit Message Generation, Repo-Level Coding* OT did not assess: Aider: https://github.com/paul-gauthier/aider https://github.com/paul-gauthier/aider : > Aider works best with GPT-4o & Claude 3.5 Sonnet and can connect to almost any LLM. https://aider.chat/ https://aider.chat/ : > Aider has one of the top scores on SWE Bench. SWE Bench is a challenging software engineering benchmark where aider solved real GitHub issues from popular open source projects like django, scikitlearn, matplotlib, etc SWE Bench benchmark: https://www.swebench.com/ https://www.swebench.com/