3 ms·
They've posted their own run of the Aider benchmark [1] if you want to compare, it achieved 57.1%. [1]: https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5
by Deathmax 2y ago
They've posted their own run of the Aider benchmark [1] if you want to compare, it achieved 57.1%.
[1]: https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5/Qwen2.5-Coder/qwen2.5-coder-instruct.jpg https://qianwen-res.oss-cn-beijing.aliyuncs.com/Qwen2.5/Qwen...
- reissbaker 2y agoOof. I'm really not sure why companies keep releasing these mini coding models; 57.1% is worse than gpt-3.5-turbo, and running it locally will be slower than OpenAI's API. I guess you could use it if you took your laptop into the woods, but with such poor coding ability, would you even want to? The Qwen2.5-72B model seems to do pretty well on coding benchmarks, though — although no word about Aider yet.