3 ms·
This predates the o1 release, but the folks behind Devin did do some early evaluation of o1 vs 4o vs Devin back in September: https://x.com/cognition_labs/stat
by cbhl 2y ago
This predates the o1 release, but the folks behind Devin did do some early evaluation of o1 vs 4o vs Devin back in September:
https://x.com/cognition_labs/status/1834292718174077014 https://x.com/cognition_labs/status/1834292718174077014
I'd expect a very different experience with Devin vs the IDE-forks -- it provides status updates in Slack, runs CI, and when it's done it puts up a pull request in GitHub.
- Yusefmosiah 2y agoThanks, but that comparison is for old models, a different, non-shipped version of Devin called “Devin-base”, and doesn’t include Claude. Slack integration, automatically pushing to CI, etc., are relatively low-value compared to the questions of “does it write better code than alternatives?”, “can I depend on it to solve hard problems?”, “will I still need a Cursor and/or ChatGPT Pro subscription to debug Devin’s mistakes?”