4 ms·
I was actually about to make a comment on that. They got that number from some dumb "experiment" they did where told two groups of developers to implement an H
by Bjorkbat 2y ago
I was actually about to make a comment on that. They got that number from some dumb "experiment" they did where told two groups of developers to implement an HTTP server in Javascript. The group with Copilot got the job done 55% faster (https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/?ocid=AIDcmmc3fhtaow_SEM__k_Cj0KCQjwir2xBhC_ARIsAMTXk85qF--5Z_RO3fCcRnFE6itQ3cKFQvIGLnsyaF-Gwd4FdZFoisM7GtUaAou4EALw_wcB_k_#finding-2-but-speed-is-important-too https://github.blog/2022-09-07-research-quantifying-github-c...)
So yeah, they did an experiment with <100 developers asking them to create implement something that only took the control group 3 hours to finish from scratch, and from this we got the "55% more productive" statistic.
- chasd00 2y agoThat's really disappointing and a little bit scary. That seems like a task tailor fit for an LLM, it's been done a million times before, follows an established protocol, and the implementations are all basically the same. Granted a lot of code is like that but a more realistic task may have been something like "create an online ordering system for a cake company that sells different kinds of cakes, has a sale every so often, and uses coupons printed in the local paper". That task is more ambiguous and better reflects what software devs are tasked with day to day IMO.
- deleted 2y ago[deleted]