4 ms·
You can see my earlier comment somewhere here, but I feel obliged to remind everyone that the claim that Github Copilot made developers "55% more productive" ca
by Bjorkbat 2y ago
You can see my earlier comment somewhere here, but I feel obliged to remind everyone that the claim that Github Copilot made developers "55% more productive" came from a study where they asked 100 developers to implement an HTTP server in Javascript, split the group up roughly 50/50, and gave one group Github Copilot. The Copilot group did it in an hour and 11 minutes, whereas the control group got it done 2 hours and 41 minutes.
https://github.blog/2022-09-07-research-quantifying-github-copilots-impact-on-developer-productivity-and-happiness/?ocid=AIDcmmc3fhtaow_SEM__k_Cj0KCQjwir2xBhC_ARIsAMTXk85qF--5Z_RO3fCcRnFE6itQ3cKFQvIGLnsyaF-Gwd4FdZFoisM7GtUaAou4EALw_wcB_k_#finding-2-but-speed-is-important-too https://github.blog/2022-09-07-research-quantifying-github-c...
That's where the 55% number is coming from. It's coming from this experiment, and only this experiment.
So yeah, if you're wondering why you aren't somehow 50% more productive when using Github Copilot, it's probably because you're not implementing a simple task that's been done to death in countless tutorials.
- mucle6 2y agoDo you not have a good experience with github copilot? I recently started using it and it will write functions that are 90% right based on comments, and I just modify what I want. I've felt at least a 2x speedup, maybe even 4x. That said I'm working on a new project where I'm writing a lot of code. For making small changes I could see how its much less valuable.
- chasd00 2y ago"I just modify what I want." i think that's key, you use the llm to get something started and then change/fix/enhance to get what you want. That works for me too but the folks that want to prompt an llm from nothing to a finished application are in for a rough ride.