2 ms·
> Surveys confirm that for many workers, AI tools like ChatGPT reduce their productivity I'm pretty suspicious of that survey (the one that always gets cited a
by simonw 2y ago
> Surveys confirm that for many workers, AI tools like ChatGPT reduce their productivity
I'm pretty suspicious of that survey (the one that always gets cited as proof that Copilot makes developers less productive, which then inevitably gets used to argue that all generative AI makes developers left productive): https://resources.uplevelteam.com/gen-ai-for-coding https://resources.uplevelteam.com/gen-ai-for-coding
If I was running a company like https://uplevelteam.com/ https://uplevelteam.com/ that sells developer productivity metrics software, one of the smartest marketing moves I could make would be to put out a report that makes a bold, contrarian claim about a hot topic like AI coding productivity. Guaranteed to get a ton of press coverage from that.
Is the survey itself any good? I filled in the "request a copy" form and it's a two page infographic! Precious few confirmable details on how they actually ran it: https://static.simonwillison.net/static/2025/uplevel-genai-productivity.pdf https://static.simonwillison.net/static/2025/uplevel-genai-p...
Here's what they say about their methodology:
> Metrics were evaluated prior to implementation of Copilot from January 9 through April 9, 2023 versus after implementation from January 8 through April 7, 2024. This time period was selected to remove the effects of seasonality.
> Data on Copilot access was provided to Uplevel Data Labs across several enterprise engineering customers for a total of 351 developers in the TEST group (with Copilot access) and 434 in the CONTROL group (without Copilot access). The developers in the CONTROL group were similar to those in the TEST group in terms of role, working days, and PR volume in each period.
- strict9 2y agoI said it in a confusing way, but I do believe it increases productivity, at least for me. But it's hazy and hard to measure. I am very rarely stuck on hard problems like I was 3+ years ago. But I lose time in other ways. I've never measured it so it's really just a feeling. I am also skeptical of a company selling something to sponsor or put out a report directly related to what they're selling.
- pcthrowaway 2y agoIf I add in all the time I spend reading about AI now, or wading through AI slop while researching something, any productivity gains I may see from actually using AI effectively are more than cancelled out. Interestingly, AI tooling existing in its current form may be making people collectively less productive, even if individually it might make one somewhat more productive at very specific tasks.
- rightbyte 2y ago> I am very rarely stuck on hard problems like I was 3+ years ago. I am stuck on easy problems. Either bugs that are obvious afterwards or cooperation problems. Almost never hard problems.