3 ms·
Related: Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current
by LLMCodeAuditor 6mo ago
Related:
Unfortunately, given participant feedback and surveys, we believe that the data from our new experiment gives us an unreliable signal of the current productivity effect of AI tools. The primary reason is that we have observed a significant increase in developers choosing not to participate in the study because they do not wish to work without AI, which likely biases downwards our estimate of AI-assisted speedup.
(https://metr.org/blog/2026-02-24-uplift-update/ https://metr.org/blog/2026-02-24-uplift-update/)
This was a huge red flag! Within a year a large majority of devs became so whiny and lazy that METR couldn't fill the "no AI" bucket for their study - it's not like this was a full-time job, just a quick gig, and it was still too much effort for their poor LLM-addled brains. At the time I thought it was a terrible psychological omen.
I am so glad I don't use this stuff.
- econ 6mo agoThe next study will be just to ask and measure how fast they run away.