2 ms·

I'm a SWE who's been using coding agents daily for the last 6 months and I'm still skeptical. For my team at least, the productivity boost is difficult to quan

I'm a SWE who's been using coding agents daily for the last 6 months and I'm still skeptical. For my team at least, the productivity boost is difficult to quantify objectively. Our products and services have still tons of issues that AI isn't going to solve magically. It's pretty clear that AI is allowing to move faster for some tasks, but it's also detrimental for other things. We're going to learn how to use these tools more efficiently, but right now, I'm not convinced about the productivity gain.

david_shaw 8mo ago
> I'm a SWE who's been using coding agents daily for the last 6 months and I'm still skeptical. What improvements have you noticed over that time? It seems like the models coming out in the last several weeks are dramatically superior to those mid-last year. Does that match your experience?
- nrclark 8mo ago
  Not the grandparent, but I've used most of the OpenAI models that have been released in the last year. Out of all of them, o3 was the best at the programming tasks I do. I liked it a lot more than I like GPT 5.2 Thinking/Pro. Overall, I'm not at all convinced that models are making forward progress in general.
- yodsanklai 8mo ago
  Yes, it matches my experience. Now I can throw tasks at the agent and have it write a full PR, with tests, good summary. Or it can review things and make good suggestions that a casual reviewer or non-expert would have missed. It can also take a bunch of logs as input, find the issue, fix the code. I can't deny it's impressive and useful. What I'm still skeptical about is how much more productive it makes us. In my case, coding is maybe 50% of my job, and I work on complex and novel systems. The agent gives me the illusion I don't need to think anymore, but it's not the case. Agents slow me down in many cases too, I'm not learning and improving as I used to.
techpression 8mo ago
In a team of one at work I see clear benefits, but having worked in many different team sizes for most of my career I can see how it quickly would go down, especially if you care about quality. And even with the latest models it’s a constant battle against legacy training data, which has gotten worse over time. ”I have to spend 45 minutes explaining why a one minute AI generated PR is bad code” was how an old colleague summarized it.
SoftTalker 8mo ago
Is your backlog and/or your velocity increasing, decreasing, or the same? That's really the ultimate question.

david_shaw 8mo ago

> I'm a SWE who's been using coding agents daily for the last 6 months and I'm still skeptical. What improvements have you noticed over that time? It seems like the models coming out in the last several weeks are dramatically superior to those mid-last year. Does that match your experience?

nrclark 8mo ago
Not the grandparent, but I've used most of the OpenAI models that have been released in the last year. Out of all of them, o3 was the best at the programming tasks I do. I liked it a lot more than I like GPT 5.2 Thinking/Pro. Overall, I'm not at all convinced that models are making forward progress in general.
yodsanklai 8mo ago
Yes, it matches my experience. Now I can throw tasks at the agent and have it write a full PR, with tests, good summary. Or it can review things and make good suggestions that a casual reviewer or non-expert would have missed. It can also take a bunch of logs as input, find the issue, fix the code. I can't deny it's impressive and useful. What I'm still skeptical about is how much more productive it makes us. In my case, coding is maybe 50% of my job, and I work on complex and novel systems. The agent gives me the illusion I don't need to think anymore, but it's not the case. Agents slow me down in many cases too, I'm not learning and improving as I used to.

techpression 8mo ago

In a team of one at work I see clear benefits, but having worked in many different team sizes for most of my career I can see how it quickly would go down, especially if you care about quality. And even with the latest models it’s a constant battle against legacy training data, which has gotten worse over time. ”I have to spend 45 minutes explaining why a one minute AI generated PR is bad code” was how an old colleague summarized it.

SoftTalker 8mo ago

Is your backlog and/or your velocity increasing, decreasing, or the same? That's really the ultimate question.