3 ms·

I can see an argument that the productivity gains are illusory / don’t translate to economic productivity. I’m not denying the possibility. However, most of th

I can see an argument that the productivity gains are illusory / don’t translate to economic productivity. I’m not denying the possibility. However, most of the engineers I respect have gone from being skeptics a year ago to convinced today. I don’t personally know any true holdouts any more. If there are studies that disprove productivity gains more than six months ago, I’m happy to believe that it was true of the AIs that were available at the time. But I’m going to need something much more recent before I disbelieve my lyin’ eyes where it pertains to the AIs available today.

oudlys 4mo ago
There is an observational study that was published in March 2026 that followed 4000 teams over 2 years. It shows, in my view, exactly that the productivity gains don't translate into economic value. Here is the report: https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways And my commentary: https://unessays.substack.com/p/talk-is-cheap https://unessays.substack.com/p/talk-is-cheap
- asdfasgasdgasdg 4mo ago
  If it was published in March 2026, even if the data was collected up to the day the study was published, 7/8ths of it would fail my “within the last six months” test. But I am looking forward to the results of future studies on this topic!
  - oudlys 4mo ago
    I get wanting to wait for more data. And thinking that LLMs have improved enough that this will change. My view is that it's not really about how good the models are - it's about how we're using them. Understanding what you've built is an important part of value creation, and LLMs eliminate that.
dminik 4mo ago
Its funny, I've noticed the same thing, but did not come to the same conclusion. I currently don't have work access to Claude Code, but most of my teammates do. Watching from the outside, the cycle seems to look like this: 1. Experience some success, which hooks you into relying on AI. 2. The AI keeps failing at some task, but you don't want to stop. Keep trying over and over again. 3. Run out of tokens and take a break. Now, sometimes 1 doesn't happen. Sometimes 2 doesn't happen. 3 is a certainty though. Now, if you told me that the productivity gain from 1 is enough to offset the loss from 2 and 3, I could believe you. But I also wouldn't be surprised if it didn't.
- chillacy 4mo ago
  As I work with Claude more and gain a feel for its capabilities, I tend to run into 2 far less often, as I'll decompose my messages more for the current model limitations. The threshold also changes each release.
techblueberry 4mo ago
I’m going back to being a holdout, but it’s nuanced - My theory into why LLMs don’t lead to the colloquial definition of productivity would be something like - if code was never the bottleneck than generating code faster doesn’t result in more meaningful output. Even if you take for granted that AI is as good as the best people say in writing code. And Ive spent a lot of time generating codes, I won’t disagree - Then the question becomes - does this change your daily incentives such that you reach for code as the solution to your problems rather than something else (coordinating with your colleagues? Product management? Planning and Design? So from a holistic perspective, I think intentionally limiting your own AI usage is the best approach for maximum long-term productivity.
- asdfasgasdgasdg 4mo ago
  I’m not completely closed to your idea but if code was never the bottleneck why did so many organizations always feel so chronically low on coders? And of course this requires the AI to be no help at all with what is actually the bottleneck.
  - techblueberry 4mo ago
    So, one is, it probably does depend on workflow. Some folks probably are doing things that can be accelerated by AI, and I think if you’re a small team with a good product head and know what you’re doing then AI probably helps a lot. But what if the problem you’re trying to solve is the altogether too often problem of like getting teams that are dependent on you to upgrade the library they use. And what if the library is a breaking change, and last year they upgraded to the library on your advice and it broke production and now they’re suss and want to accept all changes, and integrating that library change isn’t in their critical path so they’re just not going to spend time on it, even if you submit the MR them. Even if you show them their tests pass after the change. Importantly to the above, you probably need more devs to do more of the above in parallel. You don’t hire devs to write more code, you hire more devs to carry on the mental load of a broader scope of work. Even in the before times, so much code got stuck at the integration step. But because all that is hard, instead you go and codegen to fix an obscure bug that sure makes a few customers happy, but no one thought was a limiting factor for paying your company more money. It’s not that I don’t think AI can help, I think it’s a prerequisite for the job and everyone should use it. It’s more that I think in the grand scheme of things, people will bias towards using it for tasks that aren’t in the critical path - refactors, tech debt, bug smashing, tool building; and I think it could really help devex and that’s good. But I think people are bad at knowing the difference between “my job feels a bit easier” or “I’m more productive” and “this task had an impact on the bottom line” and when you extrapolate that out to a whole engineering org, that’s where the productivity statistics get lost. I’ll addd one data point to this is like this thread itself. So many people on AI skepticism threads point to their own subjective experience as evidence we’re not in a bubble, and sort of ignore the entire concept of economics. I’m not saying we’re in 100% in a bubble, but subjective experience isn’t great evidence of it. And this is just sort of one of the factors, what about the increased cost and mental load of supporting more software? What about junior engineers who feel pressured to ship work but don’t actually learn the software engineering? What about lost context from not intimately understanding your software?

oudlys 4mo ago

There is an observational study that was published in March 2026 that followed 4000 teams over 2 years. It shows, in my view, exactly that the productivity gains don't translate into economic value. Here is the report: https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways https://www.faros.ai/blog/ai-acceleration-whiplash-takeaways And my commentary: https://unessays.substack.com/p/talk-is-cheap https://unessays.substack.com/p/talk-is-cheap

asdfasgasdgasdg 4mo ago
If it was published in March 2026, even if the data was collected up to the day the study was published, 7/8ths of it would fail my “within the last six months” test. But I am looking forward to the results of future studies on this topic!
- oudlys 4mo ago
  I get wanting to wait for more data. And thinking that LLMs have improved enough that this will change. My view is that it's not really about how good the models are - it's about how we're using them. Understanding what you've built is an important part of value creation, and LLMs eliminate that.

dminik 4mo ago

Its funny, I've noticed the same thing, but did not come to the same conclusion. I currently don't have work access to Claude Code, but most of my teammates do. Watching from the outside, the cycle seems to look like this: 1. Experience some success, which hooks you into relying on AI. 2. The AI keeps failing at some task, but you don't want to stop. Keep trying over and over again. 3. Run out of tokens and take a break. Now, sometimes 1 doesn't happen. Sometimes 2 doesn't happen. 3 is a certainty though. Now, if you told me that the productivity gain from 1 is enough to offset the loss from 2 and 3, I could believe you. But I also wouldn't be surprised if it didn't.

chillacy 4mo ago
As I work with Claude more and gain a feel for its capabilities, I tend to run into 2 far less often, as I'll decompose my messages more for the current model limitations. The threshold also changes each release.

techblueberry 4mo ago

I’m going back to being a holdout, but it’s nuanced - My theory into why LLMs don’t lead to the colloquial definition of productivity would be something like - if code was never the bottleneck than generating code faster doesn’t result in more meaningful output. Even if you take for granted that AI is as good as the best people say in writing code. And Ive spent a lot of time generating codes, I won’t disagree - Then the question becomes - does this change your daily incentives such that you reach for code as the solution to your problems rather than something else (coordinating with your colleagues? Product management? Planning and Design? So from a holistic perspective, I think intentionally limiting your own AI usage is the best approach for maximum long-term productivity.

asdfasgasdgasdg 4mo ago
I’m not completely closed to your idea but if code was never the bottleneck why did so many organizations always feel so chronically low on coders? And of course this requires the AI to be no help at all with what is actually the bottleneck.
- techblueberry 4mo ago
  So, one is, it probably does depend on workflow. Some folks probably are doing things that can be accelerated by AI, and I think if you’re a small team with a good product head and know what you’re doing then AI probably helps a lot. But what if the problem you’re trying to solve is the altogether too often problem of like getting teams that are dependent on you to upgrade the library they use. And what if the library is a breaking change, and last year they upgraded to the library on your advice and it broke production and now they’re suss and want to accept all changes, and integrating that library change isn’t in their critical path so they’re just not going to spend time on it, even if you submit the MR them. Even if you show them their tests pass after the change. Importantly to the above, you probably need more devs to do more of the above in parallel. You don’t hire devs to write more code, you hire more devs to carry on the mental load of a broader scope of work. Even in the before times, so much code got stuck at the integration step. But because all that is hard, instead you go and codegen to fix an obscure bug that sure makes a few customers happy, but no one thought was a limiting factor for paying your company more money. It’s not that I don’t think AI can help, I think it’s a prerequisite for the job and everyone should use it. It’s more that I think in the grand scheme of things, people will bias towards using it for tasks that aren’t in the critical path - refactors, tech debt, bug smashing, tool building; and I think it could really help devex and that’s good. But I think people are bad at knowing the difference between “my job feels a bit easier” or “I’m more productive” and “this task had an impact on the bottom line” and when you extrapolate that out to a whole engineering org, that’s where the productivity statistics get lost. I’ll addd one data point to this is like this thread itself. So many people on AI skepticism threads point to their own subjective experience as evidence we’re not in a bubble, and sort of ignore the entire concept of economics. I’m not saying we’re in 100% in a bubble, but subjective experience isn’t great evidence of it. And this is just sort of one of the factors, what about the increased cost and mental load of supporting more software? What about junior engineers who feel pressured to ship work but don’t actually learn the software engineering? What about lost context from not intimately understanding your software?