5 ms·

This article bases its argument on the predicate that AI _at worst_ will increase developer productivity be 0-10%. But several studies have found that not to be

This article bases its argument on the predicate that AI _at worst_ will increase developer productivity be 0-10%. But several studies have found that not to be true at all. AI can, and does, make some people less effective

dhorthy 1y ago
definitely - the standford video has a slide about how many cases caused people to be even slower than without AI
- mcny 1y ago
  Question for discussion - what steps can I take as a human to set myself up for success where success is defined by AI made me faster, more efficient etc?
  - 0xblacklight 1y ago
    In many cases (though not all) it's the same thing that makes for great engineering managers: smart generalists with a lot of depth in maybe a couple of things (so they have an appreciation for depth and complexity) but a lot of breadth so they can effectively manage other specialists, and having great technical communication skills - be able to communicate what you want done and how without over-specifying every detail, or under-specifying tasks in important ways.
  - Planktonne 1y ago
    >where success is defined by AI made me faster, more efficient etc? I think this attitude is part of the problem to me; you're not aiming to be faster or more efficient (and using AI to get there), you're aiming to use AI (to be faster and more efficient). A sincere approach to improvement wouldn't insist on a tool first.
- keeda 1y ago
  According to the Stanford video the only cases (statistically speaking) where that happened was high-complexity tasks for legacy / low popularity languages, no? I would imagine that is a small minority of projects. Indeed, the video cites the overall productivity boost at 15 - 20% IIRC.
CharlesW 1y ago
Both (1) "AI can, and does, make some people less effective" and (2) "the average productivity boost (~20%) is significant" (per Stanford's analysis) can be true. The article at the link is about how to use AI effectively in complex codebases. It emphasizes that the techniques described are "not magic", and makes very reasonable claims.
- dingnuts 1y ago
  the techniques described sound like just as much work, if not more, than just writing the code. the claimed output isn't even that great, it's comparable to the speed you would expect a skilled engineer to move at in a startup environment
  - CharlesW 1y ago
    > the techniques described sound like just as much work, if not more, than just writing the code. That's very fair, and I believe that's true for you and for many experienced software developers who are more productive than the average developer. For me, AI-assisted coding is a significant net win.
    - dhorthy 1y ago
      I tend to think about it like vim - you will feel slow and annoyed for the first few weeks, but investing in these skills are massive +EV long term
      - criemen 1y ago
        Yet a lot of people never bother to learn vim, and are still outstanding and productive engineers. We're surely not seeing any memos "Reflexive vim usage is now a baseline expectation at [our company]" (context: https://x.com/tobi/status/1909251946235437514 https://x.com/tobi/status/1909251946235437514) The as-of-yet unanswered question is: Is this the same? Or will non-LLM-using engineers be left behind?
f59b3743 1y ago
[flagged]
- dingnuts 1y ago
  I have
  - dhorthy 1y ago
    me too :)
simonw 1y ago
"AI can, and does, make some people less effective" So those people should either stop using it or learn to use it productively. We're not doomed to live in a world where programmers start using AI, lose productivity because of it and then stay in that less productive state.
- bgwalter 1y ago
  If managers are convinced by stakeholders who relentlessly put out pro-"AI" blog posts, then a subset of programmers can be forced to at least pretend to use "AI". They can be forced to write in their performance evaluation how much (not if, because they would be fired) "AI" has improved their productivity.
telliott1984 1y ago
There's also the more insidious gap between perceived productivity and actual productivity. Doesn't help that nobody can agree on how to measure productivity even without AI.

dhorthy 1y ago

definitely - the standford video has a slide about how many cases caused people to be even slower than without AI

mcny 1y ago
Question for discussion - what steps can I take as a human to set myself up for success where success is defined by AI made me faster, more efficient etc?
- 0xblacklight 1y ago
  In many cases (though not all) it's the same thing that makes for great engineering managers: smart generalists with a lot of depth in maybe a couple of things (so they have an appreciation for depth and complexity) but a lot of breadth so they can effectively manage other specialists, and having great technical communication skills - be able to communicate what you want done and how without over-specifying every detail, or under-specifying tasks in important ways.
- Planktonne 1y ago
  >where success is defined by AI made me faster, more efficient etc? I think this attitude is part of the problem to me; you're not aiming to be faster or more efficient (and using AI to get there), you're aiming to use AI (to be faster and more efficient). A sincere approach to improvement wouldn't insist on a tool first.
keeda 1y ago
According to the Stanford video the only cases (statistically speaking) where that happened was high-complexity tasks for legacy / low popularity languages, no? I would imagine that is a small minority of projects. Indeed, the video cites the overall productivity boost at 15 - 20% IIRC.

CharlesW 1y ago

Both (1) "AI can, and does, make some people less effective" and (2) "the average productivity boost (~20%) is significant" (per Stanford's analysis) can be true. The article at the link is about how to use AI effectively in complex codebases. It emphasizes that the techniques described are "not magic", and makes very reasonable claims.

dingnuts 1y ago
the techniques described sound like just as much work, if not more, than just writing the code. the claimed output isn't even that great, it's comparable to the speed you would expect a skilled engineer to move at in a startup environment
- CharlesW 1y ago
  > the techniques described sound like just as much work, if not more, than just writing the code. That's very fair, and I believe that's true for you and for many experienced software developers who are more productive than the average developer. For me, AI-assisted coding is a significant net win.
  - dhorthy 1y ago
    I tend to think about it like vim - you will feel slow and annoyed for the first few weeks, but investing in these skills are massive +EV long term
    - criemen 1y ago
      Yet a lot of people never bother to learn vim, and are still outstanding and productive engineers. We're surely not seeing any memos "Reflexive vim usage is now a baseline expectation at [our company]" (context: https://x.com/tobi/status/1909251946235437514 https://x.com/tobi/status/1909251946235437514) The as-of-yet unanswered question is: Is this the same? Or will non-LLM-using engineers be left behind?

f59b3743 1y ago

[flagged]

dingnuts 1y ago
I have
- dhorthy 1y ago
  me too :)

simonw 1y ago

"AI can, and does, make some people less effective" So those people should either stop using it or learn to use it productively. We're not doomed to live in a world where programmers start using AI, lose productivity because of it and then stay in that less productive state.

bgwalter 1y ago
If managers are convinced by stakeholders who relentlessly put out pro-"AI" blog posts, then a subset of programmers can be forced to at least pretend to use "AI". They can be forced to write in their performance evaluation how much (not if, because they would be fired) "AI" has improved their productivity.

telliott1984 1y ago

There's also the more insidious gap between perceived productivity and actual productivity. Doesn't help that nobody can agree on how to measure productivity even without AI.