4 ms·

This reminds me of themes I recently saw in [Harness Engineering is not Enough: Why Software Factories Fail](https://www.youtube.com/watch?v=Ib5GBkD555M https:/

by gashad 2mo ago

This reminds me of themes I recently saw in [Harness Engineering is not Enough: Why Software Factories Fail](https://www.youtube.com/watch?v=Ib5GBkD555M https://www.youtube.com/watch?v=Ib5GBkD555M) (Warning: the last 3 slides seem like an advertisement). One thing I liked is how Dex has a little graphic he glossed over showing software development is - 25% planning & aligning with other teams - 25% coding - 25% testing/verifying - 25% code review/rework One argument was that agentic coding speeds up that coding part a bunch. So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do.

whateveracct 2mo ago
> So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do. Amdahl's Law should be familiar to anyone with a 4y computer science/engineering degree. Why aren't they applying it to their own throughput?
- t-writescode 2mo ago
  Well, there’s probably several reasons: 1) because they’re not doing those others spheres of work 2) because they don’t think about the work nor how tired they are afterwards 3) they’re proselytizing AI work as the future and that conflicts with that vision 4) the other shoe hasn’t dropped 5) they really don’t see it 6) some people genuinely hate programming and this helps them skip that. I’m sure others as well.
- ChadNauseam 2mo ago
  Let's apply it: - 25% planning & aligning with other teams - 25% coding - 25% testing/verifying - 25% code review/rework I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary). So that leaves us with: - 92.5% planning & aligning with other teams - 2.5% coding - 2.5% testing/verifying - 2.5% code review/rework Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.
steveBK123 2mo ago
The other hard to measure part is that I see/hear a lot of LLM usage going towards dev work that was never prioritized before. Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM. Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base. Plenty of other examples of this.
- leoedin 2mo ago
  The quality of my bash scripts is 100x what they used to be. Proper help messages, flag parsing, functions, internal variables, using sed all the time (I could never wrap my head around that). Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions. Is my life any better? Dunno. But it’s satisfying (until there’s a bug)
  - nasretdinov 2mo ago
    > the main issue is that they’re completely undebbugable now Well, that means the quality actually dropped then :). Looking impressive isn't equal to quality, understandability and reliability is
    - gmassman 2mo ago
      Not necessarily…. As with all engineering choices there are tradeoffs involved. There can be both advantages and disadvantages to using AI to build scripts. And more likely than not you’ll need AI to help debug the scripts when they have issues. Real tradeoffs to consider.
    - xboxnolifes 2mo ago
      100x UX quality, not DX quality.
  - mckee_plus_plus 2mo ago
    I switched to js for dev related scripting after seeing how much python scripting is used for portable c++ projects like chromium. A node glob() or a regexp string.replace call is probably easier to read than spaghetti shell. And your llm might do a much better job of creating clean, readable and testable code.
coffeebeqn 2mo ago
I use CC to do the testing and verifying in multiple deployment environments. It’s faster at things like coming up a plausible test payload or ten, querying a bunch of metrics from datadog, querying and verifying state from a db before and after testing, scanning all the deployed logs for relevant errors. For me that has been more of an uplift than the coding. I still feel like communication it’s worst at. So writing tickets, getting a decision out of a group, documenting only the important things, etc. it’s still not that good at
altern8 2mo ago
AI is great because my job is a lot easier, but it produces some extremely crappy code that would've NEVER passed a code review 1 year ago. You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand. I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code. Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.
- serial_dev 2mo ago
  > it produces some extremely crappy code that would've NEVER passed a code review 1 year ago The 10x speed up comes from the fact that we have all given up on properly reviewing each others code and we just say “meh, it will be fine, lgtm”
  - QwenGlazer9000 2mo ago
    This. The bar has never been lower.
  - altern8 2mo ago
    Yes, exactly my point. BUT, to me if we hadn't lowered the bar MASSIVELY, it would NOT be fine. I've seen people at my work that will push or even deploy features that don't do what their main purpose is and they just dismiss the issue saying it's a bug. As an example, a button that's supposed to open a modal window if you click on it. If you go test it and the modal doesn't open, they'll say "oh, there's a bug". You developed or even deployed something that is wrong and completely useless. Just one year ago, this would be unheard of and you would've gotten a really hard time for it. Now it's just a bug and part of how we're doing things. Like I said, I just do what I'm told at work but to me this is just wild. There's no way it won't come back to bite us.
  - dools 2mo ago
    Code review has always been pointless
agumonkey 2mo ago
Agentic tools changed the workflow in our org, we were very tense before with a simple yet careful team process.. Since last winter we're now pushing a lot more but the teamwork (which was brittle before) is now mostly gone, everybody can roll on its own, but the review process didn't scale and now monthly deliveries are full of "seems to work". nobody checks properly, nobody reviews, nobody tighten any bolts.. and nobody cares much anyway, the system allows it.
andrekandre 2mo ago
> - 25% planning & aligning with other teams > - 25% coding > - 25% testing/verifying > - 25% code review/rework legit question: has anyone got ai to do all the above? i've heard people trying ai-dlc [0] but it sounds like a slog... [0] https://aws.amazon.com/jp/blogs/devops/ai-driven-development-life-cycle/ https://aws.amazon.com/jp/blogs/devops/ai-driven-development...

by gashad 2mo ago

whateveracct 2mo ago

> So maybe there's 2x speedup in coding. But that's only a small speedup in the totality of everything software engineers do. Amdahl's Law should be familiar to anyone with a 4y computer science/engineering degree. Why aren't they applying it to their own throughput?

t-writescode 2mo ago
Well, there’s probably several reasons: 1) because they’re not doing those others spheres of work 2) because they don’t think about the work nor how tired they are afterwards 3) they’re proselytizing AI work as the future and that conflicts with that vision 4) the other shoe hasn’t dropped 5) they really don’t see it 6) some people genuinely hate programming and this helps them skip that. I’m sure others as well.
ChadNauseam 2mo ago
Let's apply it: - 25% planning & aligning with other teams - 25% coding - 25% testing/verifying - 25% code review/rework I'd say that thanks to LLM assistance I'm 10x faster at coding, code review/rework, and testing/verifying. (LLMs can partially automate testing/verifying too, and the code is higher quality now as well so less testing/verifying is necessary). So that leaves us with: - 92.5% planning & aligning with other teams - 2.5% coding - 2.5% testing/verifying - 2.5% code review/rework Obviously, that makes zero sense as a split. If you saw any organization doing that, you'd suggest having fewer teams, more silos, etc. Maybe you have designers produce code, instead of showing the designs to coders and having the coders implement it. Maybe you force all your engineers to dogfood the product that way they can identify issues themselves rather than needing QA teams to do it. And so on. so the last category, "planning & aligning with other teams", falls too.

steveBK123 2mo ago

The other hard to measure part is that I see/hear a lot of LLM usage going towards dev work that was never prioritized before. Suddenly devs who were cranking out features with no interest in infrastructure are attacking giant refactors to make the code more understandable to the LLM. Other devs are using LLMs to build themselves quality of life SDLC tools completely separate from the core code base. Plenty of other examples of this.

leoedin 2mo ago
The quality of my bash scripts is 100x what they used to be. Proper help messages, flag parsing, functions, internal variables, using sed all the time (I could never wrap my head around that). Of course, the main issue is that they’re completely undebbugable now. My bash scripts used to be a sequential list of commands, now they’re 500 lines of variable laden functions. Is my life any better? Dunno. But it’s satisfying (until there’s a bug)
- nasretdinov 2mo ago
  > the main issue is that they’re completely undebbugable now Well, that means the quality actually dropped then :). Looking impressive isn't equal to quality, understandability and reliability is
  - gmassman 2mo ago
    Not necessarily…. As with all engineering choices there are tradeoffs involved. There can be both advantages and disadvantages to using AI to build scripts. And more likely than not you’ll need AI to help debug the scripts when they have issues. Real tradeoffs to consider.
  - xboxnolifes 2mo ago
    100x UX quality, not DX quality.
- mckee_plus_plus 2mo ago
  I switched to js for dev related scripting after seeing how much python scripting is used for portable c++ projects like chromium. A node glob() or a regexp string.replace call is probably easier to read than spaghetti shell. And your llm might do a much better job of creating clean, readable and testable code.

coffeebeqn 2mo ago

I use CC to do the testing and verifying in multiple deployment environments. It’s faster at things like coming up a plausible test payload or ten, querying a bunch of metrics from datadog, querying and verifying state from a db before and after testing, scanning all the deployed logs for relevant errors. For me that has been more of an uplift than the coding. I still feel like communication it’s worst at. So writing tickets, getting a decision out of a group, documenting only the important things, etc. it’s still not that good at

altern8 2mo ago

AI is great because my job is a lot easier, but it produces some extremely crappy code that would've NEVER passed a code review 1 year ago. You could argue that it speeds up development by 5X or more, but then it slows down testing/verifying, code review, and in many cases it makes it impossible to review/rework by hand. I just do what I'm told at work but even though I'm sure I'm in the minority I'm extremely skeptical that LLMs can produce any good-quality code. Looks to me like we've just lowered the bar--by a lot--and stopped looking at the code that goes out. Every time I look into how Claude implemented something it's completely insane, with no way to refactor it or maintain it in the future.

serial_dev 2mo ago
> it produces some extremely crappy code that would've NEVER passed a code review 1 year ago The 10x speed up comes from the fact that we have all given up on properly reviewing each others code and we just say “meh, it will be fine, lgtm”
- QwenGlazer9000 2mo ago
  This. The bar has never been lower.
- altern8 2mo ago
  Yes, exactly my point. BUT, to me if we hadn't lowered the bar MASSIVELY, it would NOT be fine. I've seen people at my work that will push or even deploy features that don't do what their main purpose is and they just dismiss the issue saying it's a bug. As an example, a button that's supposed to open a modal window if you click on it. If you go test it and the modal doesn't open, they'll say "oh, there's a bug". You developed or even deployed something that is wrong and completely useless. Just one year ago, this would be unheard of and you would've gotten a really hard time for it. Now it's just a bug and part of how we're doing things. Like I said, I just do what I'm told at work but to me this is just wild. There's no way it won't come back to bite us.
- dools 2mo ago
  Code review has always been pointless

agumonkey 2mo ago

Agentic tools changed the workflow in our org, we were very tense before with a simple yet careful team process.. Since last winter we're now pushing a lot more but the teamwork (which was brittle before) is now mostly gone, everybody can roll on its own, but the review process didn't scale and now monthly deliveries are full of "seems to work". nobody checks properly, nobody reviews, nobody tighten any bolts.. and nobody cares much anyway, the system allows it.

andrekandre 2mo ago

> - 25% planning & aligning with other teams > - 25% coding > - 25% testing/verifying > - 25% code review/rework legit question: has anyone got ai to do all the above? i've heard people trying ai-dlc [0] but it sounds like a slog... [0] https://aws.amazon.com/jp/blogs/devops/ai-driven-development-life-cycle/ https://aws.amazon.com/jp/blogs/devops/ai-driven-development...