6 ms·

“We use Claude Code too” I understand that it’s probably impossible to sell non-AI-assisted solutions to AI-pilled companies (even when their headaches are AI-

by fwlr 3mo ago

“We use Claude Code too” I understand that it’s probably impossible to sell non-AI-assisted solutions to AI-pilled companies (even when their headaches are AI-induced), but my gut reaction to “take an AI-inflated codebase and apply AI deflation to it” is something like “that’s akin to applying two rounds of lossy transcoding; the errors don’t cancel out, they cross-multiply”.

OneDeuxTriSeiGo 3mo ago
NGL I'd argue there's a certain appeal to "use AI to prototype a feature as fast as possible and focus your engineer hours on building a comprehensive testing and fuzzing plan" followed by a "remove and review everything that can be cut without breaking the tests" cleanup pass.
- fwlr 3mo ago
  I do see the appeal, it’s easy to imagine that workflow working, and working well - but it’s hard to how it avoids this fate: https://youtu.be/QEzhxP-pdos https://youtu.be/QEzhxP-pdos
- Cyberdog 3mo ago
  The “cleanup pass” never happens, though. It’ll just be new feature on top of new feature until it’s too large to refactor.
  - colordrops 3mo ago
    Except it does. It's not some law of physics. I've done exactly this on multiple projects, both personal and corporate.
  - dymk 3mo ago
    You're describing a problem that's plagued corporate software development for decades. You just get to the "unmaintainable ball of mud" stage faster now. Every few days I spent a while on codebase architecture improvements after landing a slew of features.
  - OneDeuxTriSeiGo 3mo ago
    That's a leadership failure not a technical failure. If you can't dig your heels in and force time to tidy up technical debt then you need to find someone who can and let them take over as PM.
khasan222 3mo ago
Ngl I’m doing this right now for a client. Part of my strategy is to write out e2e tests that get a certain baseline of functionality, and then use that as the check for any change that I make to the codebase to make sure it continues to work. So workflow for a full web app is make e2e tests for all use cases. Then add a very strict duplication checker, and linter, and then just tell the ai to hit a certain duplication limit like 3%, check the linter, and add unit tests to ~95% or greater of the code. With the right CI and other checks that are deterministic you can really do a lot with a codebase.
- zie1ony 3mo ago
  I have also experimented with mutation testing. But the side effect of this approach is that it keeps public interfaces intact, and most of cases you don't want that.
  - khasan222 3mo ago
    What do you mean when you say public interfaces?
sunnybeetroot 3mo ago
Not exactly. It’s more a kin to giving an electric drill to a tradesman vs a screw driver. The tradesman will use the electric drill effectively.
- dawnerd 3mo ago
  People keep making this analogy not understanding that trades folks will use the right tool for the job, not just whatever is newer / more advanced. Air nailers exist but hammers are still used. Drills can screw in screws but screwdrivers are still used. You wouldn’t use an electric drill for a lot of jobs. People will also try to equate it to an electric saw vs hand saw, but again time and place for both.
  - sunnybeetroot 3mo ago
    I’m not discounting that other tools are to still be used. My reply is to the OP that was saying that quality gets worse applying an AI by an expert to an AI coded project. Time and place for both I agree.
  - olelele 3mo ago
    In my grandmas cottage I found an old handmade mechanical screwdriver that operates from manual pressure. That’s what got replaced by today’s power tools.
xmcp123 3mo ago
I’ve reviewed and worked on a lot of vibe coded apps (I’m an engineer, 20 YOE). I can basically split it into 3 groups. 1) Pure vibe code. No software experience. 2) AI with someone who knows the software development process and some things about software, but can’t code. 3) Engineers using AI assistance, reading/reviewing code, forcing structure. If someone can pay to replace #1 with #3 it’s very worth it. The quality between each of these tiers is enormous. I actually got curious and asked AI to look at each module in a codebase, and tell me about who wrote it without looking at git. It successfully profiled all 3 of these groups and correctly attached them to the right module.
- zie1ony 3mo ago
  Thanks for writing this. In my refactoring process I also start with code profiling. Opus is good, at finding hand written code, which sometimes is a good starting point of understand the codebase in general.
zem 3mo ago
the claude giveth and the claude taketh away. I could definitely use claude in a tightly directed manner to clean up a slopified codebase (and I would enjoy doing so), you just need to think of it as closer to a power tool than an agent.
adam_arthur 3mo ago
The level of "slop" produced by AI is a direct function of skill of the developer and broadness of the prompts. Broad prompts by unskilled users results in a complete mess. Targeted prompts by a skilled person reviewing the code produces something better. Quality of application varies widely, and generally agree with the categories mentioned by the sibling post.
- aytigra 3mo ago
  My experience as well, I've been developing a native macos app using CC. As a web dev I didn't know much about the stack. Nothing too fancy a kind of folder gallery-player with tags embedded in filenames, a bit like TagSpaces. Process was - produced a detailed feature spec - multiple iteration of "I want this and that", make it into coherent spec", "this this and that is not correct, change to that". Made it write architecture spec(which I didn't read because too unfamiliar) and split it into tasks. Then it was implementing tasks, after each I did a change/fix those ~10 things iteration and spec corrections. It was good to a point, but then when I started to hit performance problems I had to step in look at the code, and very often fight with CC, confront its "this is the only way", force it to do web search for proper ways to deal with problems and even explain very simple things about proper DB usage. At some point it asked me something like "is it ok for schema migration to just fail or we need to implement complicated handling?", I have answered "it just shouldn't leave app locked in schema failure", and guess what was CC solution? - it wrote an error handler which just drops DB and recreates fresh one on ANY schema failure. And if I didn't happen to peek at the code and ask wtf it is doing, that would've been an exiting UX. I've spent about month's worth of $20 CC subscription tokens using Opus 4.8 on xhigh, AND about 70 hours of my time to get it to a point where it is good. So "anyone can just code what they want now" is correct only to a point, MVP will work, but beyond that experience will be subpar, and it still needs lots and lots of iterations of explaining what you want. Then because normal user knows very little about how software works they won't be able to ask AI the right questions, confront it and rate of improvement vs token usage will hit rock bottom.

by fwlr 3mo ago

OneDeuxTriSeiGo 3mo ago

NGL I'd argue there's a certain appeal to "use AI to prototype a feature as fast as possible and focus your engineer hours on building a comprehensive testing and fuzzing plan" followed by a "remove and review everything that can be cut without breaking the tests" cleanup pass.

fwlr 3mo ago
I do see the appeal, it’s easy to imagine that workflow working, and working well - but it’s hard to how it avoids this fate: https://youtu.be/QEzhxP-pdos https://youtu.be/QEzhxP-pdos
Cyberdog 3mo ago
The “cleanup pass” never happens, though. It’ll just be new feature on top of new feature until it’s too large to refactor.
- colordrops 3mo ago
  Except it does. It's not some law of physics. I've done exactly this on multiple projects, both personal and corporate.
- dymk 3mo ago
  You're describing a problem that's plagued corporate software development for decades. You just get to the "unmaintainable ball of mud" stage faster now. Every few days I spent a while on codebase architecture improvements after landing a slew of features.
- OneDeuxTriSeiGo 3mo ago
  That's a leadership failure not a technical failure. If you can't dig your heels in and force time to tidy up technical debt then you need to find someone who can and let them take over as PM.

khasan222 3mo ago

Ngl I’m doing this right now for a client. Part of my strategy is to write out e2e tests that get a certain baseline of functionality, and then use that as the check for any change that I make to the codebase to make sure it continues to work. So workflow for a full web app is make e2e tests for all use cases. Then add a very strict duplication checker, and linter, and then just tell the ai to hit a certain duplication limit like 3%, check the linter, and add unit tests to ~95% or greater of the code. With the right CI and other checks that are deterministic you can really do a lot with a codebase.

zie1ony 3mo ago
I have also experimented with mutation testing. But the side effect of this approach is that it keeps public interfaces intact, and most of cases you don't want that.
- khasan222 3mo ago
  What do you mean when you say public interfaces?

sunnybeetroot 3mo ago

Not exactly. It’s more a kin to giving an electric drill to a tradesman vs a screw driver. The tradesman will use the electric drill effectively.

dawnerd 3mo ago
People keep making this analogy not understanding that trades folks will use the right tool for the job, not just whatever is newer / more advanced. Air nailers exist but hammers are still used. Drills can screw in screws but screwdrivers are still used. You wouldn’t use an electric drill for a lot of jobs. People will also try to equate it to an electric saw vs hand saw, but again time and place for both.
- sunnybeetroot 3mo ago
  I’m not discounting that other tools are to still be used. My reply is to the OP that was saying that quality gets worse applying an AI by an expert to an AI coded project. Time and place for both I agree.
- olelele 3mo ago
  In my grandmas cottage I found an old handmade mechanical screwdriver that operates from manual pressure. That’s what got replaced by today’s power tools.

xmcp123 3mo ago

I’ve reviewed and worked on a lot of vibe coded apps (I’m an engineer, 20 YOE). I can basically split it into 3 groups. 1) Pure vibe code. No software experience. 2) AI with someone who knows the software development process and some things about software, but can’t code. 3) Engineers using AI assistance, reading/reviewing code, forcing structure. If someone can pay to replace #1 with #3 it’s very worth it. The quality between each of these tiers is enormous. I actually got curious and asked AI to look at each module in a codebase, and tell me about who wrote it without looking at git. It successfully profiled all 3 of these groups and correctly attached them to the right module.

zie1ony 3mo ago
Thanks for writing this. In my refactoring process I also start with code profiling. Opus is good, at finding hand written code, which sometimes is a good starting point of understand the codebase in general.

zem 3mo ago

the claude giveth and the claude taketh away. I could definitely use claude in a tightly directed manner to clean up a slopified codebase (and I would enjoy doing so), you just need to think of it as closer to a power tool than an agent.

adam_arthur 3mo ago

The level of "slop" produced by AI is a direct function of skill of the developer and broadness of the prompts. Broad prompts by unskilled users results in a complete mess. Targeted prompts by a skilled person reviewing the code produces something better. Quality of application varies widely, and generally agree with the categories mentioned by the sibling post.

aytigra 3mo ago
My experience as well, I've been developing a native macos app using CC. As a web dev I didn't know much about the stack. Nothing too fancy a kind of folder gallery-player with tags embedded in filenames, a bit like TagSpaces. Process was - produced a detailed feature spec - multiple iteration of "I want this and that", make it into coherent spec", "this this and that is not correct, change to that". Made it write architecture spec(which I didn't read because too unfamiliar) and split it into tasks. Then it was implementing tasks, after each I did a change/fix those ~10 things iteration and spec corrections. It was good to a point, but then when I started to hit performance problems I had to step in look at the code, and very often fight with CC, confront its "this is the only way", force it to do web search for proper ways to deal with problems and even explain very simple things about proper DB usage. At some point it asked me something like "is it ok for schema migration to just fail or we need to implement complicated handling?", I have answered "it just shouldn't leave app locked in schema failure", and guess what was CC solution? - it wrote an error handler which just drops DB and recreates fresh one on ANY schema failure. And if I didn't happen to peek at the code and ask wtf it is doing, that would've been an exiting UX. I've spent about month's worth of $20 CC subscription tokens using Opus 4.8 on xhigh, AND about 70 hours of my time to get it to a point where it is good. So "anyone can just code what they want now" is correct only to a point, MVP will work, but beyond that experience will be subpar, and it still needs lots and lots of iterations of explaining what you want. Then because normal user knows very little about how software works they won't be able to ask AI the right questions, confront it and rate of improvement vs token usage will hit rock bottom.