3 ms·

AI makes the first 90% of writing an app super easy and the last 10% way harder because you have all the subtle issues of a big codebase but none of the familia

by causal 7mo ago

AI makes the first 90% of writing an app super easy and the last 10% way harder because you have all the subtle issues of a big codebase but none of the familiarity. Most people give up there.

esafak 7mo ago
So the way is to read every line of code along the way.
ing33k 7mo ago
Agree. I’ve also noticed that feature creep tends to increase when AI is writing most of the code.
DougN7 7mo ago
Well put. And that last 10% was always the hardest part, and now it’s almost impossible because emotionally you’re even less prepared for the slog ahead.
skeeter2020 7mo ago
I spent about a week doing an "experiment" greenfield app. I saw 4 types of issues: 0. It runs way too fast and far ahead. You need to slow it down, force planning only and explicitly present a multi-step (i.e. numbered plan) and say "we'll do #1 first, then do the rest in future steps". take-away: This is likely solved with experience and changing how I work - or maybe caring less? The problem is the model can produce much faster than you can consume, but it runs down dead ends that destroy YOUR context. I think if you were running a bunch of autonomous agents this would be less noticeable, but impact 1-3 negatively and get very expensive. 1. lots of "just plain wrong" details. You catch this developing or testing because it doesn't work, or you know from experience it's wrong just by looking at it. Or you've already corrected it and need to point out the previous context. take-away: If you were vibe coding you'd solve all these eventually. Addressing #0 with "MORE AI" would probably help (i.e. AI to play/validate, etc). 2. Serious runtime issues that are not necessarily bugs. Examples: it made a lot of client-side API endpoints public that didn't even need to exist, or at least needed to be scoped to the current auth. It missed basic filtering and SQL clauses that constrained data. It hardcoded important data (but not necessarily secrets) like ports, etc. It made assumptions that worked fine in development but could be big issues in public. take-away: AI starts to build traps here. Vibe coders are in big trouble because everything works but that's not really the end goal. Problems could range from 3am downtime call-outs to getting your infrastructure owned or data breaches. More serious: experienced devs who go all-in on autonomous coding might be three months from their last manual code review and be in the same position as a vibe coder. You'd need a week or more to onboard and figure out what was going on, and fix it, which is probably too late. 3. It made (at least) one huge architectural mistake (this is a pretty simple project so I'm not sure there's space for more). I saw it coming but kept going in the spirit of my experiment. take-away: TBD. I'm going to try and use AI to refactor this, but it is non trivial. It could take as long as the initial app did to fix. If you followed the current pro-AI narrative you'd only notice it when your app started to intermittently fail - or you got you cloud provider's bill.
- Schiendelman 7mo ago
  I'm a product manager, and a lot of the things I see people do wrong is because they don't have any product management experience. It takes quite a bit of work to develop a really good theory of what should be in your functional spec. Edge cases come up all the time in real software engineering, and often handling all those cases is spread across multiple engineers. A good product manager has a view of all of it, expects many of those issues from the agent, and plans for coaching it through them.
  - nevertoolate 7mo ago
    Poe’s law is strong with this one
    - Schiendelman 7mo ago
      Tell me more! I'm trying to figure out how you got that.
      - causal 7mo ago
        I think it's just sarcasm coming from the stereotypical HN attitude that Product Managers only get in the way of the real work of engineering. Ignore it; they're basically proving your point.
      - nevertoolate 7mo ago
        I understand that you are serious. I am also serious here. Have you built anything purely with LLM which is novel and is used by people who expect that their data is managed securely, and the application is well maintained so they can trust it? I have been writing specifications, rfcs, adrs, conducting architecture reviews, code reviews and what not for quite a bit of time now. Also I’ve driven cross organisational product initiatives etc. I’m experimenting with openspec with my team now on a brownfield project and have some good results. Having said all that I seriously doubt that if you treat the english language spec and your pm oversight as the sole QA pillars of a stochastic model transformer you are making a mistake.
SAI_Peregrinus 7mo ago
And as we all know, the first 90% of writing an app takes the first 90% of the time, and the last 10% takes the other 90% of the time.
- terrabitz 7mo ago
  The 90-90 rule may need an update for a POST-LLM world "The first 90% of the code accounts for the first 9% of the development time. The remaining 10% of the code accounts for the other 9000% of the development time"
socalgal2 7mo ago
Comprehension Debt https://addyosmani.com/blog/comprehension-debt/ https://addyosmani.com/blog/comprehension-debt/

by causal 7mo ago

AI makes the first 90% of writing an app super easy and the last 10% way harder because you have all the subtle issues of a big codebase but none of the familiarity. Most people give up there.

esafak 7mo ago

So the way is to read every line of code along the way.

ing33k 7mo ago

Agree. I’ve also noticed that feature creep tends to increase when AI is writing most of the code.

DougN7 7mo ago

Well put. And that last 10% was always the hardest part, and now it’s almost impossible because emotionally you’re even less prepared for the slog ahead.

skeeter2020 7mo ago

I spent about a week doing an "experiment" greenfield app. I saw 4 types of issues: 0. It runs way too fast and far ahead. You need to slow it down, force planning only and explicitly present a multi-step (i.e. numbered plan) and say "we'll do #1 first, then do the rest in future steps". take-away: This is likely solved with experience and changing how I work - or maybe caring less? The problem is the model can produce much faster than you can consume, but it runs down dead ends that destroy YOUR context. I think if you were running a bunch of autonomous agents this would be less noticeable, but impact 1-3 negatively and get very expensive. 1. lots of "just plain wrong" details. You catch this developing or testing because it doesn't work, or you know from experience it's wrong just by looking at it. Or you've already corrected it and need to point out the previous context. take-away: If you were vibe coding you'd solve all these eventually. Addressing #0 with "MORE AI" would probably help (i.e. AI to play/validate, etc). 2. Serious runtime issues that are not necessarily bugs. Examples: it made a lot of client-side API endpoints public that didn't even need to exist, or at least needed to be scoped to the current auth. It missed basic filtering and SQL clauses that constrained data. It hardcoded important data (but not necessarily secrets) like ports, etc. It made assumptions that worked fine in development but could be big issues in public. take-away: AI starts to build traps here. Vibe coders are in big trouble because everything works but that's not really the end goal. Problems could range from 3am downtime call-outs to getting your infrastructure owned or data breaches. More serious: experienced devs who go all-in on autonomous coding might be three months from their last manual code review and be in the same position as a vibe coder. You'd need a week or more to onboard and figure out what was going on, and fix it, which is probably too late. 3. It made (at least) one huge architectural mistake (this is a pretty simple project so I'm not sure there's space for more). I saw it coming but kept going in the spirit of my experiment. take-away: TBD. I'm going to try and use AI to refactor this, but it is non trivial. It could take as long as the initial app did to fix. If you followed the current pro-AI narrative you'd only notice it when your app started to intermittently fail - or you got you cloud provider's bill.

Schiendelman 7mo ago
I'm a product manager, and a lot of the things I see people do wrong is because they don't have any product management experience. It takes quite a bit of work to develop a really good theory of what should be in your functional spec. Edge cases come up all the time in real software engineering, and often handling all those cases is spread across multiple engineers. A good product manager has a view of all of it, expects many of those issues from the agent, and plans for coaching it through them.
- nevertoolate 7mo ago
  Poe’s law is strong with this one
  - Schiendelman 7mo ago
    Tell me more! I'm trying to figure out how you got that.
    - causal 7mo ago
      I think it's just sarcasm coming from the stereotypical HN attitude that Product Managers only get in the way of the real work of engineering. Ignore it; they're basically proving your point.
    - nevertoolate 7mo ago
      I understand that you are serious. I am also serious here. Have you built anything purely with LLM which is novel and is used by people who expect that their data is managed securely, and the application is well maintained so they can trust it? I have been writing specifications, rfcs, adrs, conducting architecture reviews, code reviews and what not for quite a bit of time now. Also I’ve driven cross organisational product initiatives etc. I’m experimenting with openspec with my team now on a brownfield project and have some good results. Having said all that I seriously doubt that if you treat the english language spec and your pm oversight as the sole QA pillars of a stochastic model transformer you are making a mistake.

SAI_Peregrinus 7mo ago

And as we all know, the first 90% of writing an app takes the first 90% of the time, and the last 10% takes the other 90% of the time.

terrabitz 7mo ago
The 90-90 rule may need an update for a POST-LLM world "The first 90% of the code accounts for the first 9% of the development time. The remaining 10% of the code accounts for the other 9000% of the development time"

socalgal2 7mo ago

Comprehension Debt https://addyosmani.com/blog/comprehension-debt/ https://addyosmani.com/blog/comprehension-debt/