6 ms·

If you know how to write good code you can force AI to write good code with various techniques. It's 100% doable. You just need to figure out the problems AI ha

If you know how to write good code you can force AI to write good code with various techniques. It's 100% doable. You just need to figure out the problems AI has and find solutions to make it easier for it. Ex: extremely small contexts Modularize to modules with clear boundaries and only allow the AI to work within those boundaries. Make modules pure from IO so they are easily testable. Hide modules behind interfaces etc .. You can write 100 tests that executes within a second. You can write benchmarks etc .. AI needs boundaries and small contexts to work well. If you fail to give it that it will perform poorly. You are in charge.

IdiotSavage 5mo ago
So, basically you need to micro-manage it. Where are your 10x gains now? And is it fun to work like that?
- hansmayer 5mo ago
  Amen. Instead of freeing you up - AI enslaves you - and if it was even enslaving to a superior being at least!
- forgotaccount3 5mo ago
  > you need to micro-manage it. It is significantly easier to micro-manage an AI than a suite of junior developers. The AI doesn't replace a principal engineer, it's replacing junior and weaker senior developers who need stories broken down extremely concisely to be able to get anything done. The time it takes to break down a story such that a junior through weak senior developers can pick it up and execute it well would have the AI already done with testing built around it.
  - gylterud 5mo ago
    Juniors learn. Some juniors are potential good seniors. Over time they will internalise good architecture and be able to make good judgments on their own. Micromanaging LLMs is like having Dory from Finding Nemo as your colleague. You find ways to communicate, but there is no learning going on.
    - esafak 5mo ago
      Juniors don't always learn.
      - bigstrat2003 5mo ago
        That's still better than LLMs, which are by their very nature incapable of learning. I'll take "sometimes learns" over "never learns".
      - leptons 5mo ago
        No, but you can fire them. Can you fire an AI you're paying for? Yes, but your options are another AI that is just as bad, or worse. And it's really someone's fault for hiring a bad junior. Someone did interview them, right? Maybe the person that hired them is the problem. And maybe the person that decided to go all-in on AI is also the same problem.
    - abustamam 5mo ago
      LLMs can learn, just not the same way that juniors do. When an LLM does something wrong you can always update it's rules or skills to not make that mistake again. Or you can utilize a subagent whose sole purpose is to review code to prevent that mistake. Lots of ways you can improve LLMs over time. Of course if you don't provide that feedback loop, no learning happens. I guess the same could be said of a junior, though.
      - ModernMech 5mo ago
        Building larger systems of accountability isn't usually what people mean by learning. And besides, if telling an LLM not to do something were actually reliable, then LLMs would be a lot more useful than they are. And even if that were reliable, then you're just reinventing expert systems, which didn't work.
        abustamam 5mo ago
        I'm not sure the point of contention is whether or not an arbitrary language model is capable of understanding new concepts and not make the same mistake again, as it is being used. When people compare LLMs to juniors it's "can I have it do something pretty brain numbing, and when it makes mistakes can I invest time into preventing that from happening again, either systemically or via training?" IME this is true for LLMs, at least in how my team has been utilizing them. This doesn't make juniors worthless, as they can be useful for things that LLMs aren't good at.
  - hansmayer 5mo ago
    I think if you tried working with some junior folks, you'd be quite surprised. You know, with at least some of them choosing to use their brains and all.
- sirwhinesalot 5mo ago
  This is actually what I do. I'm extremely picky about the code and force the LLM to rewrite it 1000x times until it is basically exactly what I want. You might be wondering what is the point when it would be faster for me to just write the code myself? I have ADHD and for whatever reason telling the LLM what to do instead of doing it myself bypasses the task avoidance patterns and/or focus problems I tend to suffer from. I do not find it fun, but I am thankful for it.
  - black_knight 5mo ago
    I have used LLMs a couple of times to get started on something. I don’t have ADHD, so this is not a regular occurrence for me. But when I have tried this, I have always found the LLM solution so horrible that it instantly inspired me to do it myself. So, in that sense it worked, I got unstuck, but no LLM garbage makes it into the project.
    - abalashov 5mo ago
      It's nice to know I'm not alone in this. I have definitely used slop as inspiration by negative example.
    - zephen 5mo ago
      Even pre-AI, when working with slop generated by other humans, starting with something was often better than staring at a blank screen.
    - Forgeties79 5mo ago
      That’s how I use it for writing. I am looking for alternate wording/phrases that I usually don’t use (language habits), alternate takes of any quality just to get myself thinking along different lines, etc. Rarely do I use what the tool actually spits out. I just use it as a sounding board, like I’m chatting with a (very noob) writer. It doesn’t make me much faster but it helps me break through when I just can’t get words down.
  - yakattak 5mo ago
    This is kind of how I feel I think. Putting pen to paper for me is hard.
- nijave 5mo ago
  Honestly, I think so. I do a mix of infrastructure and programming so don't tend to have any frameworks memorized. Using AI is much quicker than constantly referencing the docs. I can also switch between codebase with different frameworks and languages and make changes without spending all day reading docs. It's also pretty good at tracing code and that's fairly straight forward to verify the results manually. It can build a flow diagram in 10-30 minutes (depending on what tool calls need allowed and how many prompts it needs) versus me taking a couple hours to do the same.
- readitalready 5mo ago
  I don't micromanage it. I let my projects custom linter micromanage it. Every project should have a custom linter for their tech stack. It would check for not just syntax errors, but architectural choices as well as taste guidelines. Whenever the LLM writes bad code, I add it to my linter to check against in the future.
- andriy_koval 5mo ago
  > So, basically you need to micro-manage it. Where are your 10x gains now? And is it fun to work like that? it depends on language and infra, but some/many require lots of boilerplate and memorizing thousands of APIs, automating this is easy LLM 10x gain. I for example write SQL myself, because boilerplate is super-minimal, and core SQL is very minimal itself, there are like 20 constructs to memorize.
- bigstrat2003 5mo ago
  The 10x gains don't exist. Anyone with a modicum of programming skill and hype resistance has been saying this for a while.
  - dkersten 5mo ago
    Sure they do: it’s easy to get 10x gains if you were only producing 0.1x before.
    - layla5alive 5mo ago
      This is what I'm seeing - for people who were slow and didn't posses a lot of depth or breadth, their blast radius and impact has skyrocketed. They can now work in unfamiliar domains quickly, without any knowledge of the nitty gritty details of those domains! For me personally, it's a tradeoff of generating the first pass code 10x more quickly, but then deeply knowing and validating the code is then 10-20x more work than it would have been if I'd written it myself (and if time is of the essence, then there's the option of shallow validation/understanding in exchange for speed - which is a compromise in rigor and path towards tech debt). In the end, none of this seems like a net win (unless you don't care about quality), and it is much less enjoyable. TL;DR; While LLMs are faster to spit out first pass code, by the time I've validated and fixed the LLM's first-pass work, I could've had my "by-hand" implementation done correctly, and had much deeper understanding out of the box. Net loss.
- luckydata 5mo ago
  you do not. Just build the right tools as you go. A small example of one of the pieces of my toolchain: https://github.com/CaliLuke/lagotto https://github.com/CaliLuke/lagotto
- aldousd666 5mo ago
  They aren't 10x gains. They're more like 3.5x gains. But still worth it. By a lot.
- rDr4g0n 5mo ago
  i love my 2x gains, still hands-on with problem, not losing context, not outsourcing thinking. just automating the boring parts.
- tinyattackcat 5mo ago
  Even if it does speed me up, coding with LLMs suck all the joy out of writing software. Constantly babysitting the agent(s) and reviewing their output carefully is probably more exhausting than just writing things myself..
- joquarky 5mo ago
  Some of us have thousands of hours in Factorio, so yes.
pron 5mo ago
That doesn't quite work, and precisely for the reason I mentioned: You can definitely tell the AI to follow some strategy, but at some point the strategy will need to change, and the AI won't tell you that (even if you tell it to). Unless you read the code every time you won't know if the AI is following the strategy and producing good results or following it and producing bad results because the strategy has to change. This can happen even in small changes: the AI will follow the strategy even if the change proves it's wrong, and if you don't pay close attention, these mistakes pile up. So yes, you might get good results in one round, but not over time. What does work is to carefully review the AI's output, although the review needs to be more careful than review of human-written code because the agents are very good at hiding the time bombs they leave behind.
- lukan 5mo ago
  How do you define "bad code"? If I instruct the AI to make small modules where I can verify they work, have tests and no side effects - then it is good enough code for me. It works, is readable and can be extended - and will turn into bad code if this is not done with care.
  - throwaway173738 5mo ago
    The concept of a small module is an architecture invariant. You’re making that decision, not the LLM. And you’ve made that decision because the machine is not good at certain things. You’re doing that because you can’t trust the LLM to make that decision on its own.
    - ChicagoDave 5mo ago
      I’m doing it because as a DDD adherent, I’ve been building software that way for 15 years without GenAI and now with GenAI I can do it faster. You can’t play whack-a-mole with GenAI. You have to start from well-known principles and watch everything it produces. Every module or bounded context has to have its own invariants. You can’t fully automate software engineering with GenAI. It seems the vast majority of GenAI users think they can and end up in the same place as the OP. Maybe learn Domain-Driven Design, Event Sourcing, and then try again. The results will be dramatically improved. https://devarch.ai/ https://devarch.ai/
hansmayer 5mo ago
> You are in charge. No, if you have to do all of the stuff you have listed to kind-of-make-it-work...You are not in charge.
wombat-man 5mo ago
Yeah I agree. It's improved quite a bit just in the past few months. The code should always be reviewed, and you need to spend some time tuning your skills and agent configs. If you're still getting bad code out of your LLM tooling, you might not be using or configuring it correctly.
insane_dreamer 5mo ago
> You are in charge. Sure. That's how I work with AI, and the way I believe that AI is meant to be use -- as a companion tool. But it's a lot of work. It saves me time for certain tasks, but not others. I haven't measured my productivity gains, but they're at most 2x. But that's not "vibe coding" (which was the point of the article) or the (false) promise of "10x productivity" and "code that writes itself" that companies are being told is going to reduce their engineering headcount tenfold.
nathan_compton 5mo ago
To me it feels like controlling a power tool. These things have a sort of momentum to them, because they do stuff so fast. It's easy to let the tool get out of hand.
candu 5mo ago
"Force" is often an unrealistic expectation, though. Taking Claude Code as an example: you can add as many rules / guidelines as you want in instruction files, but they will not be followed 100% of the time, and more is not better [1]. You can of course use PreToolUse hooks to block particularly damaging actions of the "rm -rf" variety, but this is also not 100% guaranteed unless you're able to block _all_ ways of performing that damaging action (and you would be surprised: agents will happily write custom python / bash / etc. scripts to do actions you tried to block them from doing!) Tools help instruct the agent to redo work e.g. to pass linter / formatter checks or relevant tests. But I've also seen them ignore those, often enough to be noticeable: e.g. "17 of 18 tests pass, the other 1 wasn't introduced by this feature" - regardless of whether that's actually true or not, regardless of whether I put "ALWAYS make sure ALL affected tests pass" in an instruction file somewhere. This isn't to refute your main point: yes, you can improve your chances that AI will write good code. But there is no magic bullet that will force it, 100% of the time, to write good code; this is where vibe coders without requisite coding + engineering skills hit a wall. A multi-layered approach of guidelines + progressive disclosure + tools + hooks indeed reduces the probability of bad code enough to be useful for many engineering tasks. [1] https://straion.com/blog/1m-tokens-wont-save-your-engineering-standards/ https://straion.com/blog/1m-tokens-wont-save-your-engineerin...
deterministic 5mo ago
I completely agree. I have tried N different ways to use AI and the one that really works for me is to step by step getting the AI to build one modular feature at a time (a method, a basic class etc.) I then review and fix if necessary. It works really well.

IdiotSavage 5mo ago

So, basically you need to micro-manage it. Where are your 10x gains now? And is it fun to work like that?

hansmayer 5mo ago
Amen. Instead of freeing you up - AI enslaves you - and if it was even enslaving to a superior being at least!
forgotaccount3 5mo ago
> you need to micro-manage it. It is significantly easier to micro-manage an AI than a suite of junior developers. The AI doesn't replace a principal engineer, it's replacing junior and weaker senior developers who need stories broken down extremely concisely to be able to get anything done. The time it takes to break down a story such that a junior through weak senior developers can pick it up and execute it well would have the AI already done with testing built around it.
- gylterud 5mo ago
  Juniors learn. Some juniors are potential good seniors. Over time they will internalise good architecture and be able to make good judgments on their own. Micromanaging LLMs is like having Dory from Finding Nemo as your colleague. You find ways to communicate, but there is no learning going on.
  - esafak 5mo ago
    Juniors don't always learn.
    - bigstrat2003 5mo ago
      That's still better than LLMs, which are by their very nature incapable of learning. I'll take "sometimes learns" over "never learns".
    - leptons 5mo ago
      No, but you can fire them. Can you fire an AI you're paying for? Yes, but your options are another AI that is just as bad, or worse. And it's really someone's fault for hiring a bad junior. Someone did interview them, right? Maybe the person that hired them is the problem. And maybe the person that decided to go all-in on AI is also the same problem.
  - abustamam 5mo ago
    LLMs can learn, just not the same way that juniors do. When an LLM does something wrong you can always update it's rules or skills to not make that mistake again. Or you can utilize a subagent whose sole purpose is to review code to prevent that mistake. Lots of ways you can improve LLMs over time. Of course if you don't provide that feedback loop, no learning happens. I guess the same could be said of a junior, though.
    - ModernMech 5mo ago
      Building larger systems of accountability isn't usually what people mean by learning. And besides, if telling an LLM not to do something were actually reliable, then LLMs would be a lot more useful than they are. And even if that were reliable, then you're just reinventing expert systems, which didn't work.
      - abustamam 5mo ago
        I'm not sure the point of contention is whether or not an arbitrary language model is capable of understanding new concepts and not make the same mistake again, as it is being used. When people compare LLMs to juniors it's "can I have it do something pretty brain numbing, and when it makes mistakes can I invest time into preventing that from happening again, either systemically or via training?" IME this is true for LLMs, at least in how my team has been utilizing them. This doesn't make juniors worthless, as they can be useful for things that LLMs aren't good at.
- hansmayer 5mo ago
  I think if you tried working with some junior folks, you'd be quite surprised. You know, with at least some of them choosing to use their brains and all.
sirwhinesalot 5mo ago
This is actually what I do. I'm extremely picky about the code and force the LLM to rewrite it 1000x times until it is basically exactly what I want. You might be wondering what is the point when it would be faster for me to just write the code myself? I have ADHD and for whatever reason telling the LLM what to do instead of doing it myself bypasses the task avoidance patterns and/or focus problems I tend to suffer from. I do not find it fun, but I am thankful for it.
- black_knight 5mo ago
  I have used LLMs a couple of times to get started on something. I don’t have ADHD, so this is not a regular occurrence for me. But when I have tried this, I have always found the LLM solution so horrible that it instantly inspired me to do it myself. So, in that sense it worked, I got unstuck, but no LLM garbage makes it into the project.
  - abalashov 5mo ago
    It's nice to know I'm not alone in this. I have definitely used slop as inspiration by negative example.
  - zephen 5mo ago
    Even pre-AI, when working with slop generated by other humans, starting with something was often better than staring at a blank screen.
  - Forgeties79 5mo ago
    That’s how I use it for writing. I am looking for alternate wording/phrases that I usually don’t use (language habits), alternate takes of any quality just to get myself thinking along different lines, etc. Rarely do I use what the tool actually spits out. I just use it as a sounding board, like I’m chatting with a (very noob) writer. It doesn’t make me much faster but it helps me break through when I just can’t get words down.
- yakattak 5mo ago
  This is kind of how I feel I think. Putting pen to paper for me is hard.
nijave 5mo ago
Honestly, I think so. I do a mix of infrastructure and programming so don't tend to have any frameworks memorized. Using AI is much quicker than constantly referencing the docs. I can also switch between codebase with different frameworks and languages and make changes without spending all day reading docs. It's also pretty good at tracing code and that's fairly straight forward to verify the results manually. It can build a flow diagram in 10-30 minutes (depending on what tool calls need allowed and how many prompts it needs) versus me taking a couple hours to do the same.
readitalready 5mo ago
I don't micromanage it. I let my projects custom linter micromanage it. Every project should have a custom linter for their tech stack. It would check for not just syntax errors, but architectural choices as well as taste guidelines. Whenever the LLM writes bad code, I add it to my linter to check against in the future.
andriy_koval 5mo ago
> So, basically you need to micro-manage it. Where are your 10x gains now? And is it fun to work like that? it depends on language and infra, but some/many require lots of boilerplate and memorizing thousands of APIs, automating this is easy LLM 10x gain. I for example write SQL myself, because boilerplate is super-minimal, and core SQL is very minimal itself, there are like 20 constructs to memorize.
bigstrat2003 5mo ago
The 10x gains don't exist. Anyone with a modicum of programming skill and hype resistance has been saying this for a while.
- dkersten 5mo ago
  Sure they do: it’s easy to get 10x gains if you were only producing 0.1x before.
  - layla5alive 5mo ago
    This is what I'm seeing - for people who were slow and didn't posses a lot of depth or breadth, their blast radius and impact has skyrocketed. They can now work in unfamiliar domains quickly, without any knowledge of the nitty gritty details of those domains! For me personally, it's a tradeoff of generating the first pass code 10x more quickly, but then deeply knowing and validating the code is then 10-20x more work than it would have been if I'd written it myself (and if time is of the essence, then there's the option of shallow validation/understanding in exchange for speed - which is a compromise in rigor and path towards tech debt). In the end, none of this seems like a net win (unless you don't care about quality), and it is much less enjoyable. TL;DR; While LLMs are faster to spit out first pass code, by the time I've validated and fixed the LLM's first-pass work, I could've had my "by-hand" implementation done correctly, and had much deeper understanding out of the box. Net loss.
luckydata 5mo ago
you do not. Just build the right tools as you go. A small example of one of the pieces of my toolchain: https://github.com/CaliLuke/lagotto https://github.com/CaliLuke/lagotto
aldousd666 5mo ago
They aren't 10x gains. They're more like 3.5x gains. But still worth it. By a lot.
rDr4g0n 5mo ago
i love my 2x gains, still hands-on with problem, not losing context, not outsourcing thinking. just automating the boring parts.
tinyattackcat 5mo ago
Even if it does speed me up, coding with LLMs suck all the joy out of writing software. Constantly babysitting the agent(s) and reviewing their output carefully is probably more exhausting than just writing things myself..
joquarky 5mo ago
Some of us have thousands of hours in Factorio, so yes.

pron 5mo ago

That doesn't quite work, and precisely for the reason I mentioned: You can definitely tell the AI to follow some strategy, but at some point the strategy will need to change, and the AI won't tell you that (even if you tell it to). Unless you read the code every time you won't know if the AI is following the strategy and producing good results or following it and producing bad results because the strategy has to change. This can happen even in small changes: the AI will follow the strategy even if the change proves it's wrong, and if you don't pay close attention, these mistakes pile up. So yes, you might get good results in one round, but not over time. What does work is to carefully review the AI's output, although the review needs to be more careful than review of human-written code because the agents are very good at hiding the time bombs they leave behind.

lukan 5mo ago
How do you define "bad code"? If I instruct the AI to make small modules where I can verify they work, have tests and no side effects - then it is good enough code for me. It works, is readable and can be extended - and will turn into bad code if this is not done with care.
- throwaway173738 5mo ago
  The concept of a small module is an architecture invariant. You’re making that decision, not the LLM. And you’ve made that decision because the machine is not good at certain things. You’re doing that because you can’t trust the LLM to make that decision on its own.
  - ChicagoDave 5mo ago
    I’m doing it because as a DDD adherent, I’ve been building software that way for 15 years without GenAI and now with GenAI I can do it faster. You can’t play whack-a-mole with GenAI. You have to start from well-known principles and watch everything it produces. Every module or bounded context has to have its own invariants. You can’t fully automate software engineering with GenAI. It seems the vast majority of GenAI users think they can and end up in the same place as the OP. Maybe learn Domain-Driven Design, Event Sourcing, and then try again. The results will be dramatically improved. https://devarch.ai/ https://devarch.ai/

hansmayer 5mo ago

> You are in charge. No, if you have to do all of the stuff you have listed to kind-of-make-it-work...You are not in charge.

wombat-man 5mo ago

Yeah I agree. It's improved quite a bit just in the past few months. The code should always be reviewed, and you need to spend some time tuning your skills and agent configs. If you're still getting bad code out of your LLM tooling, you might not be using or configuring it correctly.

insane_dreamer 5mo ago

> You are in charge. Sure. That's how I work with AI, and the way I believe that AI is meant to be use -- as a companion tool. But it's a lot of work. It saves me time for certain tasks, but not others. I haven't measured my productivity gains, but they're at most 2x. But that's not "vibe coding" (which was the point of the article) or the (false) promise of "10x productivity" and "code that writes itself" that companies are being told is going to reduce their engineering headcount tenfold.

nathan_compton 5mo ago

To me it feels like controlling a power tool. These things have a sort of momentum to them, because they do stuff so fast. It's easy to let the tool get out of hand.

candu 5mo ago

"Force" is often an unrealistic expectation, though. Taking Claude Code as an example: you can add as many rules / guidelines as you want in instruction files, but they will not be followed 100% of the time, and more is not better [1]. You can of course use PreToolUse hooks to block particularly damaging actions of the "rm -rf" variety, but this is also not 100% guaranteed unless you're able to block _all_ ways of performing that damaging action (and you would be surprised: agents will happily write custom python / bash / etc. scripts to do actions you tried to block them from doing!) Tools help instruct the agent to redo work e.g. to pass linter / formatter checks or relevant tests. But I've also seen them ignore those, often enough to be noticeable: e.g. "17 of 18 tests pass, the other 1 wasn't introduced by this feature" - regardless of whether that's actually true or not, regardless of whether I put "ALWAYS make sure ALL affected tests pass" in an instruction file somewhere. This isn't to refute your main point: yes, you can improve your chances that AI will write good code. But there is no magic bullet that will force it, 100% of the time, to write good code; this is where vibe coders without requisite coding + engineering skills hit a wall. A multi-layered approach of guidelines + progressive disclosure + tools + hooks indeed reduces the probability of bad code enough to be useful for many engineering tasks. [1] https://straion.com/blog/1m-tokens-wont-save-your-engineering-standards/ https://straion.com/blog/1m-tokens-wont-save-your-engineerin...

deterministic 5mo ago

I completely agree. I have tried N different ways to use AI and the one that really works for me is to step by step getting the AI to build one modular feature at a time (a method, a basic class etc.) I then review and fix if necessary. It works really well.