4 ms·

LLMs don't lack the virtue of laziness: it has it if you want it to, by just having a base prompt that matches intent. I've had good success convincing claude b

by hibikir 6mo ago

LLMs don't lack the virtue of laziness: it has it if you want it to, by just having a base prompt that matches intent. I've had good success convincing claude backed agents to aim for minimal code changes, make deduplication passes, and basically every other reasonable "instinct" of a very senior dev. It's not knowledge that the models haven't integrated, but one that many don't have on their forefront with default settings. I bet we've all seen the models that over-edit everything, and act like the crazy mid-level dev that fiddles with the entire codebase without caring one bit about anyone else's changes, or any risk of knowledge loss due to overfiddling. And on Jess' comments on validating docs vs generating them... It's a traditional locking problem, with traditional solutions. And it's not as if the agent cannot read git, and realize when one thing is done first, in anticipation of the other by convention. I'm quite senior: In fact, I have been a teammate of a couple of people mention in this article. I suspect that they'd not question my engineering standards. And yet I've no seen any of that kind of debt in my LLM workflows: if anything, by most traditional forms of evaluating software quality, the projects I work on are better than what they were 5, 10 years ago, using the same metrics as back then. And it's not magic or anything, but making sure there are agents running sharing those quality priorities. But I am getting work done, instead of spending time looking for attention in conferences.

dc_giant 6mo ago
Mind sharing the instructions you give Claude to go for minimal code changes etc?
- awesome_dude 6mo ago
  I often say to Claude "you're doing X when I want Y, how can I get you to follow the Y path without fail" and Claude will respond with "Edit my claude.md to include the following" which I then ask Claude to do.
  - mettamage 6mo ago
    Ah yea I do that too. I often have reflection sessions with Claude where I ask it "how can I make sure you do behavior X so we get outcome Y?" It works relatively well but not always.
    - awesome_dude 6mo ago
      [dead]
  - sally_glance 6mo ago
    Not sure this is a great idea. The model only internalized what it was trained on and writing prompts/context for itself isn't part of that. I try to keep my context as clean as possible, mostly today's models seem smart/aligned enough to be steered by a couple of keywords.
- gherkinnn 6mo ago
  I regularly prompt and re-prompt the clanker with esoteric terms like "subtractive changes", "create by removing" and more common phrases like "make the change easy, then make the easy change", "yagni", and "vertical slices", and "WET code is desirable". It mostly works. CC's plan mode creates a plan by cleaning up first, then defining narrow, integrated steps. Mentioning "subtractive" and "yagni" appears to be a reliable enough way for an LLM to choose a minimal path. To my mind these instructions remain incantations and I feel like an alchemist of old.
  - physicles 6mo ago
    Was just listening to the Lenny’s Podcast interview with Simon Willison, who mentioned another such incantation: red/green TDD. The model knows what this means and it just does it, with a nice bump in code quality apparently. I’m trying out another, what I call the principle of path independence. It’s the idea that the code should reflect only the current requirements, and not the order in which functionality was added — in other words, if you should decide to rebuild the system again from scratch tomorrow, the code should look broadly similar to its current state. It sort of works even though this isn’t a real thing that’s in its training data.
- coalstartprob 6mo ago
  [dead]
- plaguuuuuu 6mo ago
  "Coding style should reflect the casual minimalism of expert programmers" used to to bump it up a bit, I guess it had the right activations or whatever, but I haven't bothered as much with this stuff with recent models cause (a) writing "You are a gigachad developer who is a Level 99 Staff Wizard at FAANG" doesn't work any more and (b) More or less they code like what they can see in their context; if you have a shit codebase you're going to get more shit, so theoretically I'd give a baseline coding style in some instructions, but realistically I haven't have enough motivation to bother with the last thingy I was working on.
marktani 6mo ago
I agree with your sentiment here. However: > if anything, by most traditional forms of evaluating software quality, the projects I work on are better than what they were 5, 10 years ago, using the same metrics as back then. In this side sentence you're introducing so much vagueness. Can you share insights to get some validation on your claim? What metrics are you using and how is your code from 10, 5, 0 years performing? I feel throwing in a vague claim like that unnecessarily dilutes your message and distracts from the point. But, if you do have more to share I'd be curious to learn more.
- awesome_dude 6mo ago
  The anecdote the GP is providing there rings true for me too - although I'm not sure if I am going offer better detail. I'm a proponent of architectural styles like MVC, SOLID, hexagonal architecture, etc, and in pre-LLM workflows, "human laziness" often led to technical debt: a developer might lazily leak domain logic into a controller or skip writing an interface just to save time. The code I get the LLM to emit is a lot more compliant with those BUT there is a caveat that the LLMs do have a habit of "forgetting" the specific concerns of the given file/package/etc, and I frequently have to remind it. The "metric" improvement isn't that the LLM is a better architect than a senior dev; it's that it reduces the cost of doing things the right way. The delta between "quick and dirty" and "cleanly architected" has shrunk to near zero, so the "clean" version becomes the path of least resistance. I'm seeing less "temporary" kludges because the LLM almost blindly follows my requests
  - stickfigure 6mo ago
    I don't think I'd like your code. But apparently there's enough implied YAGNI in my CLAUDE.md to prevent the unnecessary interfaces and layers of separation that you apparently like. So I guess there is a flavor for everyone.
  - plaguuuuuu 6mo ago
    I've recently had the interesting experience of working on a Clean Architecture project for the first time. Pre and post-LLM adoption. It has been... difficult. Services/modules organised by infrastructural layer rather than by feature. A mediator pattern abstracted away the handling of commands. Just in case one day you needed CreateFooCommand to be executed by a different handler, or something, I dunno. It was so hard to figure out how to navigate everything. And it felt like the entire tradeoff was for the purpose of stopping smoothbrains from adding the ORM to an API endpoint - but with the cost of this crushing accidental complexity that made it hard for everyone to hold everything in their heads, not just for me but also for the smart guys on the team. It turns out that the LLMs also performed extremely poorly. All the heavy abstractions were too hard for them (not to mention most of the developers). I knew I had no chance of shifting things away from that paradigm. But as luck would have it... we started basically vibe-rewriting it from scratch without bullshit enterprisey crap and its (a) dead simple (b) has most of the features after one month (c) even though the code is questionable, inelegant AI slop, with nearly zero regard to proper architectural design, it's way easier to deal with than before I've never felt so vindicated.
whattheheckheck 6mo ago
You should go get attention at conferences. You could write a book called Practical LLM Coding
- cmdoptesc 6mo ago
  Do you think O'Reilly would still put an animal on the cover?
  - defrost 6mo ago
    Sure, K9 - https://cms.doctorwho.tv/sites/default/files/2022-02/2et2o0000001000.jpg https://cms.doctorwho.tv/sites/default/files/2022-02/2et2o00...
  - lubujackson 6mo ago
    A mockingbird?
    - stronglikedan 6mo ago
      A bull. Bonus if the background is a china shop.

by hibikir 6mo ago

dc_giant 6mo ago

Mind sharing the instructions you give Claude to go for minimal code changes etc?

awesome_dude 6mo ago
I often say to Claude "you're doing X when I want Y, how can I get you to follow the Y path without fail" and Claude will respond with "Edit my claude.md to include the following" which I then ask Claude to do.
- mettamage 6mo ago
  Ah yea I do that too. I often have reflection sessions with Claude where I ask it "how can I make sure you do behavior X so we get outcome Y?" It works relatively well but not always.
  - awesome_dude 6mo ago
    [dead]
- sally_glance 6mo ago
  Not sure this is a great idea. The model only internalized what it was trained on and writing prompts/context for itself isn't part of that. I try to keep my context as clean as possible, mostly today's models seem smart/aligned enough to be steered by a couple of keywords.
gherkinnn 6mo ago
I regularly prompt and re-prompt the clanker with esoteric terms like "subtractive changes", "create by removing" and more common phrases like "make the change easy, then make the easy change", "yagni", and "vertical slices", and "WET code is desirable". It mostly works. CC's plan mode creates a plan by cleaning up first, then defining narrow, integrated steps. Mentioning "subtractive" and "yagni" appears to be a reliable enough way for an LLM to choose a minimal path. To my mind these instructions remain incantations and I feel like an alchemist of old.
- physicles 6mo ago
  Was just listening to the Lenny’s Podcast interview with Simon Willison, who mentioned another such incantation: red/green TDD. The model knows what this means and it just does it, with a nice bump in code quality apparently. I’m trying out another, what I call the principle of path independence. It’s the idea that the code should reflect only the current requirements, and not the order in which functionality was added — in other words, if you should decide to rebuild the system again from scratch tomorrow, the code should look broadly similar to its current state. It sort of works even though this isn’t a real thing that’s in its training data.
coalstartprob 6mo ago
[dead]
plaguuuuuu 6mo ago
"Coding style should reflect the casual minimalism of expert programmers" used to to bump it up a bit, I guess it had the right activations or whatever, but I haven't bothered as much with this stuff with recent models cause (a) writing "You are a gigachad developer who is a Level 99 Staff Wizard at FAANG" doesn't work any more and (b) More or less they code like what they can see in their context; if you have a shit codebase you're going to get more shit, so theoretically I'd give a baseline coding style in some instructions, but realistically I haven't have enough motivation to bother with the last thingy I was working on.

marktani 6mo ago

I agree with your sentiment here. However: > if anything, by most traditional forms of evaluating software quality, the projects I work on are better than what they were 5, 10 years ago, using the same metrics as back then. In this side sentence you're introducing so much vagueness. Can you share insights to get some validation on your claim? What metrics are you using and how is your code from 10, 5, 0 years performing? I feel throwing in a vague claim like that unnecessarily dilutes your message and distracts from the point. But, if you do have more to share I'd be curious to learn more.

awesome_dude 6mo ago
The anecdote the GP is providing there rings true for me too - although I'm not sure if I am going offer better detail. I'm a proponent of architectural styles like MVC, SOLID, hexagonal architecture, etc, and in pre-LLM workflows, "human laziness" often led to technical debt: a developer might lazily leak domain logic into a controller or skip writing an interface just to save time. The code I get the LLM to emit is a lot more compliant with those BUT there is a caveat that the LLMs do have a habit of "forgetting" the specific concerns of the given file/package/etc, and I frequently have to remind it. The "metric" improvement isn't that the LLM is a better architect than a senior dev; it's that it reduces the cost of doing things the right way. The delta between "quick and dirty" and "cleanly architected" has shrunk to near zero, so the "clean" version becomes the path of least resistance. I'm seeing less "temporary" kludges because the LLM almost blindly follows my requests
- stickfigure 6mo ago
  I don't think I'd like your code. But apparently there's enough implied YAGNI in my CLAUDE.md to prevent the unnecessary interfaces and layers of separation that you apparently like. So I guess there is a flavor for everyone.
- plaguuuuuu 6mo ago
  I've recently had the interesting experience of working on a Clean Architecture project for the first time. Pre and post-LLM adoption. It has been... difficult. Services/modules organised by infrastructural layer rather than by feature. A mediator pattern abstracted away the handling of commands. Just in case one day you needed CreateFooCommand to be executed by a different handler, or something, I dunno. It was so hard to figure out how to navigate everything. And it felt like the entire tradeoff was for the purpose of stopping smoothbrains from adding the ORM to an API endpoint - but with the cost of this crushing accidental complexity that made it hard for everyone to hold everything in their heads, not just for me but also for the smart guys on the team. It turns out that the LLMs also performed extremely poorly. All the heavy abstractions were too hard for them (not to mention most of the developers). I knew I had no chance of shifting things away from that paradigm. But as luck would have it... we started basically vibe-rewriting it from scratch without bullshit enterprisey crap and its (a) dead simple (b) has most of the features after one month (c) even though the code is questionable, inelegant AI slop, with nearly zero regard to proper architectural design, it's way easier to deal with than before I've never felt so vindicated.

whattheheckheck 6mo ago

You should go get attention at conferences. You could write a book called Practical LLM Coding

cmdoptesc 6mo ago
Do you think O'Reilly would still put an animal on the cover?
- defrost 6mo ago
  Sure, K9 - https://cms.doctorwho.tv/sites/default/files/2022-02/2et2o0000001000.jpg https://cms.doctorwho.tv/sites/default/files/2022-02/2et2o00...
- lubujackson 6mo ago
  A mockingbird?
  - stronglikedan 6mo ago
    A bull. Bonus if the background is a china shop.