14 ms·

How I write software with LLMs

• christofosho 7mo ago
  I like reading these types of breakdowns. Really gives you ideas and insight into how others are approaching development with agents. I'm surprised the author hasn't broken down the developer agent persona into smaller subagents. There is a lot of context used when your agent needs to write in a larger breadth of code areas (i.e. database queries, tests, business logic, infrastructure, the general code skeleton). I've also read[1] that having a researcher and then a planner helps with context management in the pre-dev stage as well. I like his use of multiple reviewers, and am similarly surprised that they aren't refined into specialized roles. I'll admit to being a "one prompt to rule them all" developer, and will not let a chat go longer than the first input I give. If mistakes are made, I fix the system prompt or the input prompt and try again. And I make sure the work is broken down as much as possible. That means taking the time to do some discovery before I hit send. Is anyone else using many smaller specific agents? What types of patterns are you employing? TIA 1. https://github.com/humanlayer/advanced-context-engineering-for-coding-agents/blob/main/ace-fca.md https://github.com/humanlayer/advanced-context-engineering-f...
  • marcus_holmes 7mo ago
    that reference you give is pretty dated now, based on a talk from August which is the Beforetimes of the newer models that have given such a step change in productivity. The key change I've found is really around orchestration - as TFA says, you don't run the prompt yourself. The orchestrator runs the whole thing. It gets you to talk to the architect/planner, then the output of that plan is sent to another agent, automatically. In his case he's using an architect, a developer, and some reviewers. I've been using a Superpowers-based [0] orchestration system, which runs a brainstorm, then a design plan, then an implementation plan, then some devs, then some reviewers, and loops back to the implementation plan to check progress and correctness. It's actually fun. I've been coding for 40+ years now, and I'm enjoying this :) [0] https://github.com/obra/superpowers https://github.com/obra/superpowers
    • indigodaddy 7mo ago
      Can you bolt superpowers onto an existing project so that it uses the approach going forward (I'm using Opencode), or would that get too messy?
      • eclipxe 7mo ago
        Yes. But gsd is even better - especially gsd2
        • marcus_holmes 7mo ago
          looks interesting, thanks for the tip :)
      • marcus_holmes 7mo ago
        These are just skills, so you can add the skills to your setup and start using them whenever you like.
        • indigodaddy 7mo ago
          Thanks
  • felixsells 7mo ago
    [flagged]
  • stavros 7mo ago
    I don't think that splitting into subagents that use the same model will really help. I need to clarify this in the post, but the split is 1) so I can use Sonnet to code and save on some tokens and 2) so I can get other models to review, to get a different perspective. It seems to me that splitting into subagents that use the same model is kind of like asking a person to wear three different hats and do three different parts of the job instead of just asking them to do it all with one hat. You're likely to get similar results.
    • chriswarbo 7mo ago
      I'm considering using subagents, as a way to manage context and delegate "simple" tasks to cheaper models (if you want to see tokens burn, watch Opus try fixing a misplaced ')' in a Lisp file!). I see what you mean w.r.t. different hats; but is it useful to have different tools available? For example, a "planner" having Web access and read-only file access, versus a "developer" having write access to files but no Web access?
      • stavros 7mo ago
        Yes, if you want to separate capabilities, definitely.
• indigodaddy 7mo ago
  This was on the front page and then got completely buried for some reason. Super weird.
  • mjmas 7mo ago
    On the front page at the moment. Position 12
    • indigodaddy 7mo ago
      Maybe I missed it. Sometimes when you're scanning for something your brain intentionally doesn't want to see it, I've noticed. Anyway I'm not Stavros obviously, just thought this was a good article.
  • stainlu 7mo ago
    [flagged]
  • wiseowise 7mo ago
    Anti-AI conspiracy, obviously.
• plastic041 7mo ago
  I wanted to know how to make softwares with LLM "without losing the benefit of knowing how the entire system works" and "intimately familiar with each project’s architecture and inner workings", while "have never even read most of their code". (Because obviously, you can't.) But OP didn't explain that. You tell LLM to create something, and then use another LLM to review it. It might make the result safer, but it doesn't mean that YOU understand the architecture. No one does.
  • ashwinsundar 7mo ago
    Hot take: you can't have your cake and eat it too. If you aren't writing code, designing the system, creating architecture, or even writing the prompt, then you're not understanding shit. You're playing slots with stochastic parrots The code grows beyond my usual comprehension, I'd have to really read through it for a while. Sometimes the LLMs can't fix a bug so I just work around it or ask for random changes until it goes away. It's not too bad for throwaway weekend projects, but still quite amusing. I'm building a project or webapp, but it's not really coding - I just see stuff, say stuff, run stuff, and copy paste stuff, and it mostly works. - Karpathy 2025
    • simonw 7mo ago
      Your Karpathy quote there is out of context. It starts with: https://twitter.com/karpathy/status/1886192184808149383 https://twitter.com/karpathy/status/1886192184808149383 There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists. Not all AI-assisted programming is vibe coding. If you're paying attention to the code that's being produced you can guide it towards being just as high quality (or even higher quality) than code you would have written by hand.
      • ashwinsundar 7mo ago
        It's appropriate for the commenter I was replying to, who asked how they can understand things, "while having never even read most of their code." I like AI-assisted programming, but if I fail to even read the code produced, then I might as well treat it like a no-code system. I can understand the high-levels of how no-code works, but as soon as it breaks, it might as well be a black box. And this only gets worse as the codebase spans into the tens of thousands of lines without me having read any of it. The (imperfect) analogy I'm working on is a baker who bakes cakes. A nearby grocery store starts making any cake they want, on demand, so the baker decides to quit baking cakes and buy them from the store. The baker calls the store anytime they want a new cake, and just tells them exactly what they want. How long can that baker call themself a "baker"? How long before they forget how to even bake a cake, and all they can do is get cakes from the grocer?
• silisili 7mo ago
  I'm not sure the notion I keep seeing of "it's ok, we still architect, it just writes the code"(paraphrased) sits well with me. I've not tested it with architecting a full system, but assuming it isn't good at it today... it's only a matter of time. Then what is our use?
  • borski 7mo ago
    LLMs can build anything. The real question is what is worth building, and how it’s delivered. That is what is still human. LLMs, by nature of not being human, cannot understand humans as well as other humans can. (See every attempt at using an LLM as a therapist) In short: LLMs will eventually be able to architect software. But it’s still just a tool
    • silisili 7mo ago
      What is the use of software eng/architect at that point? It's a tool, but one that product or C levels can use directly as I see it?
      • borski 7mo ago
        Yes, for building something But for building the right thing? Doubtful. Most of a great engineer’s work isn’t writing code, but interrogating what people think their problems are, to find what the actual problems are. In short: problem solving, not writing code.
        • mattmanser 7mo ago
          Where's this delusion come from recently that great engineers didnt write code? What a load of crap. All you're doing is describing a different job role. What you're talking about is BA work, and a subset of engineers are great at it, but most are just ok. You're claiming a part of the job that was secondary, and not required, is now the whole job.
          • 7mo ago
• jumploops 7mo ago
  This is similar to how I use LLMs (architect/plan -> implement -> debug/review), but after getting bit a few times, I have a few extra things in my process: The main difference between my workflow and the authors, is that I have the LLM "write" the design/plan/open questions/debug/etc. into markdown files, for almost every step. This is mostly helpful because it "anchors" decisions into timestamped files, rather than just loose back-and-forth specs in the context window. Before the current round of models, I would religiously clear context and rely on these files for truth, but even with the newest models/agentic harnesses, I find it helps avoid regressions as the software evolves over time. A minor difference between myself and the author, is that I don't rely on specific sub-agents (beyond what the agentic harness has built-in for e.g. file exploration). I say it's minor, because in practice the actual calls to the LLMs undoubtedly look quite similar (clean context window, different task/model, etc.). One tip, if you have access, is to do the initial design/architecture with GPT-5.x Pro, and then take the output "spec" from that chat/iteration to kick-off a codex/claude code session. This can also be helpful for hard to reason about bugs, but I've only done that a handful of times at this point (i.e. funky dynamic SVG-based animation snafu).
  • lelele 7mo ago
    > The main difference between my workflow and the authors, is that I have the LLM "write" the design/plan/open questions/debug/etc. into markdown files, for almost every step. > > This is mostly helpful because it "anchors" decisions into timestamped files, rather than just loose back-and-forth specs in the context window. Would you please expand on this? Do you make the LLM append their responses to a Markdown file, prefixed by their timestamps, basically preserving the whole context in a file? Or do you make the LLM update some reference files in order to keep a "condensed" context? Thank you.
    • aix1 7mo ago
      Not the GP, but I currently use a hierarchy of artifacts: requirements doc -> design docs (overall and per-component) -> code+tests. All artifacts are version controlled. Each level in the hierarchy is empirically ~5X smaller than the level below. This, plus sharding the design docs by component, helps Claude navigate the project and make consistent decision across sessions. My workflow for adding a feature goes something like this: 1. I iterate with Claude on updating the requirements doc to capture the desired final state of the system from the user's perspective. 2. Once that's done, a different instance of Claude reads the requirements and the design docs and updates the latter to address all the requirements listed in the former. This is done interactively with me in the loop to guide and to resolve ambiguity. 3. Once the technical design is agreed, Claude writes a test plan, usually almost entirely autonomously. The test plan is part of each design doc and is updated as the design evolves. 3a. (Optionally) another Claude instance reviews the design for soundness, completeness, consistency with itself and with the requirements. I review the findings and tell it what to fix and what to ignore. 4. Claude brings unit tests in line with what the test plan says, adding/updating/removing tests but not touching code under test. 4a. (Optionally) the tests are reviewed by another instance of Claude for bugs and inconsistencies with the test plan or the style guide. 5. Claude implements the feature. 5a. (Optionally) another instance reviews the implementation. For complex changes, I'm quite disciplined to have each step carried out in a different session so that all communinications are done via checked-in artifacts and not through context. For simple changes, I often don't bother and/or skip the reviews. From time to time, I run standalone garbage collection and consistency checks, where I get Claude to look for dead code, low-value tests, stale parts of the design, duplication, requirements-design-tests-code drift etc. I find it particularly valuable to look for opportunities to make things simpler or even just smaller (fewer tokens/less work to maintain). Occasionally, I find that I need to instruct Claude to write a benchmark and use it with a profiler to opimise something. I check these in but generally don't bother documenting them. In my case they tend to be one-off things and not part of some regression test suite. Maybe I should just abandon them & re-create if they're ever needed again. I also have a (very short) coding style guide. It only includes things that Claude consistently gets wrong or does in ways that are not to my liking.
• akhrail1996 7mo ago
  Genuine question: what's the evidence that the architect → developer → reviewer pipeline actually produces better results than just... talking to one strong model in one session? The author uses different models for each role, which I get. But I run production agents on Opus daily and in my experience, if you give it good context and clear direction in a single conversation, the output is already solid. The ceremony of splitting into "architect" and "developer" feels like it gives you a sense of control and legibility, but I'm not convinced it catches errors that a single model wouldn't catch on its own with a good prompt.
  • troupo 7mo ago
    > produces better results than just... talking to one strong model in one session? I think the author admits that it doesn't, doesn't realise it and just goes on: --- start quote --- On projects where I have no understanding of the underlying technology (e.g. mobile apps), the code still quickly becomes a mess of bad choices. However, on projects where I know the technologies used well (e.g. backend apps, though not necessarily in Python), this hasn’t happened yet --- end quote ---
  • imiric 7mo ago
    Evidence? My friend, most of the practices in this field are promoted and adopted based on hand-waving, feelings, and anecdata from influencers. Maybe you should write and share your own article to counter this one.
    • z3t4 7mo ago
      Also if something is fun, we prefer to to it that way instead of the boring way. Then it depends on how many mines you step on, after a while you try to avoid the mines. That's when your productivity goes down radically. If we see something shiny we'll happily run over the minefield again though.
  • jaredklewis 7mo ago
    > what's the evidence What’s the evidence for anything software engineers use? Tests, type checkers, syntax highlighting, IDEs, code review, pair programming, and so on. In my experience, evidence for the efficacy of software engineering practices falls into two categories: - the intuitions of developers, based in their experiences. - scientific studies, which are unconvincing. Some are unconvincing because they attempt to measure the productivity of working software engineers, which is difficult; you have to rely on qualitative measures like manager evaluations or quantitative but meaningless measures like LOC or tickets closed. Others are unconvincing because they instead measure the practice against some well defined task (like a coding puzzle) that is totally unlike actual software engineering. Evidence for this LLM pattern is the same. Some developers have an intuition it works better.
• xhale 7mo ago
  Hi, anyone has a simple example/scaffold how to set up agents/skills like this? I’ve looked at the stavrobots repo and only saw an AGENTS.md. Where do these skills live then? (I have seen obra/superpowers mentioned in the comments, but that’s already too complex and with an ui focus)
  • lbreakjai 7mo ago
    I played with this over the weekend: https://github.com/marcosloic/notion-agent-hive https://github.com/marcosloic/notion-agent-hive Ultimately, it's just a bunch of markdown files that live in an `/agents` folder, with some meta-information that will depend on the harness you use.
  • stavros 7mo ago
    These skills live in my home directory, that's why they aren't in the repos. I can upload them if you want.
    • xhale 7mo ago
      [dead]
    • kkarolis 7mo ago
      Not original commenter, but would be curious (and thankful) to see it.
      • stavros 7mo ago
        I've updated the post with them, let me know if they work!
        • xhale 7mo ago
          Thanks! (btw you have a typo on the second reviewer in the architect file. Is opencode smart enough to figure out the correct one?)
• biang15343100 7mo ago
  [dead]
• thenthenthen 7mo ago
  Haha love the Sleight of hand irregular wall clock idea. I once had a wall clock where the hand showing the seconds would sometimes jump backwards, it was extremely unsettling somehow because it was random. It really did make me question my sanity.
  • kqr 7mo ago
    This used to be one of my recurring nightmares when I was a child. The three I remember were (1) clocks suddenly starting to go backwards, either partially or completely; (2) radio turning on without being able to turn it off, and (3) house fire. There really is something about clocks.
• ForgotMyUUID 7mo ago
  TL DR; Don't, please :)
• imiric 7mo ago
  Ah, another one of these. I'm eager to learn how a "social climber" talks to a chatbot. I'm sure it's full of novel insight, unlike thousands of other articles like this one.
  • indigodaddy 7mo ago
    Dudes not a social climber. You are making a lot of assumptions.
• zapkyeskrill 7mo ago
  What's the point of writing this? In a few weeks a new model will come out and make your current work pattern obsolete (a process described in the post itself)
  • zingar 7mo ago
    Solidifying the ideas in writing helps the author improve them, and helps them and the rest of us understand what to look for in the next generation of models.
• kleiba 7mo ago
  I write very little code these days, so I've been following the AI development mostly from the backseat. One aspect I fail to grasp perfectly is what the practical differences are between CLI (so terminal-based) agents and ones fully integrated into an IDE. Could someone chime in and give their opinion on what are the pros and cons of either approach?
  • rullelito 7mo ago
    For me, I use an IDE if I plan to look at the code.
    • kleiba 7mo ago
      So, to you basically the distinction is "fully vibe-coded" vs. "with human in the loop"?
  • huthuthukhuo 7mo ago
    [dead]
  • claud_ia 7mo ago
    [dead]
  • zingar 7mo ago
    I guess you’re probably looking for someone who uses cursor etc to answer but here’s a data point from someone a bit off the beaten path. My editor supports both modes (emacs). I have the editor integration features (diff support etc) turned off and just use emacs to manage 5+ shells that each have a CLI agent (one of Claude, opencode, amp free) running in them. If I want to go deep into a prompt then I’ll write a markdown file and iterate on it with a CLI.
    • kleiba 7mo ago
      I noticed that OpenCode requires per their own website "a modern terminal emulator" - so, no problem in Emacs? Are you running M-x term?
• prpl 7mo ago
  I am enjoying the RePPIT framework from Mihail Eric. I think it’s a better formalization of developing without resulting to personas.
• codeflo 7mo ago
  I know the argument I'm going to make is not original, but with every passing week, it's becoming more obvious that if the productivity claims were even half true, those "1000x" LLM shamans would have toppled the economy by now. Were are the slop-coded billion dollar IPOs? We should have one every other week.
  • wiseowise 7mo ago
    They’re busy writing applications for their dogs and building “jerk me off” functionality into their OpenClaw fork. Once they’re done you’ll be sorry you ever asked.
  • zingar 7mo ago
    Writing pieces of code that beat average human level is solved. Organizing that code is on its way to being solved (posts like this hint at it). Finding problems that people will pay money to have solved by software is a different entirely more complicated matter (tbh I doubt anyone could prove right now that this absolutely is or isn’t solvable - but given the change we’ve seen already I place no bets against AI). Also even if agents could do everything the societal obstacles to change are extensive (sometimes for very good, sometimes for bad reasons) so I’m expecting it to take another year or two serious change to occur.
    • ugtr3 7mo ago
      “ Finding problems that people will pay money to have solved by software is a different entirely more complicated matter ” Come on you’re taking the piss, surely.
  • user34283 7mo ago
    Last time I read about a Codex update, I think it mentioned that a million developers tried the tool. Don't most companies use AI in software development today? And yes, I know that some companies are not doing that because of privacy and reliability concerns or whatever. With many of them it's a bit of a funny argument considering even large banks managed to adopt agentic AI tools. Short of government and military kind of stuff, everybody can use it today.
• cpt_sobel 7mo ago
  In the plethora of all these articles that explain the process of building projects with LLMs, one thing I never understood it why the authors seem to write the prompts as if talking to a human that cares how good their grammar or syntax is, e.g.: > I'd like to add email support to this bot. Let's think through how we would do this. and I'm not not even talking about the usage of "please" or "thanks" (which this particular author doesn't seem to be doing). Is there any evidence that suggests the models do a better job if I write my prompt like this instead of "wanna add email support, think how to do this"? In my personal experience (mostly with Junie) I haven't seen any advantage of being "polite", for lack of a better word, and I feel like I'm saving on seconds and tokens :)
  • raincole 7mo ago
    Because some people like to be polite? Is it this hard to understand? Your hand-written prompts are unlikely to take significant chunk of context window anyway.
    • cpt_sobel 7mo ago
      Polite to whom?
      • qsera 7mo ago
        I think it is easier to be polite always and not switch between polite and non-polite mode depending on who you are talking to.
        • cpt_sobel 7mo ago
          I get what you're saying, but I'm not talking about swearing at the model or anything, I'm only implying that investing energy in formulating a syntactically nice sentence doesn't or shouldn't bring any value, and that I don't care if I hurt the model's feelings (it doesn't have any). Note, why would the author write "Email will arrive from a webhook, yes." instead of "yy webhook"? In the second case I wouldn't be impolite either, I might reply like this in an IM to a colleague I work with every day.
• sdevonoes 7mo ago
  Agent bots are the new “TODO” list apps. Seems cool and all, but I wish I could see someone writing useful software with LLMs, at least once. So much power in our hands, and soon another Facebook will appear built entirely by LLMs. What a fucking waste of time and money. It’s getting tiring.
• justboy1987 7mo ago
  [flagged]
• neonstatic 7mo ago
  > Before that, code would quickly devolve into unmaintainability after two or three days of programming, but now I’ve been working on a few projects for weeks non-stop, growing to tens of thousands of useful lines of code, with each change being as reliable as the first one. I'm glad it works for the author, I just don't believe that "each change being as reliable as the first one" is true. > I no longer need to know how to write code correctly at all, but it’s now massively more important to understand how to architect a system correctly, and how to make the right choices to make something usable. I agree that knowing the syntax is less important now, but I don't see how the latter claim has changed with the advent of LLMs at all? > On projects where I have no understanding of the underlying technology (e.g. mobile apps), the code still quickly becomes a mess of bad choices. However, on projects where I know the technologies used well (e.g. backend apps, though not necessarily in Python), this hasn’t happened yet, even at tens of thousands of SLoC. Most of that must be because the models are getting better, but I think that a lot of it is also because I’ve improved my way of working with the models. I think the author is contradicting himself here. Programs written by an LLM in a domain he is not knowledgable about are a mess. Programs written by an LLM in a domain he is knowledgeable about are not a mess. He claims the latter is mostly true because LLMs are so good??? My take after spending ~2 weeks working with Claude full time writing Rust: - Very good for language level concepts: syntax, how features work, how features compose, what the limitations are, correcting my wrong usage of all of the above, educating me on these things - Very good as an assistant to talk things through, point out gaps in the design, suggest different ways to architect a solution, suggest libraries etc. - Good at generating code, that looks great at the first glance, but has many unexplained assumptions and gaps - Despite lack of access to the compiler (Opus 4.6 via Web), most of the time code compiles or there are trivially fixable issues before it gets to compile - Has a hard to explain fixation on doing things a certain way, e.g. always wants to use panics on errors (panic!, unreachable!, .expect etc) or wants to do type erasure with Box<dyn Any> as if that was the most idiomatic and desirable way of doing things - I ended up getting some stuff done, but it was very frustrating and intellectually draining - The only way I see to get things done to a good standard is to continuously push the model to go deeper and deeper regarding very specific things. "Get x done" and variations of that idea will inevitably lead to stuff that looks nice, but doesn't work. So... imo it is a new generation compiler + code gen tool, that understands human language. It's pretty great and at the same time it tires me in ways I find hard to explain. If professional programming going forward would mean just talking to a model all day every day, I probably would look for other career options.
• lbreakjai 7mo ago
  It's interesting to see some patterns starting to emerge. Over time, I ended up with a similar workflow. Instead of using plan files within the repository, I'm using notion as the memory and source of truth. My "thinker" agent will ask questions, explore, and refine. It will write a feature page in notion, and split the implementation into tasks in a kanban board, for an "executor" to pick up, implement, and pass to a QA agent, which will either flag it or move it to human review. I really love it. All of our other documentation lives in notion, so I can easily reference and link business requirements. I also find it much easier to make sense of the steps by checking the tickets on the board rather than in a file. Reviewing is simpler too. I can pick the ticket in the human review column, read the requirements again, check the QA comments, and then look at the code. Had a lot of fun playing with it yesterday, and I shared it here: https://github.com/marcosloic/notion-agent-hive https://github.com/marcosloic/notion-agent-hive
  • Cthulhu_ 7mo ago
    No criticism or anything, but it really does feel / sound like you (and others who embraced LLMs and agentic coding) aspire to be more of a product manager than a coder. Thing is, a "real" PM comes with a lot more requirements and there's less demand for them - more requirements in that you need to be a people person and willing to spend at least half your time in meetings, and less demand because one PM will organize the work for half a dozen developers (minimum). Some people say LLM assisted coding will cost a lot of developers' jobs, but posts like this imply it'll cost (solve?) a lot of management / overhead too. Mind you I've always thought project managers are kinda wasteful, as a software developer I'd love for Someone Else to just curate a list of tasks and their requirements / acceptance criteria. But unfortunately that's not the reality and it's often up to the developers themselves to create the tasks and fill them in, then execute them. Which of course begs the question, why do we still have a PM? (the above is anecdotal and not a universal experience I'm sure. I hope.)
    • lbreakjai 7mo ago
      I worked with some excellent PMs in the past, it's an entirely different skillset. This wasn't really meant to replace what they do. I really wanted something with which to work at feature-level. That is, after all the hard work of figuring out _what_ to build has been done. > as a software developer I'd love for Someone Else to just curate a list of tasks and their requirements / acceptance criteria That's interesting. In every team I worked in, I always fought really hard against anyone but developers being able to write tickets on the board.
• oytis 7mo ago
  I find the same problem applying to coding too. Even with everyone acting in good faith and reviewing everything themselves before pushing, you have essentially two reviwers instead of a writer and a reviewer, and there is no etiquette mandating how thoroughly the "author" should review their PR yet. It doesn't help if the amount of code to review gets larger (why would you go into agentic coding otherwise?)
• takwatanabe 7mo ago
  We build and run a multi-agent system. Today Cursor won. For a log analysis task — Cursor: 5 minutes. Our pipeline: 30 minutes. Still a case for it: 1. Isolated contexts per role (CS vs. engineering) — agents don't bleed into each other 2. Hard permission boundaries per agent 3. Local models (Qwen) for cheap routine tasks Multi-agent loses at debugging. But the structure has value.
• danbruc 7mo ago
  I randomly clicked and scrolled through the source code of Stavrobot - The largest thing I’ve built lately is an alternative to OpenClaw that focuses on security. [1] and that is not great code. I have not used any AI to write code yet but considered trying it out - is this the kind of code I should expect? Or maybe the other way around, has someone an example of some non-trivial code - in size and complexity - written by an AI - without babysitting - and the code being really good? [1] https://github.com/skorokithakis/stavrobot https://github.com/skorokithakis/stavrobot
  • mbesto 7mo ago
    > and that is not great code When you say "is not great code" can you elaborate? Does the code work or not?
    • danbruc 7mo ago
      I don't know, I would assume it works but I would not expect it to be free of bugs. But that is the baseline for code, being correct - up to some bugs - is the absolute minimum requirement, code quality starts from there - is it efficient, is it secure, is it understandable, is it maintainable, ...
      • mbesto 7mo ago
        So do you expect it not to be free of bugs because you've run a comprehensive test on it, read all of the code yourself or are you just concluding that because you know it was generated by an LLM?
        • danbruc 7mo ago
          It has not been formally verified which is essentially the only way to achieve code without defects with reasonable confidence. There are several studies that have found that there are roughly between one and twenty bugs per thousand lines of code in any software, this project has several thousand lines of code, so I would expect several bugs if written by humans and I have no reason to assume that large language models outperform humans in this respect, not at last because they are trained on code written by humans and have been trained to generate code as written by humans.
• miguelgrinberg 7mo ago
  > One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. It's always easier to blame the prompt and convince yourself that you have some sort of talent in how you talk to LLMs that other's don't. In my experience the differences are mostly in how the code produced by the LLM is reviewed. Developers who have experience reviewing code are more likely to find problems immediately and complain they aren't getting great results without a lot of hand holding. And those who rarely or never reviewed code from other developers are invariably going to miss stuff and rate the output they get higher.
  • or_am_i 7mo ago
    It's always easier to blame the model and convince yourself that you have some sort of talent in reviewing LLM's work that others don't. In my experience the differences are mostly in how the code produced by LLM is prompted and what context is given to the agent. Developers who have experience delegating their work are more likely to prevent downstream problems from happening immediately and complain their colleagues cannot prompt as efficiently without a lot of hand holding. And those who rarely or never delegated their work are invariably going to miss crucial context details and rate the output they get lower.
    • loloquwowndueo 7mo ago
      Never takes long for the “you’re holding it wrong” crowd to pop in.
      • darkerside 7mo ago
        That's a terrible reason for a mass consumer tool to fail, and a perfectly reasonable one for a professional power tool to fail
    • hellosimon 7mo ago
      Partly true, but I think there's a real skill in catching subtle logic errors in generated code too not just prompting well. Both matter.
• peterweisz 7mo ago
  Great article. I'd recommmend to make guardrails and benchmarking an integral part of prompt engineering. Think of it as kind of a system prompt to your Opus 4.6 architect: LangChain, RAG, LLm-as-a-judge, MCP. When I think about benchmarks I always ask it to research for external DB or other ressources as a referencing guardrail
• zingar 7mo ago
  Big +1 for opencode which for my purposes is interchangeable or better than Claude and can even use anthropic models via my GitHub copilot pro plan. I use it and Claude when one or the other hits token limits. Edit: a comment below reminded me why I prefer opencode: a few pages in on a Claude session and it’s scrolling through the entire conversation history on every output character. No such problem on OC.
• chuckauto 7mo ago
  [dead]
• diven_rastdus 7mo ago
  [dead]
• openclaw01 7mo ago
  [dead]
• zingar 7mo ago
  On using different models: GitHub copilot has an API that gives you access to many different models from many different providers. They are very transparent about how they use your data[1]; in some cases it’s safer to use a model through them than through the original provider. You can point Claude at the copilot models with some hackery[2] and opencode supports copilot models out of the box. Finally, copilot is quite generous with the amount of usage you get from a Github pro plan (goes really far with Sonnet 4.6 which feels pretty close to Opus 4.5), and they’re generous with their free pro licenses for open source etc. Despite having stuck to autocomplete as their main feature for too long, this aspect of their service is outstanding. [1]: https://docs.github.com/en/copilot/reference/ai-models/model-hosting https://docs.github.com/en/copilot/reference/ai-models/model... [2]: https://github.com/ericc-ch/copilot-api https://github.com/ericc-ch/copilot-api
• gehsty 7mo ago
  When I use Claude code to work on a hobby project it feels like doom scrolling… I can’t get my head around if the hobby is the making or the having, but fair to say I’ve felt quite dissatisfied at the end of my hobby sessions lately so leaning towards the former.
  • Levitating 7mo ago
    Agreed, I code for fun. But I am not sure if I still find it fun if the LLM just makes what I want.
• sara_builds 7mo ago
  [dead]
• Alvarito1983 7mo ago
  [dead]
• ilovetux 7mo ago
  I like the approach outlined in the article. These days having a roadmap for yourself while cruising at highway speeds helps make sense of the chaos. One big pain point that has existed forever and has never really been addresses adequately is the ability to come up with requirements. Sure, it sounds easy, I need the app to do x, y and z. But requirements change in real time because of lack of foresight, change of business needs, an unexpected roadblock and more contribute to changing requirements. So, the advice to come up with the requirements by yourself or with the LLM miss the biggest pain point. I'd like to see a resurgence of flow charts, IPO (Input, Processing and Output) charts and other tools to organize requirements spring up to help with really nailing down requirements. I will say, though, some of the pain is relieved because the agent can perform a huge refactor in a couple of minutes, but that opens a whole new can of worms.
• ryan14975 7mo ago
  [dead]
• mkovach 7mo ago
  What hurts the most is that the em dash used to be a small, rebellious literary act that I truly enjoyed employing. A simple, useful hinge in a sentence where it could change its mind. Now? It indicates when an LLM got too frisky with clause boundaries and maintains a phobia of semicolons.
• rednafi 7mo ago
  The world could use one less "how I slop" article at this point. This reminds me of the early Medium days when everyone would write articles on how to make HTTP endpoints or how to use Pandas. There’s not much skill involved in hauling agents, and you can still do it without losing your expertise in the stuff you actually like to work with. For me, I work with these tools all the time, and reading these articles hasn’t added anything to my repertoire so far. It gives me the feeling of "bikeshedding about tools instead of actually building something useful with them." We are collectively addicted to making software that no one wants to use. Even I don’t consistently use half the junk I built with these tools. Another thing is that everyone yapping about how great AI is isn’t actually showing the tools’ capabilities in building greenfield stuff. In reality, we have to do a lot more brownfield work that’s super boring, and AI isn’t as effective there.
• elophanto_agent 7mo ago
  [dead]
• edlebert_f 7mo ago
  [flagged]
• entrustai 7mo ago
  [dead]
• cousin_it 7mo ago
  I'm thinking more and more that there's an ethical problem with using LLMs for programming. You might be reusing someone's GPL code with the license washed off. It's especially worrisome if the results end up in a closed product, competing with the open source project and making more money than it. Of course neither you nor the AI companies will face any consequence, the government is all-in and won't let you be hurt. But ethically, people need to start asking themselves some questions. For me personally, in my projects there's not a single line of LLM code. At most I ask LLMs for advice about specific APIs. And the more I think about it, the more I want to stop doing even that.
  • grzesiaka 7mo ago
    The same here. I find big AI-corpos pretty evil and drastically misaligned with broader well-being of the society.
  • 3form 7mo ago
    I would also add: if you're paying, supporting their cause with your money. Sometimes I would like to have magical make-my-project tool for my selfish reasons; sometimes I know it would be a bad choice to fall behind on what's to come. But I really, really don't want to support that future.
• frig57 7mo ago
  Stavbot eh? Fava beans https://youtu.be/gyy-RcI6pxE?si=e0QPg3jWvwDojKSP https://youtu.be/gyy-RcI6pxE?si=e0QPg3jWvwDojKSP
• shell0x 7mo ago
  Isn't that what Amp code essentially does? I've used Codex and Claude Max but I keep going back to Amp https://ampcode.com/models https://ampcode.com/models It uses different models for different modes. I just find it to be faster and it often gets things right at the first attempt, but YMMV.
• OtomotO 7mo ago
  > One thing I’ve noticed is that different people get wildly different results with LLMs, so I suspect there’s some element of how you’re talking to them that affects the results. Which is Fortuna's work... stochastic models are like that. And confirmation bias is another phenomenon as well as "how do LLMs align with my worldview" whether I see them more positively or more negatively.
• pathikrit 7mo ago
  10-15 years ago HN was the place where everyone showed off their new shiny toys and people would jump over themselves to try some new framework or db or tool. Now, so much negative sentiment about AI coding. I bet if LLMs came 15 years ago, HN would be brimming with excitement. What happened?
  • deleted 7mo ago
    [deleted]
  • timdellinger 7mo ago
    The difference is that CEOs fired a lot of people, using "AI can replace human coders" as an excuse. Also: there are claims all over the headlines along the lines of "We built something amazing without human coders." Both claims are loud and are flooding the discussion, but under the hood it's mostly a slop disaster. So the negative sentiment is a natural response (and a dose of realism).
    • kody 7mo ago
      Mix in exhaustion from previous hype cycles (NFTs?) and folks pushing the narrative that if you currently write any code manually, you’re obsolete and it’s hard to not want to push back negatively. Even if you find the tech interesting/useful.
  • dncornholio 7mo ago
    You should be critical. If you made something, you should proof that it works. Especially in todays world. This article contains no proof that their work actually works.
    • stavros 7mo ago
      Except that all of it is open source and you can go try it. So this is basically me saying "I made a really nice cake, here, try some" and you're plugging your ears and going "lalalala can't prove this cake is good, lalalala it's shit".
• fedeb95 7mo ago
  This is interesting and goes beyond the usual AI hype. It's the beginning of a structured and efficient use of new tools (aka software engineering).
• bhekanik 7mo ago
  [dead]
• bigblind 7mo ago
  > On projects where I have no understanding of the underlying technology (e.g. mobile apps), the code still quickly becomes a mess of bad choices. However, on projects where I know the technologies used well (e.g. backend apps, though not necessarily in Python), this hasn’t happened yet, even at tens of thousands of SLoC. Most of that must be because the models are getting better, but I think that a lot of it is also because I’ve improved my way of working with the models. I wonder whether at some point we'll get a translation model, that translates relatively vague requests into sound architectural decisions, with some embedded knowledge of the environment you're building in, and that can ask clarifying questions when there are multiple options with different tradeoffs.
  • mlnj 7mo ago
    Is that not already possible with Markdown spec files and planning mode?
    • bigblind 7mo ago
      I guess? At least there you can review the plan, but is this planning mode any better at making architectural decisions than when you prompt an LLM and let it make the changes directly? (it might be, just not sure.)
• mk_chan 7mo ago
  In my experience an LLM does 2 things: 1. Bring you up to some average-LLM level when you don’t have the skills/knowledge to actually do what you want. 2. Work at 80-90% of your capacity but WAY faster than you physically could depending on how much context you provide it. If you don’t provide it sufficient context to do what/how you want it to do, of course it might default to something you don’t want.
• benterris 7mo ago
  One thing I don't get with this workflow, and all the ones we see in similar articles: do the authors run their agents in YOLO mode (full unchecked permission on their machine)? It seems their agents have full edit rights (scoped to a directory, which seems reasonable), but can also run tests autonomously (which means they can run any code), which equates to full read/write access on the machine? I mean, there are ways to sandbox agents in dedicated containers, but it requires quite a bit of setup, and none of these articles mention it, so I guess they are YOLOing it?
  • neobrain 7mo ago
    Claude has a sandbox mode that uses bubblewrap to build a lightweight filesystem sandbox that only exposes the project directory: https://code.claude.com/docs/en/sandboxing https://code.claude.com/docs/en/sandboxing It's disabled by default though, and in general (especially with other agents) you very much still have to get out of your way to get any sort of reasonable access control indeed. In principle though, just running the agent CLI in something like firejail would get you very far if you know what you're doing.
• jbergqvist 7mo ago
  I've found that spending most of my time on design before any code gets written makes the biggest difference. The way I think about it: the model has a probability distribution over all possible implementations, shaped by its training data. Given a vague prompt, that distribution is wide and you're likely to get something generic. As you iterate on a design with the model (really just refining the context), the distribution narrows towards a subset of implementations. By the time the model writes code, you've constrained the space enough that most of what it produces is actually what you want.
  • ugtr3 7mo ago
    Yea but that design part is the most expensive part. The code generation is pretty trivial - the advantage of llm’s is the power to search through pre-trained information spaces - much faster than a human could. Issue is again.. probabilistic. So there’s variance.’
• hirehalai 7mo ago
  [dead]
• gwbas1c 7mo ago
  > On the flip side, when I’m not familiar enough with the technology to be on top of the architecture, I tend to not catch bad decisions that the LLM makes. This leads to the LLM building more and more on top of those bad decisions, eventually getting in a state where it can’t untangle the mess. You know this happens when you keep telling the LLM the code doesn’t work, it says “I know why! Let me fix it” and keeps breaking things more and more. That exact thing happens with people too! Specifically when a cheap entrepreneur hires a novice developer and can't give the developer appropriate mentoring and reviews.
  • stavros 7mo ago
    That's actually very true, I didn't realize. Huh, interesting, thanks.
• A7OM 7mo ago
  [dead]
• leontloveless 7mo ago
  [flagged]
• highfrequency 7mo ago
  > I’ll tell the LLM my main goal (which will be a very specific feature or bugfix e.g. “I want to add retries with exponential backoff to Stavrobot so that it can retry if the LLM provider is down”), and talk to it until I’m sure it understands what I want. This step takes the most time, sometimes even up to half an hour of back-and-forth until we finalize all the goals, limitations, and tradeoffs of the approach, and agree on what the end architecture should look like. This sounds sensible, but also makes me wonder how much time is actually being saved if implementing a "very specific feature or bugfix" still takes an hour of back and forth with an LLM. Can't help but think that this is still just an awkward intermediate phase of development with adolescent LLMs where we need to think about implementation choices at all.
  • stavros 7mo ago
    Small features or bugfixes generally take a minute or two of conversation.
• devlinMckhay 7mo ago
  the failure mode section is the most honest thing i have read about this whole thing. I hit that exact wall building something evenings after work. you miss one bad architectural decision because you are tired or in a hurry, and three sessions later the llm is confidently making it worse and you are not even sure when it started going wrong. the only thing that helped was slowing down on the planning side even when i did not feel like i had time for it.
• agenticbtcio 7mo ago
  [dead]
• thomasBln 7mo ago
  [dead]
• mihir_kanzariya 7mo ago
  [flagged]
• zihotki 7mo ago
  Just like with many other submissions, I see a great I-shaped senior developer with a developed gut feeling who's able to do big chunks of work. I wonder how the team members, if any, survive such throughput. I also wonder if there was any quantification applied for the prompts/results, cost analysis, etc.
• kul_ 7mo ago
  LLMs are great at aggregating docs, blogs and other sources out there into a single interface and there has been nothing like it before. When it comes to coding however, the place where you really need help is the place where you get stuck and that for most people would be the intersection of domain and tech. LLMs need a LOT of baby sitting to be somewhat useful here. If I have to prompt a LLM for hours just to get the correct code, why would I even use it when the tangible output is just carefully thought out few 100 lines of code!
• ossianericson 7mo ago
  [flagged]
• anvevoice 7mo ago
  [dead]
• leontloveless 7mo ago
  [flagged]
• denzen 7mo ago
  > Pine Town is a whimsical infinite multiplayer canvas of a meadow, where you get your own little plot of land to draw on. Most people draw… questionable content Doesn't help that _pine_ is one way of saying penis in french
• vicchenai 7mo ago
  [dead]
• andai 7mo ago
  > Since LLMs have become good at programming, I’ve been using them to make stuff nonstop, and it’s very exciting that we’re at the beginning of yet another entirely unexplored frontier. Making software? It sounds funny but I've heard an interesting argument along those lines. The reason software is slow and bloated and kind of unreliable is because... It can be. There's so little competition. If there was actual competition, then there would be pressure to make it not be shit. But apparently no such pressure exists.
• clawbridge 7mo ago
  Pine town is an interesting idea
• IanCal 7mo ago
  > I’m planning to write something about this too, but this one is more of an art piece: It’s a ticking wall clock that ticks seconds irregularly, but is always accurate to the minute (with its time getting synced over the internet). It has various modes, one mode has variable tick timing, from 500 ms to 1500 ms, which is delightfully infuriating. Another mode ticks imperceptibly more quickly than a second, but then pauses for a second randomly, making the unsuspecting observer question their sanity. Another one races to :59 at double speed and then waits there for thirty seconds, and the last one is simply a normal clock, because all the irregular ticking drives me crazy. I like this, I've always had a soft spot for interesting clocks, and the first description has a strong nostalgia for me from the Discworld: > Someone very clever—certainly someone much cleverer than whoever had trained that imp—must have made the clock for the Patrician’s waiting room. It went tick-tock like any other clock. But somehow, and against all usual horological practice, the tick and the tock were irregular. Tick tock tick…and then the merest fraction of a second longer before…tock tick tock…and then a tick a fraction of a second earlier than the mind’s ear was now prepared for. The effect was enough, after ten minutes, to reduce the thinking processes of even the best-prepared to a sort of porridge. The Patrician must have paid the clockmaker quite highly.
• tliptay 7mo ago
  I've seen something similar with forecasting. I can get more accurate forecasts by having distinct agents do different research tasks, summarize, and then feed those summaries to an agent that uses judgment to forecast. I think it is all about managing context.
• freshtake 7mo ago
  "I try to understand things as much as I can while planning, even if I’m unfamiliar with the specific technology. If I manage to steer the LLM well, it saves a lot of trouble later on." This is the most important part of the article. Key takeaway.