13 ms·

I'm really sorry to hear this, because part of my goal here is to help push back against the idea that "programming skills are useless now, anyone can get an LL

by simonw 1y ago

I'm really sorry to hear this, because part of my goal here is to help push back against the idea that "programming skills are useless now, anyone can get an LLM to write code for them". I think existing software development skills get a whole lot more valuable with the addition of coding agents. You can take everything you've learned up to this point and accelerate the impact you can have with this new family of tools. I said a version of this in the post: > AI tools amplify existing expertise. The more skills and experience you have as a software engineer the faster and better the results you can get from working with LLMs and coding agents. A brand new vibe coder may be able to get a cool UI out of ChatGPT, but they're not going to be able to rig up a set of automated tests with continuous integration and continuous deployment to a Kubernetes cluster somewhere. They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for.

jmspring 1y ago
I need to read through this some more, but there has been another genetic coding paradigm referred to as spec driven development. I’ll find the link in the morning, but I kinda joke - it’s vibe coding for people who know how to define a problem and iterate on it. I’ve got a project reimplementing a service I want to make more uniform. Claude has produced a lot of stuff that would have taken me weeks to do.
- senko 1y ago
  GitHub's SpecKit is an example: https://github.com/github/spec-kit https://github.com/github/spec-kit Spec-Driven Development treats the spec as the source of truth and the code as an artifact. As you develop, you modify/add to the spec and the codebase gets updated to reflect it. Personally I'm doubtful it can compete with traditional artisanal software engineering, as it's (IMHO) boils down to "if only you can spec it precisely enough, it'll work" and we've tried this with 5GL and (to some extent) BDD, and it doesn't get you to 100%. I do think it's interesting enough to explore, and most of us could use a bit more details in our Jira tickets.
  - piva00 1y ago
    That was exactly what UML wanted to do, and it almost never worked out in practice. Seems to be just a rehashing of the same idea but instead of XML, and diagrams, it's now some free-text to be interpreted by LLMs, so much less deterministic and will probably fail just like UML failed. People also tend to forget about Peter Naur's take on "Programming as Theory Building" [0], the program is, in itself, the theory of what's implemented. A spec cannot replace that. [0] https://pages.cs.wisc.edu/~remzi/Naur.pdf https://pages.cs.wisc.edu/~remzi/Naur.pdf
    - andrewgleave 1y ago
      Theory building is the secret sauce, and all variants of "this is how to use AI effectively" I've seen are inferior to the epistemologically sound theory Naur outlines in his paper.
  - WOTERMEON 1y ago
    Hey! No J word tolerated in here!
  - Kzozjj 1y ago
    We already invented languages for succinctly describing what the computer should do. They’re call programming languages. “The code is the documentation” is not a joke. Logic that’s useful in the real world is complex and messy. You need additional documentation (why did the code end up like it is, etc) but code is the most expressive way we’ve got for describing how a computer should work.
senko 1y ago
> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for. I wonder what the practical limits are. As a senior dev on a greenfield solo project it's too exhausting for me to have two parallel agents (front/back), most of the time they're waiting for me to spec, review or do acceptance test. Feels like sprinting, not something I could do day in and day out. Might be due to tasks being too fine grained, but assuming larger ones are proportionally longer to spec and review, I don't see more than two (or, okay, three, maybe I'm just slow) being a realistic scenario. More than that, I think we're firmly in the vibe coding (or maybe spec-driven vibe coding) territory.
- afro88 1y ago
  At least on a team, the limit is the team's time to review all the code. We've also found that vibe engineered (or "supervised vibing" as I call it) code tends to have more issues in code review because of a false sense of security creating blind spots when self reviewing. Even more burden on the team. We're experimenting with code review prompts and sub agents. Seems local reviews are best, so the bulk of the burden is on the vibing engineer, rather than the team.
  - bartread 1y ago
    Do you have a sense for how much overhead this is all adding? Or, to put it another way, what I’m really asking is what productivity gain (or loss) are you seeing versus traditional engineering?
    - rhetocj23 1y ago
      Very curious to hear responses about this too
      - oblio 1y ago
        The problem with this is that software engineering is a very unorganized and fashion/emotion driven domain. We don't have reliable productivity numbers for basically... anything. I <feel> that I'm more productive with statically typed languages but I haven't seen large scale, reliable studies. Same with unit tests, integration tests, etc. And then there are all the types of software engineering: web frontend, web API, mobile frontend, command line frontend, Windows GUI, MacOS GUI, Linux backend (10 million different stacks), Windows backend (1 million different stacks), throwaway projects, WordPress webpages, etc, etc.
lemming 1y ago
While this is true, I definitely find that the style of the work changes a lot. It becomes much more managerial, and less technical. I feel much more like a mix of project and people manager, but without the people. I feel like the jury is still out on whether I’m overall more productive, but I do feel like I have less fun.
- benterix 1y ago
  My lessons so far: 1. Less fun. 2. A lot of more "review fatigue". 3. Tons of excess code I'd never put in there in the first place. 4. Frustration with agents being too optimistic which with time verges on the ludicurous ("Task #3 has been completed successfully with 98% tests failing. [:useless_emojis:]") 5. Frustration with agents routinely getting down a rabbit hole or going in circles, the effort needed to get that straight (Anthropic plainly advises to start from scratch in such cases - which is sound advice, but makes me feel like I just lost the last 5 hours of my life without even learning anything new). I stopped using agents and use LLMs very sparingly (e.g. for review - they sometimes find some details I missed and occasionally have an interesting solution) but I'm enjoying my work so much more without them.
  - exe34 1y ago
    98% tests sounds really great though, give that LLM a raise!
  - resonious 1y ago
    I think one of the tricks is to just stop using the agent as soon as you see signs of funny business. If it starts BSing me with failing tests, I just turn it off immediately and git reset (maybe after taking a quick peek)
    - Sammi 1y ago
      Yeah I make maybe two or three attempts at getting it to write a plan that it is able to follow coherently. But after that I pull the escape hatch and *gasp* program by hand. I've made this mistake of doubling down after a few initial failures to solve an issue, by trying to make this super duper comprehensive and highly detailed and awesome plan that it will finally be able to implement correctly. But it just gets worse and worse the more I try, because it fundamentally is not understanding what is going on, so it will inevitably find an opportunity to go massively off rails, and the further down you lead it the more impressible the derailment will be. My experience is that going around in endless circles with the model is just a waste of time when you could have just done it yourself in the time you've wasted.
deanCommie 1y ago
I don't think OP thinks his skills are useless per se now, but that the way to apply those skills now feels less fun and enjoyable. Which makes perfect sense - even putting aside the dopamine benefits of getting into a coding flow state. Coding is craftsmanship - in some cases artistry. You're describing Vibe Engineering as management. And sure, a great manager can make more of an impact increasing the productivity of an entire team than a great coder can make by themselves. And sure, some of the best managers are begrudging engineers who stepped up when needed to and never stepped down. But most coders still don't want to be managers - and it's not from a lack of skill or interest in people - it's just not what they chose. LLM-based vibe coding and engineering is turning the creative craftsmanship work of coding into technical middle management. Even if the result is more "productivity", it's a bit sad.
- ako 1y ago
  But does anybody really care about what you like? What about all those other professions that got replaced by technology, did anybody care what they liked? The big question is how is software going to be build most efficiently and most effectively in the future and how do you prepare yourself for this new world. Otherwise you’ll end up with all those other professions that got replaced, like the mineworkers, hoping that the good old days will someday return.
  - zwnow 1y ago
    Its reasonable to stay away from something one considers dystopian considering the industry is not even sure about the usefulness of coding agents in professional environments. When the tractors replaced the horses, everyone could agree they outperform horses. The result was easily measurable. Its not that simple with LLM agents owned by big corporations.
    - ako 1y ago
      Sure, it's not yet clear what impact LLMs will have on software development, but the impact it will have will not depend on if developers like to use it or not. If it is going to make software development 10x faster, companies will adopt it, whether devs like it or not.
saulpw 1y ago
I appreciate what you're trying to do, but for myself, I'm not depressed because my skills are less valuable. I enjoyed the money but it was never about that for me. I'm depressed because I don't like the way this new coding feels in my brain. My focus and attention are my most precious resources and vibe coding just shatters them. I want to be absorbed in a coding flow where I see all the levels of the system and can elegantly bend the system to my will. Instead I'm stuck reviewing someone/something else's code which is always a grind, never a flow. And I can feel something terrible happening in my brain, which at best can be described as demotivation, and at worst just utter disinterest. It's like if I were a gardener and I enjoyed touching dirt and singing to plants, and you're here on Gardener News extolling the virtues of these newfangled tractors and saying they'll accelerate my impact as a gardener. But they're so loud and unpleasant and frankly grotesque and even if I refrain from using one myself, all my neighbors are using them and are producing all their own vegetables, so they don't even care to trade produce anymore--with me or anyone else. So I look out at my garden with sadness, when it gave me such joy for so many decades, and try to figure out where I should move so I can at least avoid the fumes from all the tractors.
- shafyy 1y ago
  How beautifully put, and I couldn't agree more. I feel exactly the same way. However, I am still unconvinced that software development will go down this way. But if woking as a software developer will require managing multiple agents at the same time instead of crafting your own code — you can count me out, too.
- sally_glance 1y ago
  Well said! Reading this I feel reminded of the early protests against industrialization and automation in other fields. Checks all the same boxes - insecurity and fear about the future, alienation towards the new tools, ... Not saying AI is similar in impact to the loom or something, it just occured to me how close this is to early Luddite texts.
  - bonoboTP 1y ago
    The OG alienator was the Agricultural Revolution, settling and toiling repetitively in predetermined ways, unlike the more adventurous lifestyle from before with all the camping, hunting, gathering, where circumstances brought always novel challenges, you could be a man spearing a deer, instead of just killing some docile domesticated cow. Searching for pheasant eggs and being happy if you found some, instead of going out every morning to the predictable presence of eggs in the chicken coop.
zwnow 1y ago
What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents. It used to be a field where you could get a laptop from the scrapyard and learn from there. It feels pointless. Also agents do not invent things, the creativity part is gone with them. They simply use what they've already seen, repeat the same mistakes a person made a few years ago. Its a dystopian way of working. Sure it enables one to spew out slop that might make companies money, but there is no passion, sense of exploration, personal growth. Its all just directing parrots with thumbs...
- bananapub 1y ago
  > What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents. what do you actually mean by this? it's clearly untrue - anyone get get a laptop and install linux on it and start bashing out code today, just as they could last week and last year and thirty years ago. do you mean that you think at some point in the future tooling for humans to write code won't exist? or that employers won't hire human programmers? or that your pride is hurt? or you want your hobby to also be a well-paid job? or something else?
  - zwnow 1y ago
    I mean that this "tooling" becomes inaccessible to people. At least the tooling that is relevant for jobs. Employers will eventually stop hiring human based on their programming competence. It'll translate into a low pay career for people who like to orchestrate agents.
    - flir 1y ago
      I think it's just another floor in the creaky old tower of abstraction. Machine code > ASM > 3GLs > 4GLs > visual programming > LLMs etc etc etc. Thing is, the moment you go off-piste, the LLMs get a lot less useful. I think, if you want to stay closer to the metal, you've got to aim for a niche that has a small internet footprint. So... domain knowledge or esoteric programming knowledge. One way to incorporate domain knowledge might be to become a hybrid product owner/programmer. (This is all just opinion - I'm sure a well-argued rebuttal is possible).
HSO 1y ago
Have you tried SolveIt (method, tool) from Jeremy Howard yet? I was in the first batch last year where they introduced it and going to do the second one too. It´s a very different kind of beast to what is currently being discussed.
- raducu 1y ago
  > going to do the second one too. I missed the first one, when will the second one be?
  - ayewo 1y ago
    "Signups are open [1], and will remain so until October 20th." Recently on HN [2]. [1]: https://solve.it.com/ https://solve.it.com/ [2]: https://news.ycombinator.com/item?id=45455719 https://news.ycombinator.com/item?id=45455719
exe34 1y ago
> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for. Neither can I, sadly. I have one brain cell and I can only really do one thing at a time. Doing more than one leads to a corrupted stack and I make exponentially more mistakes.
codr7 1y ago
I still haven't seen any evidence to match these repeated claims of increased efficiency. What I have seen is reports that makes a lot of sense to me claiming it's all in the user's head.
- Cthulhu_ 1y ago
  I'm working in corporate and haven't seen it yet. The main thing I see is blogs and whatnot of people building new weekend projects with LLMs, that is, greenfield, non-critical software - the type of software that, if I were to write it, I wouldn't bother with CI, tests, that kind of thing with. Sloppy projects, if you will. But happy to be corrected - is someone using these agents in their paid / professional / enterprise / team job?
  - rhetocj23 1y ago
    I’ve asked this many times on here - I never get a coherent answer
  - girvo 1y ago
    I am, but in a very narrow focus: mostly examining our existing codebase as a more powerful but fuzzier search, and a system to then generate a plan to implement and approach which I tweak. I sometimes use it to scaffold out some boilerplate for tests, but never tests themselves: no matter what I try it always ends up writing the useless straight-jacket "change alert" style tests that break on any change to the unit under test, which I despise.
  - Aeolun 1y ago
    I think most of the code in our enterprise is now written by AI. It’s all boring callcenter crud apps, so nobody is really sad they’re not writing any of that code any more. I’m not sure it makes me faster, but I think QA testing what the AI made and occassionaly adjusting it is more fun anyway. The code is absolutely lower quality, but there were always so many people producing garbage faster than I could produce something nice that the code was always terrible anyway. There’s an element of wanting to know how the thing works so at least I’ll know when it’s ready to replace me though.
llamatastic 1y ago
I really don't get the idea that LLMs somehow create value. They are burning value. We only get useful work out of them because they consume past work. They are wasteful and only useful in a very contrived context. They don't turn electricity and prompts into work, they turn electricity, prompts AND past work into lesser work. How can anyone intellectually honest not see that? Same as burning fossil fuels is great and all except we're just burning past biomass and skewing the atmosphere contents dangerously in the process.
- simonw 1y ago
  > How can anyone intellectually honest not see that? The idea that they can only solve problems that they've seen before in their training data is one of these things that seems obviously true, but doesn't hold up once you consistently use them to solve new problems over time. If you won't accept my anecdotal stories about this, consider the fact that both Gemini and OpenAI got gold medal level performance in two extremely well regarded academic competitions this year: the International Math Olympiad (IMO) and the International Collegiate Programming Contest (ICPC). This is notable because both of those contests have brand new challenges created for them that have never been published before. They cannot be in the training data already!
  - throw219080123 1y ago
    > consider the fact that both Gemini and OpenAI got gold medal level performance Yet ChatGPT 5 imagines API functions that are not there and cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub.
    - simonw 1y ago
      Which is why you run it in a coding agent loop using something like Codex CLI - then it doesn't matter if it imagines a non-existent function because it will correct itself when it tries to run the code. Can you expand on "cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub"? I have it do that all the time and it works really well for me (at least with modern "reasoning" models like GPT-5 and Claude 4.)
MintsJohn 1y ago
Of course the devil is in the details. What you say and the skills needed make sense. It's unfortunately also the easiest aspects to dismiss either under pressure as there is often little immediate payoff, or because it's simply the hard part. My experience with llms in general is that sadly, they're mostly good bullshitters. (current google search is the epitome of worthlessness, the AI summary so hard tries to make things balanced, that it just dreams up and exaggerates pros en cons for most queries). In a same way platforms like perplexity are worthless, they seem utterly unable to assign the proper value to sources they gather. Of course that doesn't stop me from using llms where they're useful; it's nice to be able to give the architecture for a solution and let the llm fill the gaps than to code the entire thing by hand. And code-completion in general is a beautiful thing (sadly not a thing where much focus is on these days, most is on getting the llm create complete solutions while i would be delighted by even better code completion) Still all in all, the more i see llms used (or the more i see (what i assume) well willing people copy/paste llm generated responses in favor of handwritten responses) on so much of the internet, resulting in a huge decline of factualness and reproducibility (in he sense, that original sources get obscured), but an increase of nice full sntences and proper grammar, the more i'm inclined to belief that in the foreseeable future llm's aren't a net positive. (in a way it's also a perfect storm, the last decade education unprioritised teaching skills that would matter especially for dealing with AI and started to educate for use of tools instead of educate general principles. The product of education became labourers for a specific job instead of higher abstract level reasoning in a general area of expertise)
- simonw 1y ago
  Google's "AI overviews" are one of the worst LLM-powered features on the market today, they're genuinely damaging the reputation of the whole industry. Meanwhile I've started using ChatGPT GPT-5 search as my default search engine! A year ago I would have laugher at the idea: https://simonwillison.net/2025/Sep/6/research-goblin/ https://simonwillison.net/2025/Sep/6/research-goblin/ And Google themselves have an "AI mode" which is a different league of quality from "AI overviews", I wrote about that one here: https://simonwillison.net/2025/Sep/7/ai-mode/ https://simonwillison.net/2025/Sep/7/ai-mode/ This is new. AI search tools almost universally sucked until OpenAI's release of o3 in April this year.
justbees 1y ago
I'm getting really great results in a VERY old (very large) codebase by having discussion with the LLM (I'm using Claude code) and making detailed roadmaps for new features or converting old features to new more useable/modern code. This means FE and BE changes usually at the same time. I think a lot of the points you make are exactly what I'm trying to do. - start with a detailed roadmap (created by the ai from a prompt and written to a file) - discuss/adjust the roadmap and give more details where needed - analyze existing features for coding style/patterns, reusable code, existing endpoints etc. (write this to a file as well) - adjust that as needed for the new feature/converted feature - did it miss something? Is there some specific way this needs to be done it couldn't have known? - step through the roadmap and give feedback at each step (I may need to step in and make changes - I may realize we missed a step, or that there's some funky thing we need to do specifically for this codebase that I forgot about - let the LLM know what the changes are and make sure it understands why those changes were made so it won't repeat bad patterns. i.e. write the change to the .md files to document the update) - write tests to make sure everything was covered... etc etc Basically all the things you would normally WANT do but often aren't given enough time to do. Or the things you would need to do to get a new dev up to speed on a project and then give feedback on their code. I know I've been accomplishing a lot more than I could do on my own. It really is like managing another dev or maybe like pair programming? Walk through the problem, decide on a solution, iterate over that solution until you're happy with the decided path - but all of that can take ~20 minutes as opposed to hours of meetings. And the end result is factors of time less than if I was doing it on my own. I recently did a task that was allotted 40 hours in less than 2 working days - so probably close to 10-12 hours after adjusting for meetings and other workday blah blah blah. And the 40 hour allotment wasn't padded. It was a big task, but doing the roadmap > detailed structure including directory structure - what should be in each file etc etc cut the time down dramatically. I would NOT be able to do this if I the human didn't understand the code extremely well and didn't make a detailed plan. We'd just end up with more bad code or bad & non-working code.
- hodgesrm 1y ago
  Thank you for this post. I don't write much code as I'm currently mostly managing people but I read it constantly. I also do product management. LLMs are very effective at locating and explaining things in complex code bases. I use Copilot to help me research the current implementation and check assumptions. I'm working to extend out in exactly the directions you describe.
pron 1y ago
I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills. It doesn't work like tools developers use and it doesn't work like people developers work with. Furthermore, techniques of working with agents today may be completely outdated a year from now. The acceleration is also inconsistent: sometimes there's an acceleration, sometimes a deceleration. Generative AI is at the same time incredibly impressive and completely unreliable. This makes it interesting, but also very uncertain. Maybe it's worth my investment to learn how to master today's agents, and maybe I'd be better off waiting until these things become better. You wrote: > Getting good results out of a coding agent feels uncomfortably close to getting good results out of a human collaborator. You need to provide clear instructions, ensure they have the necessary context and provide actionable feedback on what they produce. That is true (about people) but misses out the most important thing for me: it's not about the information I give them, but about the information they give me. For good results, regardless of their skill level, I need to absolutely trust that they tell me what challenges they've run into and what new knowledge they've gained that I may have missed in my own understanding of the problem. If that doesn't happen, I won't get good results. If that kind of communication only reliably happens through code I have to read, it becomes inefficient. If I can't trust an agent to tell me what I need to know (and what I trust when working with people) then the whole experience breaks down.
- whstl 1y ago
  > doesn't work like people developers work with I don't know. This is true for people working in an environment that provides psychological safety, has room for mistakes and rewards hard work. This might sound cynical, but in all other places I see the "lying to cover your ass" behavior present in one form or another.
- schwartzworld 1y ago
  > incredibly impressive and completely unreliable. There have been methods of protecting against this since before AI, and they still apply. LLMs work great with test driven development, for example. I would say that high-level knowledge and good engineering practices more important than ever, but they were always important.
j45 1y ago
Software developers can 10x-100x productivity/effectiveness with LLMs. Non developers can go from 0x to 1x. And I'm happy for people finally being able to learn about building software one way. And then learn why vibe coding often creates more quickly disposable code.
alganet 1y ago
What is the other part of your goal?
- simonw 1y ago
  Sparking more conversations about practices that work for doing serious production-quality software development with LLMs, especially in larger teams and on larger projects. Having a good counter to people who use "vibe coding" as a dismissive term for anything where an LLM is used to help product software.
  - alganet 1y ago
    That sounds so familiar. Have you ever considered that LLMs are the new PHP? - A reasonable technology that can be used to deliver great value. - People hate it. - Terrible first impression. The wrong way of using PHP is much more popular than the good stuff. - People are very dismissive, they won't even listen to your argument.
tarunkotia 1y ago
This has been experience as well. If there is a hard problem which needs to be addressed, generative code helps me break the inertia by generating the first draft and then I get really curious to poke holes in the generated code. I tend to procrastinate when I come across a gnarly issue or something I am not really familiar with, justifying by saying I need a big block of time to work on it. I use generative code as a pushy "mom/boss/coworker/spouse" to get stuff done.
jlarocco 1y ago
So it's automating away the fun parts, and leaving the humans to rig up automated tests and setup continuous integration... And unfortunately people who get to architect anything are a small subset of developers.
tcdent 1y ago
> they're not going to be able to rig up a set of automated tests with continuous integration and continuous deployment to a Kubernetes cluster somewhere. Honestly, I have a ton of experience in system administration, and I'm super comfortable at a command line and using AWS tooling. But, my new approach is to delegate almost all of that to Claude, which can access AWS via the command-line interface and generate configuration files for me and validate that they work correctly. It has dramatically reduced the amount of time that I spend fiddling with and understanding the syntax of infra config files.
b112 1y ago
I remember reading a sci-fi book, where time was.. sharded? And people from different times were thrust together. I think it was a Phoenician army, which had learned to ride and battle bareback. And were introduced to the stability of stirrups and saddle. They were like daemons on those stirrup equipped horses. They had all the agility of wielding weapons and engaging in battle by hanging onto mane, and body with legs, yet now had (to them) a crazy easy and stable platform. When the battle came, the Phoenicians just tore through those armies who had grown up with the stirrup. There was no comparison in skill or capability. (Note: I'm positive some of the above may be wrong, but can't find the story and so am just stating it as best able) My point is, are we in that age? Are we the last skilled, deeply knowledgeable coders? I grew up learning to write eeproms on burners via the C64. Writing machine language because my machines were too slow otherwise. Needing to find information from massive paper manuals. I had to work it all out myself often, because no internet no code examples, just me thinking of how things could be done. Another person who grew up with some of the same tools and computers, once said we are the last generation to understand the true, full stack. Now I wonder, is it the same with coding? Are we it? The end?
VBprogrammer 1y ago
I really hope you are right here, and to be honest it does reflect my limited experience with where I've used AI so far. But I'm also not ready to bet the farm on it. Seriously considering taking our savings and equity out of our house in a London adjacent area, and moving to a lower cost of living area, so that we're practically debt free. At that point we can survive on a full time minimum wage job, anything more than that is a bonus.
closeparen 1y ago
>accelerate the impact you can have with this new family of tools. Tech spent the last 10 years drilling into engineers' heads that scaling your impact is not about writing more or better code, but about influencing the work of other engineers through collaboration, process, documentation, etc. Even the non-managerial "senior IC" tracks are mostly about doing this with greater and and greater numbers of people. I wonder if we will start to see recognition in career tracks for people who are actually just extraordinarily productive by themselves or in small groups, or if you'll pretty much just have to be a startup founder to get paid for that.

by simonw 1y ago

jmspring 1y ago

I need to read through this some more, but there has been another genetic coding paradigm referred to as spec driven development. I’ll find the link in the morning, but I kinda joke - it’s vibe coding for people who know how to define a problem and iterate on it. I’ve got a project reimplementing a service I want to make more uniform. Claude has produced a lot of stuff that would have taken me weeks to do.

senko 1y ago
GitHub's SpecKit is an example: https://github.com/github/spec-kit https://github.com/github/spec-kit Spec-Driven Development treats the spec as the source of truth and the code as an artifact. As you develop, you modify/add to the spec and the codebase gets updated to reflect it. Personally I'm doubtful it can compete with traditional artisanal software engineering, as it's (IMHO) boils down to "if only you can spec it precisely enough, it'll work" and we've tried this with 5GL and (to some extent) BDD, and it doesn't get you to 100%. I do think it's interesting enough to explore, and most of us could use a bit more details in our Jira tickets.
- piva00 1y ago
  That was exactly what UML wanted to do, and it almost never worked out in practice. Seems to be just a rehashing of the same idea but instead of XML, and diagrams, it's now some free-text to be interpreted by LLMs, so much less deterministic and will probably fail just like UML failed. People also tend to forget about Peter Naur's take on "Programming as Theory Building" [0], the program is, in itself, the theory of what's implemented. A spec cannot replace that. [0] https://pages.cs.wisc.edu/~remzi/Naur.pdf https://pages.cs.wisc.edu/~remzi/Naur.pdf
  - andrewgleave 1y ago
    Theory building is the secret sauce, and all variants of "this is how to use AI effectively" I've seen are inferior to the epistemologically sound theory Naur outlines in his paper.
- WOTERMEON 1y ago
  Hey! No J word tolerated in here!
- Kzozjj 1y ago
  We already invented languages for succinctly describing what the computer should do. They’re call programming languages. “The code is the documentation” is not a joke. Logic that’s useful in the real world is complex and messy. You need additional documentation (why did the code end up like it is, etc) but code is the most expressive way we’ve got for describing how a computer should work.

senko 1y ago

> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for. I wonder what the practical limits are. As a senior dev on a greenfield solo project it's too exhausting for me to have two parallel agents (front/back), most of the time they're waiting for me to spec, review or do acceptance test. Feels like sprinting, not something I could do day in and day out. Might be due to tasks being too fine grained, but assuming larger ones are proportionally longer to spec and review, I don't see more than two (or, okay, three, maybe I'm just slow) being a realistic scenario. More than that, I think we're firmly in the vibe coding (or maybe spec-driven vibe coding) territory.

afro88 1y ago
At least on a team, the limit is the team's time to review all the code. We've also found that vibe engineered (or "supervised vibing" as I call it) code tends to have more issues in code review because of a false sense of security creating blind spots when self reviewing. Even more burden on the team. We're experimenting with code review prompts and sub agents. Seems local reviews are best, so the bulk of the burden is on the vibing engineer, rather than the team.
- bartread 1y ago
  Do you have a sense for how much overhead this is all adding? Or, to put it another way, what I’m really asking is what productivity gain (or loss) are you seeing versus traditional engineering?
  - rhetocj23 1y ago
    Very curious to hear responses about this too
    - oblio 1y ago
      The problem with this is that software engineering is a very unorganized and fashion/emotion driven domain. We don't have reliable productivity numbers for basically... anything. I <feel> that I'm more productive with statically typed languages but I haven't seen large scale, reliable studies. Same with unit tests, integration tests, etc. And then there are all the types of software engineering: web frontend, web API, mobile frontend, command line frontend, Windows GUI, MacOS GUI, Linux backend (10 million different stacks), Windows backend (1 million different stacks), throwaway projects, WordPress webpages, etc, etc.

lemming 1y ago

While this is true, I definitely find that the style of the work changes a lot. It becomes much more managerial, and less technical. I feel much more like a mix of project and people manager, but without the people. I feel like the jury is still out on whether I’m overall more productive, but I do feel like I have less fun.

benterix 1y ago
My lessons so far: 1. Less fun. 2. A lot of more "review fatigue". 3. Tons of excess code I'd never put in there in the first place. 4. Frustration with agents being too optimistic which with time verges on the ludicurous ("Task #3 has been completed successfully with 98% tests failing. [:useless_emojis:]") 5. Frustration with agents routinely getting down a rabbit hole or going in circles, the effort needed to get that straight (Anthropic plainly advises to start from scratch in such cases - which is sound advice, but makes me feel like I just lost the last 5 hours of my life without even learning anything new). I stopped using agents and use LLMs very sparingly (e.g. for review - they sometimes find some details I missed and occasionally have an interesting solution) but I'm enjoying my work so much more without them.
- exe34 1y ago
  98% tests sounds really great though, give that LLM a raise!
- resonious 1y ago
  I think one of the tricks is to just stop using the agent as soon as you see signs of funny business. If it starts BSing me with failing tests, I just turn it off immediately and git reset (maybe after taking a quick peek)
  - Sammi 1y ago
    Yeah I make maybe two or three attempts at getting it to write a plan that it is able to follow coherently. But after that I pull the escape hatch and *gasp* program by hand. I've made this mistake of doubling down after a few initial failures to solve an issue, by trying to make this super duper comprehensive and highly detailed and awesome plan that it will finally be able to implement correctly. But it just gets worse and worse the more I try, because it fundamentally is not understanding what is going on, so it will inevitably find an opportunity to go massively off rails, and the further down you lead it the more impressible the derailment will be. My experience is that going around in endless circles with the model is just a waste of time when you could have just done it yourself in the time you've wasted.

deanCommie 1y ago

I don't think OP thinks his skills are useless per se now, but that the way to apply those skills now feels less fun and enjoyable. Which makes perfect sense - even putting aside the dopamine benefits of getting into a coding flow state. Coding is craftsmanship - in some cases artistry. You're describing Vibe Engineering as management. And sure, a great manager can make more of an impact increasing the productivity of an entire team than a great coder can make by themselves. And sure, some of the best managers are begrudging engineers who stepped up when needed to and never stepped down. But most coders still don't want to be managers - and it's not from a lack of skill or interest in people - it's just not what they chose. LLM-based vibe coding and engineering is turning the creative craftsmanship work of coding into technical middle management. Even if the result is more "productivity", it's a bit sad.

ako 1y ago
But does anybody really care about what you like? What about all those other professions that got replaced by technology, did anybody care what they liked? The big question is how is software going to be build most efficiently and most effectively in the future and how do you prepare yourself for this new world. Otherwise you’ll end up with all those other professions that got replaced, like the mineworkers, hoping that the good old days will someday return.
- zwnow 1y ago
  Its reasonable to stay away from something one considers dystopian considering the industry is not even sure about the usefulness of coding agents in professional environments. When the tractors replaced the horses, everyone could agree they outperform horses. The result was easily measurable. Its not that simple with LLM agents owned by big corporations.
  - ako 1y ago
    Sure, it's not yet clear what impact LLMs will have on software development, but the impact it will have will not depend on if developers like to use it or not. If it is going to make software development 10x faster, companies will adopt it, whether devs like it or not.

saulpw 1y ago

I appreciate what you're trying to do, but for myself, I'm not depressed because my skills are less valuable. I enjoyed the money but it was never about that for me. I'm depressed because I don't like the way this new coding feels in my brain. My focus and attention are my most precious resources and vibe coding just shatters them. I want to be absorbed in a coding flow where I see all the levels of the system and can elegantly bend the system to my will. Instead I'm stuck reviewing someone/something else's code which is always a grind, never a flow. And I can feel something terrible happening in my brain, which at best can be described as demotivation, and at worst just utter disinterest. It's like if I were a gardener and I enjoyed touching dirt and singing to plants, and you're here on Gardener News extolling the virtues of these newfangled tractors and saying they'll accelerate my impact as a gardener. But they're so loud and unpleasant and frankly grotesque and even if I refrain from using one myself, all my neighbors are using them and are producing all their own vegetables, so they don't even care to trade produce anymore--with me or anyone else. So I look out at my garden with sadness, when it gave me such joy for so many decades, and try to figure out where I should move so I can at least avoid the fumes from all the tractors.

shafyy 1y ago
How beautifully put, and I couldn't agree more. I feel exactly the same way. However, I am still unconvinced that software development will go down this way. But if woking as a software developer will require managing multiple agents at the same time instead of crafting your own code — you can count me out, too.
sally_glance 1y ago
Well said! Reading this I feel reminded of the early protests against industrialization and automation in other fields. Checks all the same boxes - insecurity and fear about the future, alienation towards the new tools, ... Not saying AI is similar in impact to the loom or something, it just occured to me how close this is to early Luddite texts.
- bonoboTP 1y ago
  The OG alienator was the Agricultural Revolution, settling and toiling repetitively in predetermined ways, unlike the more adventurous lifestyle from before with all the camping, hunting, gathering, where circumstances brought always novel challenges, you could be a man spearing a deer, instead of just killing some docile domesticated cow. Searching for pheasant eggs and being happy if you found some, instead of going out every morning to the predictable presence of eggs in the chicken coop.

zwnow 1y ago

What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents. It used to be a field where you could get a laptop from the scrapyard and learn from there. It feels pointless. Also agents do not invent things, the creativity part is gone with them. They simply use what they've already seen, repeat the same mistakes a person made a few years ago. Its a dystopian way of working. Sure it enables one to spew out slop that might make companies money, but there is no passion, sense of exploration, personal growth. Its all just directing parrots with thumbs...

bananapub 1y ago
> What about the accessibility of software development? Its completely vanishing for people that can not afford to pay for these agents. what do you actually mean by this? it's clearly untrue - anyone get get a laptop and install linux on it and start bashing out code today, just as they could last week and last year and thirty years ago. do you mean that you think at some point in the future tooling for humans to write code won't exist? or that employers won't hire human programmers? or that your pride is hurt? or you want your hobby to also be a well-paid job? or something else?
- zwnow 1y ago
  I mean that this "tooling" becomes inaccessible to people. At least the tooling that is relevant for jobs. Employers will eventually stop hiring human based on their programming competence. It'll translate into a low pay career for people who like to orchestrate agents.
  - flir 1y ago
    I think it's just another floor in the creaky old tower of abstraction. Machine code > ASM > 3GLs > 4GLs > visual programming > LLMs etc etc etc. Thing is, the moment you go off-piste, the LLMs get a lot less useful. I think, if you want to stay closer to the metal, you've got to aim for a niche that has a small internet footprint. So... domain knowledge or esoteric programming knowledge. One way to incorporate domain knowledge might be to become a hybrid product owner/programmer. (This is all just opinion - I'm sure a well-argued rebuttal is possible).

HSO 1y ago

Have you tried SolveIt (method, tool) from Jeremy Howard yet? I was in the first batch last year where they introduced it and going to do the second one too. It´s a very different kind of beast to what is currently being discussed.

raducu 1y ago
> going to do the second one too. I missed the first one, when will the second one be?
- ayewo 1y ago
  "Signups are open [1], and will remain so until October 20th." Recently on HN [2]. [1]: https://solve.it.com/ https://solve.it.com/ [2]: https://news.ycombinator.com/item?id=45455719 https://news.ycombinator.com/item?id=45455719

exe34 1y ago

> They're also not going to be able to direct three different agents at once in different areas of a large project that they've designed the architecture for. Neither can I, sadly. I have one brain cell and I can only really do one thing at a time. Doing more than one leads to a corrupted stack and I make exponentially more mistakes.

codr7 1y ago

I still haven't seen any evidence to match these repeated claims of increased efficiency. What I have seen is reports that makes a lot of sense to me claiming it's all in the user's head.

Cthulhu_ 1y ago
I'm working in corporate and haven't seen it yet. The main thing I see is blogs and whatnot of people building new weekend projects with LLMs, that is, greenfield, non-critical software - the type of software that, if I were to write it, I wouldn't bother with CI, tests, that kind of thing with. Sloppy projects, if you will. But happy to be corrected - is someone using these agents in their paid / professional / enterprise / team job?
- rhetocj23 1y ago
  I’ve asked this many times on here - I never get a coherent answer
- girvo 1y ago
  I am, but in a very narrow focus: mostly examining our existing codebase as a more powerful but fuzzier search, and a system to then generate a plan to implement and approach which I tweak. I sometimes use it to scaffold out some boilerplate for tests, but never tests themselves: no matter what I try it always ends up writing the useless straight-jacket "change alert" style tests that break on any change to the unit under test, which I despise.
- Aeolun 1y ago
  I think most of the code in our enterprise is now written by AI. It’s all boring callcenter crud apps, so nobody is really sad they’re not writing any of that code any more. I’m not sure it makes me faster, but I think QA testing what the AI made and occassionaly adjusting it is more fun anyway. The code is absolutely lower quality, but there were always so many people producing garbage faster than I could produce something nice that the code was always terrible anyway. There’s an element of wanting to know how the thing works so at least I’ll know when it’s ready to replace me though.

llamatastic 1y ago

I really don't get the idea that LLMs somehow create value. They are burning value. We only get useful work out of them because they consume past work. They are wasteful and only useful in a very contrived context. They don't turn electricity and prompts into work, they turn electricity, prompts AND past work into lesser work. How can anyone intellectually honest not see that? Same as burning fossil fuels is great and all except we're just burning past biomass and skewing the atmosphere contents dangerously in the process.

simonw 1y ago
> How can anyone intellectually honest not see that? The idea that they can only solve problems that they've seen before in their training data is one of these things that seems obviously true, but doesn't hold up once you consistently use them to solve new problems over time. If you won't accept my anecdotal stories about this, consider the fact that both Gemini and OpenAI got gold medal level performance in two extremely well regarded academic competitions this year: the International Math Olympiad (IMO) and the International Collegiate Programming Contest (ICPC). This is notable because both of those contests have brand new challenges created for them that have never been published before. They cannot be in the training data already!
- throw219080123 1y ago
  > consider the fact that both Gemini and OpenAI got gold medal level performance Yet ChatGPT 5 imagines API functions that are not there and cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub.
  - simonw 1y ago
    Which is why you run it in a coding agent loop using something like Codex CLI - then it doesn't matter if it imagines a non-existent function because it will correct itself when it tries to run the code. Can you expand on "cannot figure out basic solutions even when pointed to the original source code of libraries on GitHub"? I have it do that all the time and it works really well for me (at least with modern "reasoning" models like GPT-5 and Claude 4.)

MintsJohn 1y ago

Of course the devil is in the details. What you say and the skills needed make sense. It's unfortunately also the easiest aspects to dismiss either under pressure as there is often little immediate payoff, or because it's simply the hard part. My experience with llms in general is that sadly, they're mostly good bullshitters. (current google search is the epitome of worthlessness, the AI summary so hard tries to make things balanced, that it just dreams up and exaggerates pros en cons for most queries). In a same way platforms like perplexity are worthless, they seem utterly unable to assign the proper value to sources they gather. Of course that doesn't stop me from using llms where they're useful; it's nice to be able to give the architecture for a solution and let the llm fill the gaps than to code the entire thing by hand. And code-completion in general is a beautiful thing (sadly not a thing where much focus is on these days, most is on getting the llm create complete solutions while i would be delighted by even better code completion) Still all in all, the more i see llms used (or the more i see (what i assume) well willing people copy/paste llm generated responses in favor of handwritten responses) on so much of the internet, resulting in a huge decline of factualness and reproducibility (in he sense, that original sources get obscured), but an increase of nice full sntences and proper grammar, the more i'm inclined to belief that in the foreseeable future llm's aren't a net positive. (in a way it's also a perfect storm, the last decade education unprioritised teaching skills that would matter especially for dealing with AI and started to educate for use of tools instead of educate general principles. The product of education became labourers for a specific job instead of higher abstract level reasoning in a general area of expertise)

simonw 1y ago
Google's "AI overviews" are one of the worst LLM-powered features on the market today, they're genuinely damaging the reputation of the whole industry. Meanwhile I've started using ChatGPT GPT-5 search as my default search engine! A year ago I would have laugher at the idea: https://simonwillison.net/2025/Sep/6/research-goblin/ https://simonwillison.net/2025/Sep/6/research-goblin/ And Google themselves have an "AI mode" which is a different league of quality from "AI overviews", I wrote about that one here: https://simonwillison.net/2025/Sep/7/ai-mode/ https://simonwillison.net/2025/Sep/7/ai-mode/ This is new. AI search tools almost universally sucked until OpenAI's release of o3 in April this year.

justbees 1y ago

I'm getting really great results in a VERY old (very large) codebase by having discussion with the LLM (I'm using Claude code) and making detailed roadmaps for new features or converting old features to new more useable/modern code. This means FE and BE changes usually at the same time. I think a lot of the points you make are exactly what I'm trying to do. - start with a detailed roadmap (created by the ai from a prompt and written to a file) - discuss/adjust the roadmap and give more details where needed - analyze existing features for coding style/patterns, reusable code, existing endpoints etc. (write this to a file as well) - adjust that as needed for the new feature/converted feature - did it miss something? Is there some specific way this needs to be done it couldn't have known? - step through the roadmap and give feedback at each step (I may need to step in and make changes - I may realize we missed a step, or that there's some funky thing we need to do specifically for this codebase that I forgot about - let the LLM know what the changes are and make sure it understands why those changes were made so it won't repeat bad patterns. i.e. write the change to the .md files to document the update) - write tests to make sure everything was covered... etc etc Basically all the things you would normally WANT do but often aren't given enough time to do. Or the things you would need to do to get a new dev up to speed on a project and then give feedback on their code. I know I've been accomplishing a lot more than I could do on my own. It really is like managing another dev or maybe like pair programming? Walk through the problem, decide on a solution, iterate over that solution until you're happy with the decided path - but all of that can take ~20 minutes as opposed to hours of meetings. And the end result is factors of time less than if I was doing it on my own. I recently did a task that was allotted 40 hours in less than 2 working days - so probably close to 10-12 hours after adjusting for meetings and other workday blah blah blah. And the 40 hour allotment wasn't padded. It was a big task, but doing the roadmap > detailed structure including directory structure - what should be in each file etc etc cut the time down dramatically. I would NOT be able to do this if I the human didn't understand the code extremely well and didn't make a detailed plan. We'd just end up with more bad code or bad & non-working code.

hodgesrm 1y ago
Thank you for this post. I don't write much code as I'm currently mostly managing people but I read it constantly. I also do product management. LLMs are very effective at locating and explaining things in complex code bases. I use Copilot to help me research the current implementation and check assumptions. I'm working to extend out in exactly the directions you describe.

pron 1y ago

I'm not sure that having the patience to work with something with a very inconsistent performance and that frequently lies is an extension of existing development skills. It doesn't work like tools developers use and it doesn't work like people developers work with. Furthermore, techniques of working with agents today may be completely outdated a year from now. The acceleration is also inconsistent: sometimes there's an acceleration, sometimes a deceleration. Generative AI is at the same time incredibly impressive and completely unreliable. This makes it interesting, but also very uncertain. Maybe it's worth my investment to learn how to master today's agents, and maybe I'd be better off waiting until these things become better. You wrote: > Getting good results out of a coding agent feels uncomfortably close to getting good results out of a human collaborator. You need to provide clear instructions, ensure they have the necessary context and provide actionable feedback on what they produce. That is true (about people) but misses out the most important thing for me: it's not about the information I give them, but about the information they give me. For good results, regardless of their skill level, I need to absolutely trust that they tell me what challenges they've run into and what new knowledge they've gained that I may have missed in my own understanding of the problem. If that doesn't happen, I won't get good results. If that kind of communication only reliably happens through code I have to read, it becomes inefficient. If I can't trust an agent to tell me what I need to know (and what I trust when working with people) then the whole experience breaks down.

whstl 1y ago
> doesn't work like people developers work with I don't know. This is true for people working in an environment that provides psychological safety, has room for mistakes and rewards hard work. This might sound cynical, but in all other places I see the "lying to cover your ass" behavior present in one form or another.
schwartzworld 1y ago
> incredibly impressive and completely unreliable. There have been methods of protecting against this since before AI, and they still apply. LLMs work great with test driven development, for example. I would say that high-level knowledge and good engineering practices more important than ever, but they were always important.

j45 1y ago

Software developers can 10x-100x productivity/effectiveness with LLMs. Non developers can go from 0x to 1x. And I'm happy for people finally being able to learn about building software one way. And then learn why vibe coding often creates more quickly disposable code.

alganet 1y ago

What is the other part of your goal?

simonw 1y ago
Sparking more conversations about practices that work for doing serious production-quality software development with LLMs, especially in larger teams and on larger projects. Having a good counter to people who use "vibe coding" as a dismissive term for anything where an LLM is used to help product software.
- alganet 1y ago
  That sounds so familiar. Have you ever considered that LLMs are the new PHP? - A reasonable technology that can be used to deliver great value. - People hate it. - Terrible first impression. The wrong way of using PHP is much more popular than the good stuff. - People are very dismissive, they won't even listen to your argument.

tarunkotia 1y ago

This has been experience as well. If there is a hard problem which needs to be addressed, generative code helps me break the inertia by generating the first draft and then I get really curious to poke holes in the generated code. I tend to procrastinate when I come across a gnarly issue or something I am not really familiar with, justifying by saying I need a big block of time to work on it. I use generative code as a pushy "mom/boss/coworker/spouse" to get stuff done.

jlarocco 1y ago

So it's automating away the fun parts, and leaving the humans to rig up automated tests and setup continuous integration... And unfortunately people who get to architect anything are a small subset of developers.

tcdent 1y ago

> they're not going to be able to rig up a set of automated tests with continuous integration and continuous deployment to a Kubernetes cluster somewhere. Honestly, I have a ton of experience in system administration, and I'm super comfortable at a command line and using AWS tooling. But, my new approach is to delegate almost all of that to Claude, which can access AWS via the command-line interface and generate configuration files for me and validate that they work correctly. It has dramatically reduced the amount of time that I spend fiddling with and understanding the syntax of infra config files.

b112 1y ago

I remember reading a sci-fi book, where time was.. sharded? And people from different times were thrust together. I think it was a Phoenician army, which had learned to ride and battle bareback. And were introduced to the stability of stirrups and saddle. They were like daemons on those stirrup equipped horses. They had all the agility of wielding weapons and engaging in battle by hanging onto mane, and body with legs, yet now had (to them) a crazy easy and stable platform. When the battle came, the Phoenicians just tore through those armies who had grown up with the stirrup. There was no comparison in skill or capability. (Note: I'm positive some of the above may be wrong, but can't find the story and so am just stating it as best able) My point is, are we in that age? Are we the last skilled, deeply knowledgeable coders? I grew up learning to write eeproms on burners via the C64. Writing machine language because my machines were too slow otherwise. Needing to find information from massive paper manuals. I had to work it all out myself often, because no internet no code examples, just me thinking of how things could be done. Another person who grew up with some of the same tools and computers, once said we are the last generation to understand the true, full stack. Now I wonder, is it the same with coding? Are we it? The end?

VBprogrammer 1y ago

I really hope you are right here, and to be honest it does reflect my limited experience with where I've used AI so far. But I'm also not ready to bet the farm on it. Seriously considering taking our savings and equity out of our house in a London adjacent area, and moving to a lower cost of living area, so that we're practically debt free. At that point we can survive on a full time minimum wage job, anything more than that is a bonus.

closeparen 1y ago

>accelerate the impact you can have with this new family of tools. Tech spent the last 10 years drilling into engineers' heads that scaling your impact is not about writing more or better code, but about influencing the work of other engineers through collaboration, process, documentation, etc. Even the non-managerial "senior IC" tracks are mostly about doing this with greater and and greater numbers of people. I wonder if we will start to see recognition in career tracks for people who are actually just extraordinarily productive by themselves or in small groups, or if you'll pretty much just have to be a startup founder to get paid for that.