11 ms·

The role of developer skills in agentic coding

336 points by BerislavLopac 2y ago

bob1029 2y ago
> supervised agent This is the trick. Human in the loop, not human hiding in an ivory tower after uttering a single command. This is ~effectively what I see a lot of shops doing right now: "Clean up the codebase please. Apply best practices :D. OH. By the way, heres a laundry list of 100 things to NOT do: <list begins>". I get a lot more uplift out of use cases like: "Please generate a custom Stream implementation that is read-only and sources bytes from an underlying chunked representation. Mock the chunk loading part. Primarily demonstrate the ReadAsync method and transition logic between chunks."
usrbinbash 2y ago
Here is how I use it: As a writing assistant that lives in my IDE, and as a very very cool and sophisticated rubber duck that can answer me. Something I do quite a lot is throwing back and forth a discussion over a particular piece of code, usually provided with little to no context (because that's my task to worry about), hammering it until we get that functionality correct, then presenting it with broader context to fit it in (or I simply do that part by hand). Here is how I don't use it: As an agent that gets broad goals that he is supposed to fulfill on its own. Why? Because the time and effort I have to invest to ensure that the output of an agentic system is in line with what I actually try to accomplish, is simply too much, for all the reasons outlined in this excellent article. Ironically, this is even more true, since using AI as an incredibly capable writing assistant, already speeds up my workflow considerably. So in a way, less agentic AI empowers me in a way that makes me more critical of the additional time I'd have to invest to play around the quirks of agentic AI.
- kubanczyk 2y ago
  This doesn't parse for me: > a discussion over a particular piece of code [...] hammering it until we get that functionality correct Care to provide an example of sorts? > then presenting it with broader context to fit it in So after you have a function you might convert it to a method of a class. Stuff like that?
  - usrbinbash 2y ago
    > Care to provide an example of sorts? For example, recently I needed to revise some code I wrote a few years back, re-implementing a caching mechanism to make it work across networked instances of the same software. I had a rough idea how I wanted to do that, and used an LLM to flesh out the idea. The conversation starts with an instruction that I don't want any code written until I ask for it, then I describe the problem itself, let it list the key points, and then present my solution (all still as prose, no code so far). Next step, I ask for its comments, and patterns/implementation details how to do that, as well as alternatives to those. This is the "design phase" of the conversation. Once we zoom in on a concrete solution, I ask it to produce a minimal example code of what we discussed. Then we repeat the same process, this time discussing details about the code itself. During that phase I tell it what parts of the implementation it doesn't t need to worry about and what to focus on, keeping it from going off to mock-up-wonderland. At the end it usually gets an instruction like "alright, please write out the code implementing what we discussed so far, in the same context as before". This gives me a starting point to work from. If the solution is fairly small, I might then give it some of the context this code will live in, and ask it to "fill in the blanks" as it were...often though I do that part myself, as its mostly small refactoring and renaming. What I find so useful about this workflow, as opposed to just throwing the thing at my project directory; it prevents the AI from getting side tracked, lost, as it were, in some detail, endlessly chasing its own tail trying to make sense of some compiler error. The human in the loop (yours truly), sets the stage, presents the focus, and the starting point is no code at all, just an ephemeral description of a problem and a discussion about it, grounding all the latter steps of the interaction. Hope that makes sense.
decompiled_dev 2y ago
I think of the modern developer being more like a shepherd rather than a builder now. You have to vibe with the machine, but you need to make sure they stay on easy terrain and give them structure instead of letting them simply free graze.
jvanderbot 2y ago
I'm surprised to not see an obvious one (for me): Use AI around the periphery. There's very often a heap of dev tools, introspection, logging, conversion, etc tools that need to be build and maintained. I've had a lot of luck using agents to make and fix these. For example a tool that collates data and logs in a bespoke planning system. It is a lot of generated boilerplate off the critical path to build these tools and I just don't want to do it most days.
- aulin 2y ago
  In my every day experience that's pretty risky. The periphery as you call it is often an area where you lack the expertise to spot and correct AI mistakes. I am thinking about build systems and shell scripts. I see people everyday going to AI before even looking at the docs and invariably failing with non-existent command line options, or worst options that break things in very subtle ways. Same people that when you tell them why don't you read the f-ing man page they go to google to look it up instead of opening a terminal. Same people that push through an unknown problem by trial and error instead of reading the docs first. But now they have this dumb counselor that steers them in the wrong direction most of the time and the whole process is even more error prone.
  - jvanderbot 2y ago
    You're wrong. I have all the expertise but none of the time to generate 100s of lines of boilerplate API calls to get the data together, and no interest in formatting it correctly for consumption, let alone doing so state fully to allow interaction. These are trivial problems to solve that are highly tedious and do not affect whatsoever the business at hand. Perfect drudgery for automation, and just scanning the result is easy to verify the output or code.
    - aulin 2y ago
      I am not wrong. You simply are not the kind of developer I am thinking about. And believe me the other kind is way more represented.
mjr00 2y ago
> 15k LOC codebase I wish articles about AI assistance would caveat this at the start. 15k LOC is a weekend hackathon project, which is all well and good, but not reflective of the work that 99% of developers are doing in their day jobs.
- jstanley 2y ago
  I, too, habitually write 7500 lines of code per day.
  - mjr00 2y ago
    For a brand new project that you're trying to get done in a weekend--aka a weekend hackathon project--it's very doable. Would not recommend committing 7500 lines of code per day at your day job though.
    - ariwilson 2y ago
      If you work 8 hours a day on Saturday and Sunday and each line has on average 40 characters, then on average you have to type about 5 characters a second to hit 7000 lines of code. There's some heavy assumptions about boilerplate or autogenerated code going on in that estimate, as I don't think very many average 5 characters a second over 16 hours.
      - Zanfa 2y ago
        40 seems way too much. “}” alone probably makes up a significant portion of lines. Another thing is auto-complete that does the majority of typing anyway.
      - Jtsummers 2y ago
        In English, the average word length is 5 characters. If you can type 60 WPM that means you're typing on average at least 300 characters per minute or at least 5 characters per second (at least because this is not counting the needed whitespace and punctuation). That makes it technically possible for a moderately capable typist to pull off 7000 lines in a day using your numbers. Pair that with IDEs and autocomplete and it becomes much more feasible, if you have a solid understanding of the objective for your code.
timdellinger 2y ago
I find that I have to steer the AI a lot, but I do have optimism that better prompting will lead to better agents. To take an example from the article: code re-use. When I'm writing code, I subconsciously have a mental inventory of what code is already there, and I'm subconsciously asking myself "hey, is this new task super similar to something that we already have working (and tested!) code for?". I haven't looked into the details of the initial prompt that a coding agent gets, but my intuition is that an addition to the prompt instructing the agent to keep an inventory of what's in the codebase, and when planning out a new batch of code, check the requirements of the new tasks against what's already there. Yes, this adds a bunch of compute cycles to the planning process, but we should be honest and say "that's just the price of an agent writing code". Better planning > ability to fix things.
- hnuser123456 2y ago
  There are certain pieces of text that appear right before some of the greatest pieces of code ever written. For example, we've all heard of NASA code requirements. If you get the LLM into the "mindset" of a top-tier professional developer before getting it to spit out code, the code quality will reflect that. If your prompt is sloppy and poorly defined, you'll get copy-pasted StackOverflow code, since that's how most SO questions look. If it's stupid but it works, it's not stupid. The hard part is that finding a local optimum for prompting style for one LLM may or may not transfer to another depending on personality post-training. And whatever style works best with all LLMs must be approaching some kind of optimum for using English to design and specify computer programs. We cannot have better programs without better program specifications.
  - yaj54 2y ago
    Can you share some examples of these certain pieces of text and greatest pieces of code?
    - hnuser123456 2y ago
      Well, if you want safety-critical code, you could have the LLM read this before asking it to write its own code: https://ieeexplore.ieee.org/document/1642624 https://ieeexplore.ieee.org/document/1642624 GP was pondering about code re-use. My typical use involves giving an entire file to the LLM and asking the LLM to give the entire file back implementing requested changes, so that it's forced to keep the full text in context and can't get too off-track by focusing on small sections of code when related changes might be needed in other parts of the file. I think all of this is getting at the fact that an LLM won't spit out perfect code in response to a lazy prompt unless it's been highly post-trained to "reinterpret" sloppy prompts just as academically as academic prompts. Just like a human programmer, you can just give the programmer project descriptions and wait for the deliverable and accept it at face value, or you can join the programmer along their journey and verify their work is according to the standards you want. And sometimes there is no other way to get a hard project done. Conversely, sometimes you can give very detailed specifications and the LLM will just ignore part of them over and over. Hopefully the training experts can continue to improve that.
ikerino 2y ago
I use Cursor for most of my development these days. This article aligns pretty closely with my experiences. A few additional observations: 1. Anecdotally, AI agents feel stuck somewhere circa ~2021. If I install newer packages, Claude will revert to outdated packages/implementations that were popular four years ago. This is incredibly frustrating to watch and correct for. Providing explicit instructions for which packages to use can mitigate the problem, but it doesn't solve it. 2. The unpredictability of these missteps makes them particularly challenging. A few months ago, I used Claude to "one-shot" a genuinely useful web app. It was fully featured and surprisingly polished. Alone, I think it would've taken a couple weeks or weekends to build. But, when I asked it to update the favicon using a provided file, it spun uselessly for an hour (I eventually did it myself in a couple minutes). A couple days ago, I tried to spin up another similarly scoped web app. After ~4 hours of agent wrangling I'm ready to ditch the code entirely. 3. This approach gives me the brazenness to pursue projects that I wouldn't have the time, expertise, or motivation to attempt otherwise. Lower friction is exciting, but building something meaningful is still hard. Producing a polished MVP still demands significant effort. 4. I keep thinking about The Tortoise and The Hare. Trusting the AI agent is tempting because progress initially feels so much faster. At the end of the day, though, I'm usually left with the feeling I'd have made more solid progress with slower, closer attention. When building by hand, I rarely find myself backtracking or scrapping entire approaches. With an AI-driven approach, I might move 10x faster but throw away ~70% of the work along the way. > These experiences mean that by no stretch of my personal imagination will we have AI that writes 90% of our code autonomously in a year. Will it assist in writing 90% of the code? Maybe. Spot on. Current environment feels like the self-driving car hype cycle. There have been a lot of bold promises (and genuine advances), but I don't see a world in the next 5 years where AI writes useful software by itself.
- Aurornis 2y ago
  > It feels like my AI agents are stuck somewhere circa ~2021. If I install newer packages or more recent versions, Claude will often revert to outdated packages/implementations that were popular four years ago. My experience is the same, though the exact dates differ. I assume LLMs gravitate toward solutions that are most represented in their training material. It's hard to keep them pulled toward newer versions without explicitly mentioning it all the time.
osigurdson 2y ago
Is Martin Fowler now just renting out space on his website?
- Apocryphon 2y ago
  What are you referring to?
  - Jtsummers 2y ago
    GP was apparently unaware until now that martinfowler.com has basically been a blog/article hosting site (though more in the highly curated sense than a generic hosting site, more akin to a trade publication) for the last couple decades and that not all the content is written by Martin Fowler himself. The author of this piece is Birgitta Böckeler.
    - bitwize 2y ago
      Kinda like rogerebert.com, even before the esteemed film critic's death.
- onionbagle 2y ago
  right!? It was a little misleading that the article was written by someone else. (Birgitta Böckeler)
- Cupprum 2y ago
  Martin Fowlers page is basically the place where Thoughtworks [1] places their articles. A lot of good stuff there on many different topics. 1: https://www.thoughtworks.com/ https://www.thoughtworks.com/
- nikolayasdf123 2y ago
  yeah, confusing. URL literally says martinfowler, and article written by someone else. what... :/
GiorgioG 2y ago
Great, so now instead of spending 8 hours writing code, I spend 8 hours "steering" an AI to write the same code. What a fucking win (for the AI companies and no one else.)
- woah 2y ago
  Then don't
johnjungles 2y ago
I actually think that most problems that took 5-20 mins are now a few minutes and it’s more about how many of those intense minutes and loops you’re going through. Also, right now engineers are hyper optimized in the code aspects but not thinking about the context into cursor and context out of cursor. Like the amount of copy paste from Notion / JIRA / Sentry and the amount of output like summarizing the git commits and PRs, slack and other “over communication” you have to do these days. This is the area I think we can more easily automate away.
adamgordonbell 2y ago
Lack of reuse AI-generated code sometimes lacks modularity, making it difficult to apply the same approach elsewhere in the application. Example: Not realising that a UI component is already implemented elsewhere, and therefore creating duplicate code. Example: Use of inline CSS styles instead of CSS classes and variables This is the big one I hit for sure. I think it's a problem with agentic RAG, where it only knows the files it's looked in and not the overall structure or where to look for things, so it just recreates them.
- nikolayasdf123 2y ago
  yep. noticed this too. but I think this is due to short context LLMs have. now my tools, mostly browser LLMs copy-paste code, are analysing small sub-problem at time. once LLMs will see whole repository + history of that repository, likely they will write abstractions that can be reused
jillesvangurp 2y ago
I use LLMs for various purposes in day to day development. I don't use any of the tools mentioned in the article because I'm using intellij and don't want to replace a tool that has lots of stuff that I use all the time. But aside from that, it's good advice and matches my experience. I've dabbled with plugins for intellij but wasn't really happy with those. But ever since chat gpt for desktop started interfacing directly with jetbrains products (and vs code as well), that's my goto tool. I realized that I like being able to pull that up with a simple keybinding and it auto connects to the IDE when I do. I don't need to replace my tools and I get to have AI support ready to go. Most of the existing plugins seem to insist on some crappy auto complete, which in a tool that offers a lot of auto complete features already is a bit of an anti feature. I don't need clippy style autocomplete. What matters here is the tool integration, not the model quality. Better tool integration means better prompts with less work and getting better answers that way. Example: I run a test, it fails with some output. I had this yesterday. So I asked, "why is this failing" and had a short discussion about what could be wrong. No need for me to specify any detail; all extracted from the IDE. We ticked off a few possible causes, I excluded them. And then it noticed a subtle change in the log messages that I had not noticed (a co-routine context switch) that turned out to be the root cause. That kind of open ended debugging is a bit of a mixed bag. Sometimes it finds stuff. Mostly it just starts proposing solutions based on a poor analysis of the problem. What works pretty reliably is: - address the TODOs / FIXMEs, especially if you give it some examples of what you expect - write documentation (very good for this) - evaluate if I covered all the edge cases (often finds stuff I want to fix) - simple code transformations (rewrite this using framework X instead of Y) I don't trust it blindly. But it's generally giving me good code and feedback. And I get to outsource a lot of the boring crap.
- marstall 2y ago
  Yes I find chatGPT/Jetbrains (RubyMine) in my case is the most usable setup I've encountered. It's like Rubymine is "home" for me - and chatGPT's macOS client has become another "home" for me so it's quite convenient that they talk to each other now. I have a little FOMO about Cursor though. ChatGPT will automatically apply its suggested changes to my open editor - but I have the sense Cursor will do a bit more? Apply changes to multiple files? And have knowledge of your whole project, not just open files? Can someone fill me in
fasteddie31003 2y ago
I've been playing around with vibe coding and I think a lot of the issues brought up could be fixed by an architecture abstraction layer that does not exist today. My idea would be something like an architecture-graph (Archigraph working title) that would recursively describe how an application works or should work. Then when an agentic coder is doing a task they can easily see the bigger picture of how an application works and hopefully writing better code. Anyone interested in working on this with me?
- rvz 2y ago
  "Vibe Coding" is not Software Engineering. It is "Slopware Engineering".
- reztip 2y ago
  I'd be interested to see how tools like Cursor/Windsurf and even Copilot are modeling this internally. This idea seems to be a natural extension to having a hierarchy/team of agents all working on a common goal. In this case I would imagine one agent serves as the architect or reviewer, and another generates code to meet spec, and they fight back and forth until converging/diverging.
- rixed 2y ago
  That abstraction layer that does not exist today looks a lot like the many high level architectural design languages that were hot in the 90s. Never heard of UML?
  - dagw 2y ago
    I bet there is an LLM based version of Rational Rose currently under development somewhere.
- rage4774 2y ago
  I was playing with the idea about setting up something like this!
- rixed 2y ago
  I've been thinking about this myself, although just planning to reuse what's already there. How do you picture the human in the loop?
ferguess_k 2y ago
I don't really like AI in IDE. I don't want them to think for me. Code completion and Intellisense is good enough. That said, I think there are 3 items that are important: - Quickly grasp a new framework or a new language. People might expect you to do so because of AI's help. 2 weeks might be the maximum, instead of the minimum. The same for juniors. - Focus on the real important things. So instead of trying to memorize a shell script you are going to use a couple of times per year, maybe use the time to learn something more fundamental. You can also use AI to help you to bootstrap the learning. If you need something for interviews, spend a week to memorize them. - Be willing to exclude AI from your thought process. If you rely AI on everything, including algorithms and designs, this might impact your understanding.
- all2 2y ago
  - Be willing to exclude AI from your thought process. If you rely AI on everything, including algorithms and designs, this might impact your understanding. Most of the time I'm using AI for problem space mapping (I'm doing dirt simple CRUD dev right now) and decomposition. It's ok at that, but even the deep research mode of Claude leaves some things to be desired. I feel like an editor now, more than an engineer. I know the kinds of things I'm looking for, and I use AI to walk a solution in. Either I use the output of the LLM as-is (for throwaway stuff) or I use it as a jumping off point for my own work _without_ the AI.
  - ferguess_k 2y ago
    >I feel like an editor now, more than an engineer. I know the kinds of things I'm looking for, and I use AI to walk a solution in. I agree. I think it's fine to do so. I usually prefer to write my code without AI (except for bootstrapping it). In my work as a DE, I mostly use AI to write scripts for me. For example, how to do this in PySpark? I kinda refused to memorize any of these because I'm simply not very interested, and I can always spend a week to memorize the fundamentals if I need. In my side projects, I use AI extensively. Same as you, I use AI for problem space mapping, or sort of. For example, I have some source code, how do I structure them better? I have read the MIDI standard and thought this piece of binary code means blah, can you please confirm for me? Well AI is OK for these kinds of work.
aschearer 2y ago
I've been using Claude to help write a complex prototype for game dev. Overall it's been a big productivity boost. However as the project has grown Claude has gotten much worse. I'm nearing 15k lines and it's borderline more trouble than it's worth. Even when it was helpful, it needed a _lot_ of guidance from me. Almost more helpful as a "rubber ducky" and for the fact that it kept me from deadlocking on analysis. That said, discussing problems and solutions with Claude often does keep things moving and sometimes reveals unexpected solutions. If Claude could write the code directly unsupervised, it would go wild and produce a ton of garbage. At least if the code it writes in the browser is any indication. It's not that it's all bad, but it's like a very eager junior dev -- potentially dangerous! Imagining a codebase that is one or two orders of magnitude larger, I think Claude would be useless. Imagining a non-expert driving the process, I think Claude would generate a very rickety proof of concept then fall over. All that said, I wish I had this tool when developing my previous game. Especially for a green field project, it feels like having access to the internet versus pulling reference manuals -- a big force multiplier.
- AndyNemmity 2y ago
  Why not just split the tasks, and have different projects only house a portion of the code?
  - danmur 2y ago
    Why dumb down your work worse because the machine can't understand it?
    - tomrod 2y ago
      Most people have a smaller context window than LLMs.
    - ChromaticPanic 2y ago
      Seems like the code is designed poorly if it can't be worked on in small chunks
svilen_dobrev 2y ago
i have mentored a few people to become Programmers. Some for months, some for years. It's like teaching someone to ride a bycicle. Hand-holding first, then hand-guiding, then short flights, then longer... Different people pick stuff at different pace and shape.. but they do learn.. if they want to. What i completely miss in these LLM parrots-agents-generators, is the learning. You can't teach them anything. They would not remember. Tabula rasa / Clean slate, every time. They may cite Shakespeare - or whatever code scrubbed from github - and concoct it to unrecognizability - but that's it. Hard rules or guardrails for every-little-thing are unsustainable to keep (and/or create) - expert-systems, rule-based no-code/low-code.. has been unsuccessful for decades). Maybe, next AI wave. And, there's no understanding. But that also applies to quite some people :/
- ebiester 2y ago
  Consider rules for projects. It's not always perfect, but it does adapt based on my instructions. For example, I have had good success in test first development as a rule. That means that I can make sure it has the specifications correct first.
- owebmaster 2y ago
  LLMs don't learn but agents do. You just need to insert that new knowledge in the prompt.
  - achierius 2y ago
    Agents are still LLMs. LLMs also have prompts. There's no difference!
    - trash_cat 2y ago
      Agents HAVE Models. There is a big difference if you give a model access to tools and perhaps some form of memory.
    - owebmaster 2y ago
      LLMs don't have prompts, you can use prompts to query LLMs. Agents are a mix of models, prompts, RAG and an event loop.
_1tem 2y ago
Developer skill is obviously still essential — you can’t steer if you couldn’t drive. But what about developer energy? Before AI I could only code about 2 hours per day (actual time spent writing code) but with Claude Code I can easily code for 5 hours straight without breaking a sweat. It feels like riding an e-bike instead of a bicycle. AI genuinely feels like Steve Jobs analogy of a bicycle for the mind — it doesn’t replace me but now I can go much farther and faster.
- senbrow 2y ago
  The ebike analogy is perfect! You still have to pedal, steer, and balance, but you're much faster overall.
- thom 2y ago
  Yeah this feels right. It helps with the bits I care about and definitely reduces friction there, but it also breaks through every wall when I’m off the happy path. I had to solve three layers of AWS bullshit yesterday to get back to training a model and if I’d had to solve them myself or bring in someone from my platform team I’d have ended up stopping short. I don’t really want to replace the parts of my job that I enjoy with AI, but I love having something to pick up the miserable bits when I’m stuck, confused, frustrated or bored. You’re absolutely right that this helps conserve energy.
- _puk 2y ago
  Love the ebike analogy! You're right though, you need to be able to steer, but you don't necessarily need to be able to map read. Case in point, I recently stood up my first project in supabase, cursor happily created the tables, secure RLS rules etc in a fraction of the time it would take me. To stop it getting spaghetti I had to add a rule "I'm developing a first version - add everything to an SQL file that tears down and recreates everything cleanly". This prevented hundreds of migration files being created, allowed me to retain and context, and every now and then ask "have you just made my database insecure", which 50:50 resulted in me learning something, or a "whoopsie, let me sort that". If I wasn't aware of this then it's highly likely my project would be full of holes. Maybe it still is, but ignorance is bliss (and 3 different LLMs can't be wrong can they?!)
ENGNR 2y ago
The opposite of vibe coding, when the agent craps out and you just do it manually = Artisanal coding. Yeah I can get on board with that.
lukev 2y ago
Along these lines, one thing Claude Code does consistently is to see a failing test and then add a conditional in the actual code to satisfy the test. I'm typically pretty gentle in real code reviews but that one is a serious "what the fuck are you even doing" if it were a human. Adding a top-level context-rule in claude.md doesn't fix it reliably.
kunzhi 2y ago
One thing that struck me after reading this article as well as all the comments here - we are providing excellent free training for the next generation of agents coming up. We are literally training our replacements! Like others I agree humans are not getting replaced anytime soon though. For all the current hype current AI technology is pretty dumb. Give it a decade or so though and everything we are currently doing will seem like Stone Age technology. Aside: sometimes I really wonder if humanity is trying to automate itself out of existence.
zombiwoof 2y ago
I see jr developers and managers not able to make the assessment the authors insight and experience provide and just checking in gobs of stuff they don’t understand It’s frustrating
andrewstuart 2y ago
“Agentic” coding? I suppose it is Thoughtworks after all working to expand mindshare by defining buzzwords.
jgilias 2y ago
> During refactoring, it failed to recognize the existing dependency injection chain Not sure if this is an argument against atheism or foxholes.
Garlef 2y ago
My central question after reading the article: Why did they choose a circle diagram over a pyramid?
- impjohn 2y ago
  I believe impact relates to area of circle as defined by the radius. A pyramid is a more hierarchical approach that perhaps was not suited. My 0.02$
nottorp 2y ago
Does anyone post articles about using these LLMs on non web dev? The internet is full of javascript/html/css info. Some wrong some obsolete some right and current, but there is data. How about the more peasant languages?
- Derbasti 2y ago
  Anecdotally, it does somewhat fine with C++ and OpenCV. Python is great, though PySide much less so. QML is almost useless. GLSL is OK, but not great. It generally can't reason well across language barriers, although it's great at converting between languages. The deeper into a nerdy domain I go, the less likely it is to understand what's going on. And more broadly, it seems that usefulness steeply declines for larger files and projects. It just can't fit enough context into its window to make sense of complicated things. As an extreme example, I told it to average two angles, and it just took the mean of them. Even after prompting, it thought nothing wrong of that. (The average of 1° and 359° is 0°, not 180°.) So it goes for many domains outside of webdev, UI, data science, scripting.
  - nyrikki 2y ago
    Even Python gets a bit touchy if you ask it to avoid common packages. An example is asking for simple Kalman filter, limiting to 2x2 matrix to avoid the need for LU decomposition. If you ad to the prompt a constraint to not use Numpy, which almost everything in the corpus does. Even with LRM's having a high enough top-k accuracy, so that at least one correct solution given in k guesses seems to be the trick. Perhaps Pyhon+Numpy is a language barrier but the errors without Numpy seem really trivial, similar to what one would see on an obscure language. It is different across different models, but getting stuck generating verification code with divide by zero to giving up and producing code that uses numpy are failure modes I have seen. Professor Subbarao Kambhampati's explanation really helps here IMHO. https://bsky.app/profile/rao2z.bsky.social/post/3lkjnrrv2qk2l https://bsky.app/profile/rao2z.bsky.social/post/3lkjnrrv2qk2... "Compiling the signal verifier" in, at least superficially to me, is a good intuition on where these fail. The limits of Top-K and heavy tail dependance in many tasks will be something painful, I think we will need more expertise and not less among programers just due to the failings of us humans and our over trust of automation etc... How we change the career path to develop tacit and technical abilities is a big question personally.
andai 2y ago
>Complicated build setups that confused both me and the AI itself. I noticed that I am capable of producing software beyond my own understanding. It wouldn't surprise me if the same is true of AI!
DeathArrow 2y ago
Imo, AI implements something useful 20% of the time while breaking existing code 80% of the time.
- nikolayasdf123 2y ago
  which AIs are you using? do you edit what they suggest yourself or just blindly copy-paste and hit run? do you ask them to write unit-tests + self-fix for what they written?
  - DeathArrow 2y ago
    I used Cursor, Augment Code, Roo Code, Aider, Cline, Github Copilot. With Claude and Deepseek and OpenAI backends. I let AI to modify files and I ask to write tests and fix them. Some tools can also run them. I am fiddling with tools like Cursor, Aider, Augment Code, Roo Code and LLMs like GPT, Sonnet, Grok, Deepseek to try to decide whether I can use AI for what I need, and if yes, identify some good workflows. I've read experiences of other people and tried my own ideas. I've burnt countless tokens, fast searches and US dollars. Working with AI for writing code is painful. It can break the code in ways you've never imagined and introduce bugs you never thought are possible. Unit testing and integration testing doesn't help much, because AI can break those, too. You can ask AI to run in loop, fixing compile errors, fixing tests, do builds, run the app and do API calls, to have the project building and tests passing. AI will be happy to do that, burning lots of dollars while at it. And after AI "fixes" the problem it introduced, you will still have to read every goddam line of the code to make sure it does what is supposed to. For greenfield projects, some people recommended crafting a very detailed plan with very detailed description and very detailed specs and feed that into the AI tool. AI can help with that, it asks questions I would never ask for an MVP and suggests stuff I would never implement for an MVP. Hurray, we have a very, very detailed plan, ready to feed into Cursor & Friends. Based on the very detailed plan, implementation takes few hours. Than, fixing compile errors and fixing failing tests takes a few more days. Then I manually test the app, see it has issues, look in the code to see where the issues can be. Make a list. Ask Cursor & Friends to fix issues one by one. They happily do it and they happily introduce compilation errors again and break tests again. So the fixing phase that last days begins again. Rinse and repeat until hopefully we spend a few weeks together (AI and I) instead on me building the MVP myself in half time. One tactic which seems a bit faster, is to just make a hierarchical tree of features, ask Cursor & Friends to implement a simple skeleton, then ask them to implement each feature, verifying myself the implementation after each step. For example, if I need to log in users, just ask to add logging in code, the ask to add an email sender service, then ask to add email verification code. Structuring the project using Vertical Slice Architecture and opening each feature folder in Cursor & Friends seems to improve the situation as the AI will have just enough context to modify or add something but can't break other parts of the code. I dislike that AI can introduce inconsistencies in code. I had some endpoint which used timestamps and AI used three different types for that DateTime, DateTimeOffset and long (UNIX time). It also introduced code to convert between the types and lots of bugs. The AI uses some folder structure for a part of the solution and other structure for other parts. It uses some naming conventions in some parts and other naming conventions in other parts. It uses multiple libraries for the same thing, like multiple JSON serializing libraries. It does things in a particular way in some parts of the application and in another way in other parts. It seems like tens of people are working in the same solution without anyone reading the code of the others. While asking AI to modify something, it will be very happy to modify things that you didn't ask to. I still need to figure out a good workflow, to reduce time and money spent, to reduce or eliminate inconsistency, to reduce bugs and compile errors. As an upside using AI to help with planning seems to be good, if I want to write the code myself, because the plan can be very thorough and I usually lack time and patience to make a very detailed plan.
securemepro 2y ago
[flagged]
codr7 2y ago
I don't get it, at all. Why are experienced developers so enthusiastic about chaining themselves to such an obviously crappy and unfulfilling experience? I like writing code and figuring stuff out, that's why I chose a career in software development in the first place.
- _acco 2y ago
  Why do we use hotkeys and snippets? There is a lot of tedium in software development and these tools help alleviate it.
  - wrasee 2y ago
    If all these tools do is wipe out all the tedium in software then we are all laughing. Happy days to come. There are tools and then there are the people holding the tools. The problem is no-one really knows which one AI is going to be.
- quest88 2y ago
  I don't enjoy the keypresses of building useful features. I like identifying what needs to be changed, and how, in abstract terms. These tools quickly help me verify those changes are right. If I need to call the VideoService to fetch some data, I don't want to spend time writing that and the tests that come with it. I'd rather outsource that part.
  - codr7 2y ago
    I don't object to abstracting, at all; or reducing labor in general. But this method of getting there makes me feel like I'm degraded to being the assistant and the machine is pulling my strings; and as a result I become dumber the more I do it, more dependent on crap tech.
  - deleted 2y ago
    [deleted]
- meowface 2y ago
  Because on net it often saves tons and tons of time and effort if you know how to use it. Luddism is a strange philosophy for a software engineer.
elif 2y ago
My solution to the authors babysitting problems is to close the iteration loop instead of embedding humans into it. Agent generated broken code? An agent can discover that, provide feedback on the pull request, and close it, forcing the coding agent to respond to the feedback. As long as you have 10 agents doing software engineering analysis for every 1 agent you have writing code, my suspicion is that a lot of this babysitting can be avoided. At least theoretically.. I haven't got all of this infrastructure linked myself to try.
FfejL 2y ago
> Example: When encountering a memory error during a Docker build, it increased the memory settings rather than questioning why so much memory was used in the first place. AI really is just like us!
nikolayasdf123 2y ago
is this just me, or none of this happens when using LLMs/AIs/Agents while coding? my experiences is overwhelmingly useful and positive, hard to imagine building software without LLM support anymore. I am literally 5~10x more productive.
croemer 2y ago
> I want to preface this by saying that AI tools are categorically and always bad at the things that I’m listing. I think there's a not missing there. Why would you preface that they are categorically and always bad? Makes more sense the other way round. Also grammar error "effected" instead of "affected" in the footer.
elric 2y ago
I'm already seeing developers spending more time communicating with their AI than with their team. I don't think that's a good evolution. Many of us aren't the best communicators, but it's a skill we typically polish as we become more senior. I worry about what will happen to junior devs who spend more time talking to/pairing with AI tham their human coworkers.
- globalise83 2y ago
  Upfront context, precise statements of what is required, immediate constructive feedback? What's not to like?
  - skydhash 2y ago
    Misunderstandings of specifications, lack of best or common practices, redoing stuff that was already tried by the team.
- kubanczyk 2y ago
  > junior devs Are there any? Honest question.
6thbit 2y ago
This diagram is painfully relatable — my team checks every single box here, and we don’t even use AI yet! Imagine when we finally do... "Misunderstood requirements" and "overly complex implementations" are practically our mascots at this point. We're slowly untangling this chaos through better upfront convos and iterative reviews, but man, habits die hard. Anyone else navigating these pitfalls totally unaided by AI?
npn 2y ago
Big words from one who does not code.
dudeinhawaii 2y ago
This article does not line up with my experiences at all. Sometimes I wonder if it's something to do with prompting or model selection. I recently built out a project where I was able to design 30+ modules and only had 4 generation errors. These were decent size modules of 700-5000 lines each. I would classify the generation errors as related to missing specification -- i.e., no you may not take an approach where you import another language runtime into memory to hack a solution. Sure, in the past, AI would lead me on goose chases, produce bad code, or otherwise fail. AI in 2025 though? No. AI has solved many quirky or complex headscratchers, async and distributed runtime bugs, etc. My error rate with Claude-3.7-sonnet and OpenAI's O3-mini has dropped to nearly zero. I think part of this is how you transfer your expert knowledge into the AI's "mindspace". I tend to prompt a paragraph which represents my requirements and constraints. Use this programming language. Cache in this way. Encrypt in this way. Prefer standard library. Use this or that algorithm. Search for the latest way to use this API and use it. Have this API surface. Etc. I'm not particularly verbose either. The thinking models tend to unravel that into a checklist, which they then run through and write a module for. "Ok, the user wants me to create a module that has these 10 features with these constraints and using these libraries." Maybe that's a matter of 25yrs of coding and being able to understand and describe the problem and all of its limits and constraints quickly but I find that I get one-shot success nearly every time. I'm not only laying out the specification, but I also have the overall spec in my mind and limit the AI to building modules to my specifications (apis/etc) rather than trying to shove all of this into context. Maybe that is the issue that some people have. Trying to shove everything (prior versions of the same code, etc) into one session. I always start brand new sessions for every core task or refactoring. "Let's add caching to this class that expires at X interval and is configurable from Y file and dependency injected to the constructor". So perhaps I'm unintentionally optimizing for AI but this fairly easy to do and has probably led to a 5-10x increase in code I'm pushing. Huge caveat here though, I mostly operate on service/backend/core lib/api code which is far less convoluted than web front-ends. It's kind of sad that front-end dev will require 100x context tokens due to intermingling of responsibilities, complex frameworks, etc. I don't envy people doing front-end dev work with AI.
deleted 2y ago
[deleted]

336 points by BerislavLopac 2y ago

> supervised agent This is the trick. Human in the loop, not human hiding in an ivory tower after uttering a single command. This is ~effectively what I see a lot of shops doing right now: "Clean up the codebase please. Apply best practices :D. OH. By the way, heres a laundry list of 100 things to NOT do: <list begins>". I get a lot more uplift out of use cases like: "Please generate a custom Stream implementation that is read-only and sources bytes from an underlying chunked representation. Mock the chunk loading part. Primarily demonstrate the ReadAsync method and transition logic between chunks."

usrbinbash 2y ago

Here is how I use it: As a writing assistant that lives in my IDE, and as a very very cool and sophisticated rubber duck that can answer me. Something I do quite a lot is throwing back and forth a discussion over a particular piece of code, usually provided with little to no context (because that's my task to worry about), hammering it until we get that functionality correct, then presenting it with broader context to fit it in (or I simply do that part by hand). Here is how I don't use it: As an agent that gets broad goals that he is supposed to fulfill on its own. Why? Because the time and effort I have to invest to ensure that the output of an agentic system is in line with what I actually try to accomplish, is simply too much, for all the reasons outlined in this excellent article. Ironically, this is even more true, since using AI as an incredibly capable writing assistant, already speeds up my workflow considerably. So in a way, less agentic AI empowers me in a way that makes me more critical of the additional time I'd have to invest to play around the quirks of agentic AI.

kubanczyk 2y ago
This doesn't parse for me: > a discussion over a particular piece of code [...] hammering it until we get that functionality correct Care to provide an example of sorts? > then presenting it with broader context to fit it in So after you have a function you might convert it to a method of a class. Stuff like that?
- usrbinbash 2y ago
  > Care to provide an example of sorts? For example, recently I needed to revise some code I wrote a few years back, re-implementing a caching mechanism to make it work across networked instances of the same software. I had a rough idea how I wanted to do that, and used an LLM to flesh out the idea. The conversation starts with an instruction that I don't want any code written until I ask for it, then I describe the problem itself, let it list the key points, and then present my solution (all still as prose, no code so far). Next step, I ask for its comments, and patterns/implementation details how to do that, as well as alternatives to those. This is the "design phase" of the conversation. Once we zoom in on a concrete solution, I ask it to produce a minimal example code of what we discussed. Then we repeat the same process, this time discussing details about the code itself. During that phase I tell it what parts of the implementation it doesn't t need to worry about and what to focus on, keeping it from going off to mock-up-wonderland. At the end it usually gets an instruction like "alright, please write out the code implementing what we discussed so far, in the same context as before". This gives me a starting point to work from. If the solution is fairly small, I might then give it some of the context this code will live in, and ask it to "fill in the blanks" as it were...often though I do that part myself, as its mostly small refactoring and renaming. What I find so useful about this workflow, as opposed to just throwing the thing at my project directory; it prevents the AI from getting side tracked, lost, as it were, in some detail, endlessly chasing its own tail trying to make sense of some compiler error. The human in the loop (yours truly), sets the stage, presents the focus, and the starting point is no code at all, just an ephemeral description of a problem and a discussion about it, grounding all the latter steps of the interaction. Hope that makes sense.

decompiled_dev 2y ago

I think of the modern developer being more like a shepherd rather than a builder now. You have to vibe with the machine, but you need to make sure they stay on easy terrain and give them structure instead of letting them simply free graze.

jvanderbot 2y ago

I'm surprised to not see an obvious one (for me): Use AI around the periphery. There's very often a heap of dev tools, introspection, logging, conversion, etc tools that need to be build and maintained. I've had a lot of luck using agents to make and fix these. For example a tool that collates data and logs in a bespoke planning system. It is a lot of generated boilerplate off the critical path to build these tools and I just don't want to do it most days.

aulin 2y ago
In my every day experience that's pretty risky. The periphery as you call it is often an area where you lack the expertise to spot and correct AI mistakes. I am thinking about build systems and shell scripts. I see people everyday going to AI before even looking at the docs and invariably failing with non-existent command line options, or worst options that break things in very subtle ways. Same people that when you tell them why don't you read the f-ing man page they go to google to look it up instead of opening a terminal. Same people that push through an unknown problem by trial and error instead of reading the docs first. But now they have this dumb counselor that steers them in the wrong direction most of the time and the whole process is even more error prone.
- jvanderbot 2y ago
  You're wrong. I have all the expertise but none of the time to generate 100s of lines of boilerplate API calls to get the data together, and no interest in formatting it correctly for consumption, let alone doing so state fully to allow interaction. These are trivial problems to solve that are highly tedious and do not affect whatsoever the business at hand. Perfect drudgery for automation, and just scanning the result is easy to verify the output or code.
  - aulin 2y ago
    I am not wrong. You simply are not the kind of developer I am thinking about. And believe me the other kind is way more represented.

mjr00 2y ago

> 15k LOC codebase I wish articles about AI assistance would caveat this at the start. 15k LOC is a weekend hackathon project, which is all well and good, but not reflective of the work that 99% of developers are doing in their day jobs.

jstanley 2y ago
I, too, habitually write 7500 lines of code per day.
- mjr00 2y ago
  For a brand new project that you're trying to get done in a weekend--aka a weekend hackathon project--it's very doable. Would not recommend committing 7500 lines of code per day at your day job though.
  - ariwilson 2y ago
    If you work 8 hours a day on Saturday and Sunday and each line has on average 40 characters, then on average you have to type about 5 characters a second to hit 7000 lines of code. There's some heavy assumptions about boilerplate or autogenerated code going on in that estimate, as I don't think very many average 5 characters a second over 16 hours.
    - Zanfa 2y ago
      40 seems way too much. “}” alone probably makes up a significant portion of lines. Another thing is auto-complete that does the majority of typing anyway.
    - Jtsummers 2y ago
      In English, the average word length is 5 characters. If you can type 60 WPM that means you're typing on average at least 300 characters per minute or at least 5 characters per second (at least because this is not counting the needed whitespace and punctuation). That makes it technically possible for a moderately capable typist to pull off 7000 lines in a day using your numbers. Pair that with IDEs and autocomplete and it becomes much more feasible, if you have a solid understanding of the objective for your code.

timdellinger 2y ago

I find that I have to steer the AI a lot, but I do have optimism that better prompting will lead to better agents. To take an example from the article: code re-use. When I'm writing code, I subconsciously have a mental inventory of what code is already there, and I'm subconsciously asking myself "hey, is this new task super similar to something that we already have working (and tested!) code for?". I haven't looked into the details of the initial prompt that a coding agent gets, but my intuition is that an addition to the prompt instructing the agent to keep an inventory of what's in the codebase, and when planning out a new batch of code, check the requirements of the new tasks against what's already there. Yes, this adds a bunch of compute cycles to the planning process, but we should be honest and say "that's just the price of an agent writing code". Better planning > ability to fix things.

hnuser123456 2y ago
There are certain pieces of text that appear right before some of the greatest pieces of code ever written. For example, we've all heard of NASA code requirements. If you get the LLM into the "mindset" of a top-tier professional developer before getting it to spit out code, the code quality will reflect that. If your prompt is sloppy and poorly defined, you'll get copy-pasted StackOverflow code, since that's how most SO questions look. If it's stupid but it works, it's not stupid. The hard part is that finding a local optimum for prompting style for one LLM may or may not transfer to another depending on personality post-training. And whatever style works best with all LLMs must be approaching some kind of optimum for using English to design and specify computer programs. We cannot have better programs without better program specifications.
- yaj54 2y ago
  Can you share some examples of these certain pieces of text and greatest pieces of code?
  - hnuser123456 2y ago
    Well, if you want safety-critical code, you could have the LLM read this before asking it to write its own code: https://ieeexplore.ieee.org/document/1642624 https://ieeexplore.ieee.org/document/1642624 GP was pondering about code re-use. My typical use involves giving an entire file to the LLM and asking the LLM to give the entire file back implementing requested changes, so that it's forced to keep the full text in context and can't get too off-track by focusing on small sections of code when related changes might be needed in other parts of the file. I think all of this is getting at the fact that an LLM won't spit out perfect code in response to a lazy prompt unless it's been highly post-trained to "reinterpret" sloppy prompts just as academically as academic prompts. Just like a human programmer, you can just give the programmer project descriptions and wait for the deliverable and accept it at face value, or you can join the programmer along their journey and verify their work is according to the standards you want. And sometimes there is no other way to get a hard project done. Conversely, sometimes you can give very detailed specifications and the LLM will just ignore part of them over and over. Hopefully the training experts can continue to improve that.

ikerino 2y ago

I use Cursor for most of my development these days. This article aligns pretty closely with my experiences. A few additional observations: 1. Anecdotally, AI agents feel stuck somewhere circa ~2021. If I install newer packages, Claude will revert to outdated packages/implementations that were popular four years ago. This is incredibly frustrating to watch and correct for. Providing explicit instructions for which packages to use can mitigate the problem, but it doesn't solve it. 2. The unpredictability of these missteps makes them particularly challenging. A few months ago, I used Claude to "one-shot" a genuinely useful web app. It was fully featured and surprisingly polished. Alone, I think it would've taken a couple weeks or weekends to build. But, when I asked it to update the favicon using a provided file, it spun uselessly for an hour (I eventually did it myself in a couple minutes). A couple days ago, I tried to spin up another similarly scoped web app. After ~4 hours of agent wrangling I'm ready to ditch the code entirely. 3. This approach gives me the brazenness to pursue projects that I wouldn't have the time, expertise, or motivation to attempt otherwise. Lower friction is exciting, but building something meaningful is still hard. Producing a polished MVP still demands significant effort. 4. I keep thinking about The Tortoise and The Hare. Trusting the AI agent is tempting because progress initially feels so much faster. At the end of the day, though, I'm usually left with the feeling I'd have made more solid progress with slower, closer attention. When building by hand, I rarely find myself backtracking or scrapping entire approaches. With an AI-driven approach, I might move 10x faster but throw away ~70% of the work along the way. > These experiences mean that by no stretch of my personal imagination will we have AI that writes 90% of our code autonomously in a year. Will it assist in writing 90% of the code? Maybe. Spot on. Current environment feels like the self-driving car hype cycle. There have been a lot of bold promises (and genuine advances), but I don't see a world in the next 5 years where AI writes useful software by itself.

Aurornis 2y ago
> It feels like my AI agents are stuck somewhere circa ~2021. If I install newer packages or more recent versions, Claude will often revert to outdated packages/implementations that were popular four years ago. My experience is the same, though the exact dates differ. I assume LLMs gravitate toward solutions that are most represented in their training material. It's hard to keep them pulled toward newer versions without explicitly mentioning it all the time.

osigurdson 2y ago

Is Martin Fowler now just renting out space on his website?

Apocryphon 2y ago
What are you referring to?
- Jtsummers 2y ago
  GP was apparently unaware until now that martinfowler.com has basically been a blog/article hosting site (though more in the highly curated sense than a generic hosting site, more akin to a trade publication) for the last couple decades and that not all the content is written by Martin Fowler himself. The author of this piece is Birgitta Böckeler.
  - bitwize 2y ago
    Kinda like rogerebert.com, even before the esteemed film critic's death.
onionbagle 2y ago
right!? It was a little misleading that the article was written by someone else. (Birgitta Böckeler)
Cupprum 2y ago
Martin Fowlers page is basically the place where Thoughtworks [1] places their articles. A lot of good stuff there on many different topics. 1: https://www.thoughtworks.com/ https://www.thoughtworks.com/
nikolayasdf123 2y ago
yeah, confusing. URL literally says martinfowler, and article written by someone else. what... :/

GiorgioG 2y ago

Great, so now instead of spending 8 hours writing code, I spend 8 hours "steering" an AI to write the same code. What a fucking win (for the AI companies and no one else.)

woah 2y ago
Then don't

johnjungles 2y ago

I actually think that most problems that took 5-20 mins are now a few minutes and it’s more about how many of those intense minutes and loops you’re going through. Also, right now engineers are hyper optimized in the code aspects but not thinking about the context into cursor and context out of cursor. Like the amount of copy paste from Notion / JIRA / Sentry and the amount of output like summarizing the git commits and PRs, slack and other “over communication” you have to do these days. This is the area I think we can more easily automate away.

adamgordonbell 2y ago

Lack of reuse AI-generated code sometimes lacks modularity, making it difficult to apply the same approach elsewhere in the application. Example: Not realising that a UI component is already implemented elsewhere, and therefore creating duplicate code. Example: Use of inline CSS styles instead of CSS classes and variables This is the big one I hit for sure. I think it's a problem with agentic RAG, where it only knows the files it's looked in and not the overall structure or where to look for things, so it just recreates them.

nikolayasdf123 2y ago
yep. noticed this too. but I think this is due to short context LLMs have. now my tools, mostly browser LLMs copy-paste code, are analysing small sub-problem at time. once LLMs will see whole repository + history of that repository, likely they will write abstractions that can be reused

jillesvangurp 2y ago

I use LLMs for various purposes in day to day development. I don't use any of the tools mentioned in the article because I'm using intellij and don't want to replace a tool that has lots of stuff that I use all the time. But aside from that, it's good advice and matches my experience. I've dabbled with plugins for intellij but wasn't really happy with those. But ever since chat gpt for desktop started interfacing directly with jetbrains products (and vs code as well), that's my goto tool. I realized that I like being able to pull that up with a simple keybinding and it auto connects to the IDE when I do. I don't need to replace my tools and I get to have AI support ready to go. Most of the existing plugins seem to insist on some crappy auto complete, which in a tool that offers a lot of auto complete features already is a bit of an anti feature. I don't need clippy style autocomplete. What matters here is the tool integration, not the model quality. Better tool integration means better prompts with less work and getting better answers that way. Example: I run a test, it fails with some output. I had this yesterday. So I asked, "why is this failing" and had a short discussion about what could be wrong. No need for me to specify any detail; all extracted from the IDE. We ticked off a few possible causes, I excluded them. And then it noticed a subtle change in the log messages that I had not noticed (a co-routine context switch) that turned out to be the root cause. That kind of open ended debugging is a bit of a mixed bag. Sometimes it finds stuff. Mostly it just starts proposing solutions based on a poor analysis of the problem. What works pretty reliably is: - address the TODOs / FIXMEs, especially if you give it some examples of what you expect - write documentation (very good for this) - evaluate if I covered all the edge cases (often finds stuff I want to fix) - simple code transformations (rewrite this using framework X instead of Y) I don't trust it blindly. But it's generally giving me good code and feedback. And I get to outsource a lot of the boring crap.

marstall 2y ago
Yes I find chatGPT/Jetbrains (RubyMine) in my case is the most usable setup I've encountered. It's like Rubymine is "home" for me - and chatGPT's macOS client has become another "home" for me so it's quite convenient that they talk to each other now. I have a little FOMO about Cursor though. ChatGPT will automatically apply its suggested changes to my open editor - but I have the sense Cursor will do a bit more? Apply changes to multiple files? And have knowledge of your whole project, not just open files? Can someone fill me in

fasteddie31003 2y ago

I've been playing around with vibe coding and I think a lot of the issues brought up could be fixed by an architecture abstraction layer that does not exist today. My idea would be something like an architecture-graph (Archigraph working title) that would recursively describe how an application works or should work. Then when an agentic coder is doing a task they can easily see the bigger picture of how an application works and hopefully writing better code. Anyone interested in working on this with me?

rvz 2y ago
"Vibe Coding" is not Software Engineering. It is "Slopware Engineering".
reztip 2y ago
I'd be interested to see how tools like Cursor/Windsurf and even Copilot are modeling this internally. This idea seems to be a natural extension to having a hierarchy/team of agents all working on a common goal. In this case I would imagine one agent serves as the architect or reviewer, and another generates code to meet spec, and they fight back and forth until converging/diverging.
rixed 2y ago
That abstraction layer that does not exist today looks a lot like the many high level architectural design languages that were hot in the 90s. Never heard of UML?
- dagw 2y ago
  I bet there is an LLM based version of Rational Rose currently under development somewhere.
rage4774 2y ago
I was playing with the idea about setting up something like this!
rixed 2y ago
I've been thinking about this myself, although just planning to reuse what's already there. How do you picture the human in the loop?

ferguess_k 2y ago

I don't really like AI in IDE. I don't want them to think for me. Code completion and Intellisense is good enough. That said, I think there are 3 items that are important: - Quickly grasp a new framework or a new language. People might expect you to do so because of AI's help. 2 weeks might be the maximum, instead of the minimum. The same for juniors. - Focus on the real important things. So instead of trying to memorize a shell script you are going to use a couple of times per year, maybe use the time to learn something more fundamental. You can also use AI to help you to bootstrap the learning. If you need something for interviews, spend a week to memorize them. - Be willing to exclude AI from your thought process. If you rely AI on everything, including algorithms and designs, this might impact your understanding.

all2 2y ago
- Be willing to exclude AI from your thought process. If you rely AI on everything, including algorithms and designs, this might impact your understanding. Most of the time I'm using AI for problem space mapping (I'm doing dirt simple CRUD dev right now) and decomposition. It's ok at that, but even the deep research mode of Claude leaves some things to be desired. I feel like an editor now, more than an engineer. I know the kinds of things I'm looking for, and I use AI to walk a solution in. Either I use the output of the LLM as-is (for throwaway stuff) or I use it as a jumping off point for my own work _without_ the AI.
- ferguess_k 2y ago
  >I feel like an editor now, more than an engineer. I know the kinds of things I'm looking for, and I use AI to walk a solution in. I agree. I think it's fine to do so. I usually prefer to write my code without AI (except for bootstrapping it). In my work as a DE, I mostly use AI to write scripts for me. For example, how to do this in PySpark? I kinda refused to memorize any of these because I'm simply not very interested, and I can always spend a week to memorize the fundamentals if I need. In my side projects, I use AI extensively. Same as you, I use AI for problem space mapping, or sort of. For example, I have some source code, how do I structure them better? I have read the MIDI standard and thought this piece of binary code means blah, can you please confirm for me? Well AI is OK for these kinds of work.

aschearer 2y ago

I've been using Claude to help write a complex prototype for game dev. Overall it's been a big productivity boost. However as the project has grown Claude has gotten much worse. I'm nearing 15k lines and it's borderline more trouble than it's worth. Even when it was helpful, it needed a _lot_ of guidance from me. Almost more helpful as a "rubber ducky" and for the fact that it kept me from deadlocking on analysis. That said, discussing problems and solutions with Claude often does keep things moving and sometimes reveals unexpected solutions. If Claude could write the code directly unsupervised, it would go wild and produce a ton of garbage. At least if the code it writes in the browser is any indication. It's not that it's all bad, but it's like a very eager junior dev -- potentially dangerous! Imagining a codebase that is one or two orders of magnitude larger, I think Claude would be useless. Imagining a non-expert driving the process, I think Claude would generate a very rickety proof of concept then fall over. All that said, I wish I had this tool when developing my previous game. Especially for a green field project, it feels like having access to the internet versus pulling reference manuals -- a big force multiplier.

AndyNemmity 2y ago
Why not just split the tasks, and have different projects only house a portion of the code?
- danmur 2y ago
  Why dumb down your work worse because the machine can't understand it?
  - tomrod 2y ago
    Most people have a smaller context window than LLMs.
  - ChromaticPanic 2y ago
    Seems like the code is designed poorly if it can't be worked on in small chunks

svilen_dobrev 2y ago

i have mentored a few people to become Programmers. Some for months, some for years. It's like teaching someone to ride a bycicle. Hand-holding first, then hand-guiding, then short flights, then longer... Different people pick stuff at different pace and shape.. but they do learn.. if they want to. What i completely miss in these LLM parrots-agents-generators, is the learning. You can't teach them anything. They would not remember. Tabula rasa / Clean slate, every time. They may cite Shakespeare - or whatever code scrubbed from github - and concoct it to unrecognizability - but that's it. Hard rules or guardrails for every-little-thing are unsustainable to keep (and/or create) - expert-systems, rule-based no-code/low-code.. has been unsuccessful for decades). Maybe, next AI wave. And, there's no understanding. But that also applies to quite some people :/

ebiester 2y ago
Consider rules for projects. It's not always perfect, but it does adapt based on my instructions. For example, I have had good success in test first development as a rule. That means that I can make sure it has the specifications correct first.
owebmaster 2y ago
LLMs don't learn but agents do. You just need to insert that new knowledge in the prompt.
- achierius 2y ago
  Agents are still LLMs. LLMs also have prompts. There's no difference!
  - trash_cat 2y ago
    Agents HAVE Models. There is a big difference if you give a model access to tools and perhaps some form of memory.
  - owebmaster 2y ago
    LLMs don't have prompts, you can use prompts to query LLMs. Agents are a mix of models, prompts, RAG and an event loop.

_1tem 2y ago

Developer skill is obviously still essential — you can’t steer if you couldn’t drive. But what about developer energy? Before AI I could only code about 2 hours per day (actual time spent writing code) but with Claude Code I can easily code for 5 hours straight without breaking a sweat. It feels like riding an e-bike instead of a bicycle. AI genuinely feels like Steve Jobs analogy of a bicycle for the mind — it doesn’t replace me but now I can go much farther and faster.

senbrow 2y ago
The ebike analogy is perfect! You still have to pedal, steer, and balance, but you're much faster overall.
thom 2y ago
Yeah this feels right. It helps with the bits I care about and definitely reduces friction there, but it also breaks through every wall when I’m off the happy path. I had to solve three layers of AWS bullshit yesterday to get back to training a model and if I’d had to solve them myself or bring in someone from my platform team I’d have ended up stopping short. I don’t really want to replace the parts of my job that I enjoy with AI, but I love having something to pick up the miserable bits when I’m stuck, confused, frustrated or bored. You’re absolutely right that this helps conserve energy.
_puk 2y ago
Love the ebike analogy! You're right though, you need to be able to steer, but you don't necessarily need to be able to map read. Case in point, I recently stood up my first project in supabase, cursor happily created the tables, secure RLS rules etc in a fraction of the time it would take me. To stop it getting spaghetti I had to add a rule "I'm developing a first version - add everything to an SQL file that tears down and recreates everything cleanly". This prevented hundreds of migration files being created, allowed me to retain and context, and every now and then ask "have you just made my database insecure", which 50:50 resulted in me learning something, or a "whoopsie, let me sort that". If I wasn't aware of this then it's highly likely my project would be full of holes. Maybe it still is, but ignorance is bliss (and 3 different LLMs can't be wrong can they?!)

ENGNR 2y ago

The opposite of vibe coding, when the agent craps out and you just do it manually = Artisanal coding. Yeah I can get on board with that.

lukev 2y ago

Along these lines, one thing Claude Code does consistently is to see a failing test and then add a conditional in the actual code to satisfy the test. I'm typically pretty gentle in real code reviews but that one is a serious "what the fuck are you even doing" if it were a human. Adding a top-level context-rule in claude.md doesn't fix it reliably.

kunzhi 2y ago

One thing that struck me after reading this article as well as all the comments here - we are providing excellent free training for the next generation of agents coming up. We are literally training our replacements! Like others I agree humans are not getting replaced anytime soon though. For all the current hype current AI technology is pretty dumb. Give it a decade or so though and everything we are currently doing will seem like Stone Age technology. Aside: sometimes I really wonder if humanity is trying to automate itself out of existence.

zombiwoof 2y ago

I see jr developers and managers not able to make the assessment the authors insight and experience provide and just checking in gobs of stuff they don’t understand It’s frustrating

andrewstuart 2y ago

“Agentic” coding? I suppose it is Thoughtworks after all working to expand mindshare by defining buzzwords.

jgilias 2y ago

> During refactoring, it failed to recognize the existing dependency injection chain Not sure if this is an argument against atheism or foxholes.

Garlef 2y ago

My central question after reading the article: Why did they choose a circle diagram over a pyramid?

impjohn 2y ago
I believe impact relates to area of circle as defined by the radius. A pyramid is a more hierarchical approach that perhaps was not suited. My 0.02$

nottorp 2y ago

Does anyone post articles about using these LLMs on non web dev? The internet is full of javascript/html/css info. Some wrong some obsolete some right and current, but there is data. How about the more peasant languages?

Derbasti 2y ago
Anecdotally, it does somewhat fine with C++ and OpenCV. Python is great, though PySide much less so. QML is almost useless. GLSL is OK, but not great. It generally can't reason well across language barriers, although it's great at converting between languages. The deeper into a nerdy domain I go, the less likely it is to understand what's going on. And more broadly, it seems that usefulness steeply declines for larger files and projects. It just can't fit enough context into its window to make sense of complicated things. As an extreme example, I told it to average two angles, and it just took the mean of them. Even after prompting, it thought nothing wrong of that. (The average of 1° and 359° is 0°, not 180°.) So it goes for many domains outside of webdev, UI, data science, scripting.
- nyrikki 2y ago
  Even Python gets a bit touchy if you ask it to avoid common packages. An example is asking for simple Kalman filter, limiting to 2x2 matrix to avoid the need for LU decomposition. If you ad to the prompt a constraint to not use Numpy, which almost everything in the corpus does. Even with LRM's having a high enough top-k accuracy, so that at least one correct solution given in k guesses seems to be the trick. Perhaps Pyhon+Numpy is a language barrier but the errors without Numpy seem really trivial, similar to what one would see on an obscure language. It is different across different models, but getting stuck generating verification code with divide by zero to giving up and producing code that uses numpy are failure modes I have seen. Professor Subbarao Kambhampati's explanation really helps here IMHO. https://bsky.app/profile/rao2z.bsky.social/post/3lkjnrrv2qk2l https://bsky.app/profile/rao2z.bsky.social/post/3lkjnrrv2qk2... "Compiling the signal verifier" in, at least superficially to me, is a good intuition on where these fail. The limits of Top-K and heavy tail dependance in many tasks will be something painful, I think we will need more expertise and not less among programers just due to the failings of us humans and our over trust of automation etc... How we change the career path to develop tacit and technical abilities is a big question personally.

andai 2y ago

>Complicated build setups that confused both me and the AI itself. I noticed that I am capable of producing software beyond my own understanding. It wouldn't surprise me if the same is true of AI!

DeathArrow 2y ago

Imo, AI implements something useful 20% of the time while breaking existing code 80% of the time.

nikolayasdf123 2y ago
which AIs are you using? do you edit what they suggest yourself or just blindly copy-paste and hit run? do you ask them to write unit-tests + self-fix for what they written?
- DeathArrow 2y ago
  I used Cursor, Augment Code, Roo Code, Aider, Cline, Github Copilot. With Claude and Deepseek and OpenAI backends. I let AI to modify files and I ask to write tests and fix them. Some tools can also run them. I am fiddling with tools like Cursor, Aider, Augment Code, Roo Code and LLMs like GPT, Sonnet, Grok, Deepseek to try to decide whether I can use AI for what I need, and if yes, identify some good workflows. I've read experiences of other people and tried my own ideas. I've burnt countless tokens, fast searches and US dollars. Working with AI for writing code is painful. It can break the code in ways you've never imagined and introduce bugs you never thought are possible. Unit testing and integration testing doesn't help much, because AI can break those, too. You can ask AI to run in loop, fixing compile errors, fixing tests, do builds, run the app and do API calls, to have the project building and tests passing. AI will be happy to do that, burning lots of dollars while at it. And after AI "fixes" the problem it introduced, you will still have to read every goddam line of the code to make sure it does what is supposed to. For greenfield projects, some people recommended crafting a very detailed plan with very detailed description and very detailed specs and feed that into the AI tool. AI can help with that, it asks questions I would never ask for an MVP and suggests stuff I would never implement for an MVP. Hurray, we have a very, very detailed plan, ready to feed into Cursor & Friends. Based on the very detailed plan, implementation takes few hours. Than, fixing compile errors and fixing failing tests takes a few more days. Then I manually test the app, see it has issues, look in the code to see where the issues can be. Make a list. Ask Cursor & Friends to fix issues one by one. They happily do it and they happily introduce compilation errors again and break tests again. So the fixing phase that last days begins again. Rinse and repeat until hopefully we spend a few weeks together (AI and I) instead on me building the MVP myself in half time. One tactic which seems a bit faster, is to just make a hierarchical tree of features, ask Cursor & Friends to implement a simple skeleton, then ask them to implement each feature, verifying myself the implementation after each step. For example, if I need to log in users, just ask to add logging in code, the ask to add an email sender service, then ask to add email verification code. Structuring the project using Vertical Slice Architecture and opening each feature folder in Cursor & Friends seems to improve the situation as the AI will have just enough context to modify or add something but can't break other parts of the code. I dislike that AI can introduce inconsistencies in code. I had some endpoint which used timestamps and AI used three different types for that DateTime, DateTimeOffset and long (UNIX time). It also introduced code to convert between the types and lots of bugs. The AI uses some folder structure for a part of the solution and other structure for other parts. It uses some naming conventions in some parts and other naming conventions in other parts. It uses multiple libraries for the same thing, like multiple JSON serializing libraries. It does things in a particular way in some parts of the application and in another way in other parts. It seems like tens of people are working in the same solution without anyone reading the code of the others. While asking AI to modify something, it will be very happy to modify things that you didn't ask to. I still need to figure out a good workflow, to reduce time and money spent, to reduce or eliminate inconsistency, to reduce bugs and compile errors. As an upside using AI to help with planning seems to be good, if I want to write the code myself, because the plan can be very thorough and I usually lack time and patience to make a very detailed plan.

securemepro 2y ago

[flagged]

codr7 2y ago

I don't get it, at all. Why are experienced developers so enthusiastic about chaining themselves to such an obviously crappy and unfulfilling experience? I like writing code and figuring stuff out, that's why I chose a career in software development in the first place.

_acco 2y ago
Why do we use hotkeys and snippets? There is a lot of tedium in software development and these tools help alleviate it.
- wrasee 2y ago
  If all these tools do is wipe out all the tedium in software then we are all laughing. Happy days to come. There are tools and then there are the people holding the tools. The problem is no-one really knows which one AI is going to be.
quest88 2y ago
I don't enjoy the keypresses of building useful features. I like identifying what needs to be changed, and how, in abstract terms. These tools quickly help me verify those changes are right. If I need to call the VideoService to fetch some data, I don't want to spend time writing that and the tests that come with it. I'd rather outsource that part.
- codr7 2y ago
  I don't object to abstracting, at all; or reducing labor in general. But this method of getting there makes me feel like I'm degraded to being the assistant and the machine is pulling my strings; and as a result I become dumber the more I do it, more dependent on crap tech.
- deleted 2y ago
  [deleted]
meowface 2y ago
Because on net it often saves tons and tons of time and effort if you know how to use it. Luddism is a strange philosophy for a software engineer.

elif 2y ago

My solution to the authors babysitting problems is to close the iteration loop instead of embedding humans into it. Agent generated broken code? An agent can discover that, provide feedback on the pull request, and close it, forcing the coding agent to respond to the feedback. As long as you have 10 agents doing software engineering analysis for every 1 agent you have writing code, my suspicion is that a lot of this babysitting can be avoided. At least theoretically.. I haven't got all of this infrastructure linked myself to try.

FfejL 2y ago

> Example: When encountering a memory error during a Docker build, it increased the memory settings rather than questioning why so much memory was used in the first place. AI really is just like us!

nikolayasdf123 2y ago

is this just me, or none of this happens when using LLMs/AIs/Agents while coding? my experiences is overwhelmingly useful and positive, hard to imagine building software without LLM support anymore. I am literally 5~10x more productive.

croemer 2y ago

> I want to preface this by saying that AI tools are categorically and always bad at the things that I’m listing. I think there's a not missing there. Why would you preface that they are categorically and always bad? Makes more sense the other way round. Also grammar error "effected" instead of "affected" in the footer.

elric 2y ago

I'm already seeing developers spending more time communicating with their AI than with their team. I don't think that's a good evolution. Many of us aren't the best communicators, but it's a skill we typically polish as we become more senior. I worry about what will happen to junior devs who spend more time talking to/pairing with AI tham their human coworkers.

globalise83 2y ago
Upfront context, precise statements of what is required, immediate constructive feedback? What's not to like?
- skydhash 2y ago
  Misunderstandings of specifications, lack of best or common practices, redoing stuff that was already tried by the team.
kubanczyk 2y ago
> junior devs Are there any? Honest question.

6thbit 2y ago

This diagram is painfully relatable — my team checks every single box here, and we don’t even use AI yet! Imagine when we finally do... "Misunderstood requirements" and "overly complex implementations" are practically our mascots at this point. We're slowly untangling this chaos through better upfront convos and iterative reviews, but man, habits die hard. Anyone else navigating these pitfalls totally unaided by AI?

npn 2y ago

Big words from one who does not code.

dudeinhawaii 2y ago

This article does not line up with my experiences at all. Sometimes I wonder if it's something to do with prompting or model selection. I recently built out a project where I was able to design 30+ modules and only had 4 generation errors. These were decent size modules of 700-5000 lines each. I would classify the generation errors as related to missing specification -- i.e., no you may not take an approach where you import another language runtime into memory to hack a solution. Sure, in the past, AI would lead me on goose chases, produce bad code, or otherwise fail. AI in 2025 though? No. AI has solved many quirky or complex headscratchers, async and distributed runtime bugs, etc. My error rate with Claude-3.7-sonnet and OpenAI's O3-mini has dropped to nearly zero. I think part of this is how you transfer your expert knowledge into the AI's "mindspace". I tend to prompt a paragraph which represents my requirements and constraints. Use this programming language. Cache in this way. Encrypt in this way. Prefer standard library. Use this or that algorithm. Search for the latest way to use this API and use it. Have this API surface. Etc. I'm not particularly verbose either. The thinking models tend to unravel that into a checklist, which they then run through and write a module for. "Ok, the user wants me to create a module that has these 10 features with these constraints and using these libraries." Maybe that's a matter of 25yrs of coding and being able to understand and describe the problem and all of its limits and constraints quickly but I find that I get one-shot success nearly every time. I'm not only laying out the specification, but I also have the overall spec in my mind and limit the AI to building modules to my specifications (apis/etc) rather than trying to shove all of this into context. Maybe that is the issue that some people have. Trying to shove everything (prior versions of the same code, etc) into one session. I always start brand new sessions for every core task or refactoring. "Let's add caching to this class that expires at X interval and is configurable from Y file and dependency injected to the constructor". So perhaps I'm unintentionally optimizing for AI but this fairly easy to do and has probably led to a 5-10x increase in code I'm pushing. Huge caveat here though, I mostly operate on service/backend/core lib/api code which is far less convoluted than web front-ends. It's kind of sad that front-end dev will require 100x context tokens due to intermingling of responsibilities, complex frameworks, etc. I don't envy people doing front-end dev work with AI.

deleted 2y ago

[deleted]