6 ms·

> This exact thing is what software developers have been begging for since the beginning of the profession: Receiving a detailed outline of the problem and what

> This exact thing is what software developers have been begging for since the beginning of the profession: Receiving a detailed outline of the problem and what the end result should look like. > This is often the part that slows down software development. Trying to figure out what a vague, title only, feature request actually means. But that is exactly what Software Engineering is!. It's 2026 and the notion that you can get detailed enough requirements and specifications that you can one-shot a perfect solution needs to die. In my experience AI has made us able to iterate on features or ideas much faster. Now most of the friction comes from alignment and coordination with other teams. My take is that to accelerate processes we should reduce coordination overhead and empower individuals and teams to make decisions and execute on them.

pron 5mo ago
> It's 2026 and the notion that you can get detailed enough requirements and specifications that you can one-shot a perfect solution needs to die. It's 2026 and the idea that even with detailed-enough requirements you can one-shot even a workable (let alone perfect) solution also needs to die. Anthropic failed to build even something as simple as a workable C compiler, not only with a perfect spec (and reference implementations, both of which the model trained on) but even with thousands of tests painstakingly written over many person-years. Today's models are not yet capable enough to build non-trivial production software without close and careful human supervision, even with perfect specs and perfect tests. Without a perfect spec and a perfect human-written test suite the task is even harder. Maybe in 2027.
- SirHumphrey 5mo ago
  Most software is much simpler than a c compiler.
  - pron 5mo ago
    A workable C compiler is a ~10-50KLOC program, and a fairly simple one at that (batch, with no concurrency or interaction). That Anthropic's swarm of agents wrote 100KLOC before failing is a symptom of the problem. It's certainly possible that many programs are in the sub 5KLOC range, but it's definitely not "most software". Plus, almost no software has this level of detailed spec, ready-made tests, and a selection of existing implementations of the same spec. My first thought when reading Anthropic's description of the experiment was that it is unrealistically easy. It's hard to come up with realistic jobs in the 10-50KLOC range that would be this easy for an LLM. That it failed only shows how much further we still have to go.
    - quantumleaper 5mo ago
      A bit off topic, but see how Anthropic publicity stunts went from "Claude C Compiler" with 100K LOC to the recent Bun Rust rewrite with 1M LOC (10x!) in just 3 months. I get that it's "novel" creation vs porting, but given that they reported that the C compiler cost them $20k in API costs, the Bun rewrite must be at least $200k, maybe even closer to a million. Pure madness.
      - pron 5mo ago
        Yes, the task is very different, but also it will be months to a year until we know the results of the bun experiment.
        quantumleaper 5mo ago
        I don't know how it could fail - Bun loses popularity among devs? Is it an objective metric? From what I understand, Node.js remains dominant across the industry as a whole, with Deno and Bun mostly used by startups. Anthropic can always fire the Opus/Mythos token machine gun on any problem (bugs, features, security) to ensure PR success, and there would be plenty of AI-sphere startups already drinking the kool-aid that would consider the whole vibe-coding thing to Bun's benefit.
        eudamoniac 5mo ago
        It could fail due to maintenance burden. There is a lot of code now that no one wrote.
        pron 5mo ago
        > Anthropic can always fire the Opus/Mythos token machine gun on any problem (bugs, features, security) to ensure PR success, Can they, though? They tried and failed to do it in their C compiler experiment. The experimenter wrote: "I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality."
        5mo ago
  - binary0010 5mo ago
    Not really. I can make a c compiler in a couple weeks just by looking up open source libraries and copying them. I can't make any software that people will pay me money to use without taking months/years of development, research, expiramentation and iteration. Just because the original people who invented compilers had to be genius, doesn't mean anyone has to spend much time or thought in copying that work now.
    - deleted 5mo ago
      [deleted]
    - YZF 5mo ago
      I built a compiler for a simpler language as part of my compilers course in a CS degree. It was a non-trivial exercise well beyond the majority of software applications. What open source libraries did you have in mind and what are you copying? If you can truly write a C compiler in weeks then kudos to you. How many compilers have you written so far for how many languages? I work for big tech and I would say a large % of developers are incapable of producing a working C compiler on any reasonable time scale, certainly not weeks, even with looking at open source. I'm sure they can download one and run it. Most developers today don't even know C or assembler. They don't know how to approach the C language spec. The top 5-10% of developers/engineers can do it but even for them it's non-trivial.
      - pron 5mo ago
        > It was a non-trivial exercise well beyond the majority of software applications That depends on how you count. By number of programs that may well be right, but that's not what matters in terms of impact on the industry, as software value roughly corresponds to the number of people working on a particular piece of software (or lines of code, if you wish). By number of people/LOC most software is not in the "simpler than a C compiler" category.
      - binary0010 5mo ago
        I'd copy and paste from all the thousands of open source ones, what do you mean? There are plenty of open source compilers that I can copy and paste whatever I need to. I don't get why you think this would have any level of difficulty? Of course I couldn't make a brand new compiler that was better than what's out there... Just like a game engine, I could clone one of the thousands of engines out there pretty easily - making something better or novel would be difficult. Just making a bare bones clone of what already exists by referencing documentation and pre-existing code is relatively easy now. Yeah, when I made a mediocre 3d game engine 20 years ago, it was brain breaking difficult work. I can make one infinitely better in a micro fraction of the time now because most of the hard stuff is done and can just be looked up now. Do you not agree?
- ianbutler 5mo ago
  Sorry where are we seeing that it failed? It compiled multiple projects successfully albeit less optimized. " It lacks the 16-bit x86 compiler that is necessary to boot Linux out of real mode. For this, it calls out to GCC (the x86_32 and x86_64 compilers are its own). It does not have its own assembler and linker; these are the very last bits that Claude started automating and are still somewhat buggy. The demo video was produced with a GCC assembler and linker. The compiler successfully builds many projects, but not all. It's not yet a drop-in replacement for a real compiler. The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled. The Rust code quality is reasonable, but is nowhere near the quality of what an expert Rust programmer might produce. " For faffing about with a multi agent system that seems like a pretty successful experiment to me. Source: https://www.anthropic.com/engineering/building-c-compiler https://www.anthropic.com/engineering/building-c-compiler Edit: Like I think people don't realize not even 7 months ago it wasn't writing this at all.
  - dnautics 5mo ago
    Yeah I think people are really underestimating what LLMs can do even without specs. As an example, I did an exploratory attempt to add custom software over some genuinely awful windows software for a scientific imaging station with a proprietary industrial camera. Five days later Claude and I had figured out how to USB-pcap sample images and it's operationalized and smoothly running for months now. 100% of the code written by Claude, it's all clean (reviewed it myself) pretty much all I did was unstuck it at a few places, "hey based on the file sizes it looks like the images are being sent as a 16-bit format") For day to day work, I'll often identify a bug, "hey, when I shift click on this graphical component, it's not doing the right thing". I go tell Claude to write a RED (failing) integration test, then make it pass. Zero lines of code manually written. Only occasionally do I have to intervene and rearchitect. Usually thus involves me writing about ten lines of scaffold code, explaining the architectural concept, and telling it to just go
    - pron 5mo ago
      People both underestimate and overestimate what LLMs can do. LLMs have shown very different results when autonomously writing a small program for personal use and autonomously writing production software that needs to be evolved for years.
- virgilp 5mo ago
  I wonder how knowledgeable in compilation was the engineer that attempted this. I'm pretty confident that I could produce a decent C compiler in a few weeks (or less), if given Opus 4.7 + unlimited tokens + a good test suite. (and this is not blind unsubstantiated belief in AI, I've recently rewritten a somewhat sophisticated interpreter in a week with AI; and have worked on several C++ compilers in the past, including a GCC port to a custom DSP, so I have a bit of an idea about what this would take). But yeah, this is not a "one shot" project, none of it is. One shot doesn't work even with humans - after all, this is exactly what killed waterfall as a methodology.
  - zem 5mo ago
    yeah, the key part is that there be a human in the loop, directing and course-correcting the ai while it produces code in reasonably small and well defined stages.
  - pron 5mo ago
    > I'm pretty confident that I could produce a decent C compiler in a few weeks (or less), if given Opus 4.7 + unlimited tokens + a good test suite. Of course. The point is that a full, detailed spec isn't enough (even in the rare situations it does exist, like for a C compiler). At least for the moment, you need expert humans to supervise and direct the agents. Vibe coders usually also let the agents write the tests, which mean that the only independent human validation of the software is some cursory manual inspection. That also obviously isn't enough to validate software. > One shot doesn't work even with humans - after all, this is exactly what killed waterfall as a methodology. You can one-shot a C compiler with humans. LLMs' software development ability is impressive and helpful, but it is not human-level yet, even if at some tasks the agents are better than most human programmers. And while many waterfall projects failed, many succeeded (although perhaps not as efficiently as they could have). So far I don't believe agents have been able to produce any non-trivial production software autonomously.
stingraycharles 5mo ago
Yeah I agree, such a fundamental aspect of software engineering is translating ambiguous “asks” into specific requirements. We now have a tool to convert those requirements directly into code. And yes, architecture and how to actually implement the designs are also part of the requirements. The code is just the implementation, the actual problem that needs solving is one abstraction level higher.
Cthulhu_ 5mo ago
I'm seeing decision-makers / people who write requirements starting to use AI as well in my day to day. As before, my job is to read, understand and test those requirements against the real world as I understand it. But same with code. Software engineering for the past (at least) 20 years has had a core focus of "don't trust anyone", this hasn't changed and this takes a lot of time and effort still.
- Terr_ 5mo ago
  The problem is that instead of trying to figure out what they really want/need, now we're trying to figure out what they really wanted or needed before it got obfuscated by the babble-machine.
harrall 5mo ago
Trying to figure out the best way to solve vague requirements is why I got into engineering. If I got detailed specs, I’d just be a coding robot. I push that work off onto juniors.
- hnthrow0287345 5mo ago
  If they can't at least imagine the golden path themselves and write it down, they shouldn't be in charge of the product because they will be unlikely to understand any other in depth conversations about it. And I have no idea how they'd be having coherent conversations with anyone above them either. They're also unlikely to use AI well or not identify bad-out-of-the-gate solutions. It is of course different if they're just gathering opinions or want a PoC or exploratory work done, but those aren't requirements to me. Developers are unlikely only doing development these days. There's ops and support to do as well, so more back and forth is less time doing those things and development. We need to meet in the middle about requirements otherwise developers will end up doing someone else's job for them.
ModernMech 5mo ago
It's UML and outsourcing all over again: If only we can write the perfect UML diagrams representing the ideal class hierarchy, we can just put that in an email, send it to India, then we'll get back exactly the program we wanted, no mistakes!
mmcnl 5mo ago
This is true, but funny thing is: it was also true before AI.
juanre 5mo ago
I completely agree. It's more than 40 years since I wrote my first program, and I've never seen software that was first specified and then written and all was good. The most difficult part of any non-trivial engineering is understanding the problem, and the first versions of a piece of software are how you reach that understanding. That's why I do not think that AI-powered "software factories" will ever work. It's waterfall development all over again. An architect writing UML diagrams and handing them off to the team of programmers to do the essentially mundane task of implementing... the wrong thing. AI is, however, very good at helping you go fast from the wrong first version to the less wrong second one. But you need to remember that your main task is to understand the problem that you are trying to solve.
- daxfohl 5mo ago
  Yeah and any detailed design is still likely to skip over "obvious" things like "only admin users can use admin features". Both the PM and the engineering team will understand this implicitly. But with AI, you never can tell if it's going to make that inference, or just create admin users and admin APIs with no relation between them. These are also the bugs that can most easily slip through, because the reviewer wouldn't even think to look for it.
- selicos 5mo ago
  > AI is, however, very good at helping you go fast from the wrong first version to the less wrong second one. For me using AI as an entry point into developing my own small tools for personal use, this is incredibly useful. I can start with a concept, fail miserably, then scrap it and try again a few times in an evening. And instead of brute forcing something over that time I can hand off analysis to an LLM and read about better solutions or learn more about what I'm trying to implement.
gedy 5mo ago
> Trying to figure out what a vague, title only, feature request actually means. > My take is that to accelerate processes we should reduce coordination overhead and empower individuals and teams to make decisions and execute on them. This is funny because it's exactly what the agile/scrum training taught me 20 years ago.
Philip-J-Fry 5mo ago
I don't agree. I regularly get pieces of work someone product guy has thought up in an afternoon. They only care about the happy path, and sometimes only part of the happy path. I work for a global company that has to abide by rules and regulations in each country we operate in. The product guy thinks up some feature, we implement the feature, then we're told "actually, we legally aren't allowed to do this in 90% of the markets we operate in". Cool, so we add an ability to disable it in those markets. Then they come back "We can do this in some of those markets if it's implemented with [regulatory bureaucracy], so can you do that please". Then we have to hack away at the solution because the deadline is right around the corner. This is not software engineering! None of this is related to the software. The job of a software engineer is to take a list of requirements and figure out the way we accomplish those requirements. Requirements gathering is NOT a software engineering problem. Software is implementation, product is behaviour. That's the split. The behaviour of the thing we're building needs to be known before we even try to seriously build it. If someone just held back for week and did their due diligence, we would been able to architect a solution that is scaleable, extensible, easy to maintain and can make the future easier.
- nuancebydefault 5mo ago
  > Requirements gathering is NOT a software engineering problem. Software is implementation, product is behaviour. That's the split. That's a theory but I've never seen this work in practice. A piece of software is unique. If it weren't, we'd just use the cp command. What usually happens is you get a set of requirements that looks simple. Then you start thinking about a design and see 10 different possibilities, each corresponding to a slightly different interpretation of the requirements set. You iterate a few times reviewing the designs with who set the requirements and a few peers and see more possible variations to the requirements. You need to double check its parent requirements up to the master requirements. Then you need to take time/feature/quality tradeoffs, affecting the fulfillment of requirements. Once starting to implement, you see dependencies to other software (framework, sdk, drivers, language features,...) and understand that other software is not what you thought, or has bugs. Or you see an issue with performance or see that one particular feature becomes unfeasible. That's where all the complexity goes. AI doesn't change that, but can make prototyping iterations and bug hunting faster, as long as someone holds it on a leash and understands its decisions.
thisisit 5mo ago
> we should reduce coordination overhead and empower individuals and teams to make decisions and execute on them. Improved collaboration. Says every new CEO and manager. The notion that this is ever going to be solved especially with different experience, views, agendas etc needs to die too. AI is surely not going to help and with that roadblock iterating faster doesn’t help because then people want to try just for trying.
BloondAndDoom 5mo ago
We will see smaller and smaller teams where all this overhead is minimized by handful of people, and more than ever we will see 2-5 people teams creating great software
thisisnotmyname 5mo ago
Not to mention that ai lets the domain experts create and test proof of concept implementations themselves. This alone has been a revelation for us and saves a tremendous number of design cycles.
getnormality 5mo ago
> Now most of the friction comes from alignment and coordination with other teams. Then I see a solution! Why don't we simply put the entire company on one big team?
- jimbokun 5mo ago
  Putting everyone responsible for some function of a product on one team, instead of having separate departments for separate functions, can do wonders for actually shipping and iterating on software.
  - hank9 5mo ago
    I don't remember who said it first but "software teams always ship their org chart" has always stuck with me. How a team is organized has such an outsized hand it what they build.
- necovek 5mo ago
  You joke but that's pretty much what cross functional teams are. The only other observation is that as you grow teams, communication channels multiply exponentially and at over 6-8 people communication starts breaking down. So instead you make small "companies" and set a few ground rules which software they build needs to follow, and you are back at a working org producing complex software.
jmalicki 5mo ago
This is also the part that AI speeds up the most for me, maybe 100x productivity. I start with something like this prompt: "This is a research project around <vague statement>. What do competitors, like <x>, <y>, <z> do around this, are there any blog posts or tech talks? Are there any academic approaches or recent papers around the topic? Can you survey any related open source projects? I know of <x> and <y>. Please include analysis of activity, github stars, number of downloads on npm/pypi/crates, and search the web for reviews or complaints or positive or negative blog posts from developers. All claims should have links to the original sources, preferably with quoted text where appropriate. We are going to write a research plan for how to produce this report. The implementation of the plan will spawn subagents to survey breadth, then spawn subagents for each depth topic in detail"
jimbokun 5mo ago
A previous iteration of my company had a CEO with a simple management idea that I believe worked really well: treat each product team as a mini startup. That means EVERY role needed to develop the product was in that team. No separate corporate wide QA function, infrastructure and operations function, sales function, project management function, or domain expertise function. All the people performing those functions for that project were part of the project team. Now this is somewhat hyperbole as if there is no sharing of resources whatsoever you don’t really have a single corporation. But the idea is clarifying and helps to eliminate silos and tighten communication and feedback loops. I miss that style of working. Although I try to break those barriers where I can as an individual contributor by just figuring out who needs to talk to who to make things happen and opening those channels of communication.
rerdavies 5mo ago
Nailed it. I suspect the OP is a waterfall guy (despite the token references to agile). All the references to documentation is a big clue. When I see "documentation" and "development" running in parallel, as if that's an extraordinary thing, I mentally cross out "documentation" and replace it with "input from stakeholders", which, in an agile world is... YES!! Of course those run in parallel. That's the whole point of agile. How do you translate "send an email to users" as a feature without a Document? ... also an incredibly waterfall thing. We Don't Do That Anymore. Thank goodness. Because it is incredibly inefficient (and not any less error-prone). And the chances that Some Guy who wrote the Document six months ago really understood the actual problem is...practically zero. One of my favorite waterfall stories. A friend of mine who does contract programming for <big company>, who said that her projects were always delivered exactly on time, so you never had to apply the "double the estimates rule". "So your projects always finish exactly on the delivery date original given?!" Incredulity! "Oh no. They usually take twice as long, but the difference is that, first we deliver what they asked for (which arrives exactly on the original schedule date, but is completely unusable); and then we charge them 3 times as much to deliver what they actually wanted (which takes twice as long)."
jwilliams 5mo ago
Agree. We have a bunch of tools - specs, code, tests. All of these really are models of the end outcome we're trying to capture. You could just build something, see if you're right and then build it again. If that seems ridiculous, what makes a spec special that it can work first time? Why we've not done this historically is code is annoying and (was) relatively expensive. You can rough out a spec document and get feedback from a wide variety of stakeholders -- after all, they can all read a document. If you can use AI to explore a problem space and get feedback directly, that's definitely a whole new tool in the kit.
nullsanity 5mo ago
[dead]
xnx 5mo ago
If you want to go fast, go alone. If you want to go far ... and fast go alone with AI.
_the_inflator 5mo ago
It is called product development. There is a reason why startups succeeded because the founders actually ate their own dog food as well as solved a problem that many felt. Products are no projects. Product development is so incredible hard. The reason why I like Steve Jobs is his nerdy personality, best shown in the iTunes key note. For two hours the guy plays around with the app, demonstrating what to do with it, how and why. Remember, this is the CEO. No wonder lower ranks had a hard time when they came up with ignorance about a product. Today’s so called Product Owners know next to nothing about their product nor craft. Design by committee. And this is what will will AI as bloated bottleneck: piling on a mess of documents only ever increasing context windows can barely make sense of. The lost art of lines deleted is amplified within the AI age.
jknoepfler 5mo ago
If by "much faster" you mean ~10% faster, then sure. Having actually measured gains across a few teams, I haven't found anything faster than that. And that's making no effort to amortize across the slowdowns introduced by shipping more bugs (which I also measurable, but is not necessarily damning. Yes, I'm well aware of the knee-jerk "then you need to use GenAI to write better tests!" - which, I will add, is more undiscovered, non-free work). Writing "80% of the code" is not saving "80% of the time". Code is usually trivial and writes itself when other problems are solved.
wek 5mo ago
The article is correct to emphasize the importance of definition and that this can be a bottleneck. But it is incorrect to show the layered documentation and development lines taking just as long as they did pre-coding-agents. Our team is coding much much faster at high quality then we were before coding agents. As this commenter says, if you empower individuals and small teams to define, document and they set up good practices and harnesses, they can accelerate dramatically.

pron 5mo ago

> It's 2026 and the notion that you can get detailed enough requirements and specifications that you can one-shot a perfect solution needs to die. It's 2026 and the idea that even with detailed-enough requirements you can one-shot even a workable (let alone perfect) solution also needs to die. Anthropic failed to build even something as simple as a workable C compiler, not only with a perfect spec (and reference implementations, both of which the model trained on) but even with thousands of tests painstakingly written over many person-years. Today's models are not yet capable enough to build non-trivial production software without close and careful human supervision, even with perfect specs and perfect tests. Without a perfect spec and a perfect human-written test suite the task is even harder. Maybe in 2027.

SirHumphrey 5mo ago
Most software is much simpler than a c compiler.
- pron 5mo ago
  A workable C compiler is a ~10-50KLOC program, and a fairly simple one at that (batch, with no concurrency or interaction). That Anthropic's swarm of agents wrote 100KLOC before failing is a symptom of the problem. It's certainly possible that many programs are in the sub 5KLOC range, but it's definitely not "most software". Plus, almost no software has this level of detailed spec, ready-made tests, and a selection of existing implementations of the same spec. My first thought when reading Anthropic's description of the experiment was that it is unrealistically easy. It's hard to come up with realistic jobs in the 10-50KLOC range that would be this easy for an LLM. That it failed only shows how much further we still have to go.
  - quantumleaper 5mo ago
    A bit off topic, but see how Anthropic publicity stunts went from "Claude C Compiler" with 100K LOC to the recent Bun Rust rewrite with 1M LOC (10x!) in just 3 months. I get that it's "novel" creation vs porting, but given that they reported that the C compiler cost them $20k in API costs, the Bun rewrite must be at least $200k, maybe even closer to a million. Pure madness.
    - pron 5mo ago
      Yes, the task is very different, but also it will be months to a year until we know the results of the bun experiment.
      - quantumleaper 5mo ago
        I don't know how it could fail - Bun loses popularity among devs? Is it an objective metric? From what I understand, Node.js remains dominant across the industry as a whole, with Deno and Bun mostly used by startups. Anthropic can always fire the Opus/Mythos token machine gun on any problem (bugs, features, security) to ensure PR success, and there would be plenty of AI-sphere startups already drinking the kool-aid that would consider the whole vibe-coding thing to Bun's benefit.
        eudamoniac 5mo ago
        It could fail due to maintenance burden. There is a lot of code now that no one wrote.
        pron 5mo ago
        > Anthropic can always fire the Opus/Mythos token machine gun on any problem (bugs, features, security) to ensure PR success, Can they, though? They tried and failed to do it in their C compiler experiment. The experimenter wrote: "I tried (hard!) to fix several of the above limitations but wasn’t fully successful. New features and bugfixes frequently broke existing functionality."
        5mo ago
- binary0010 5mo ago
  Not really. I can make a c compiler in a couple weeks just by looking up open source libraries and copying them. I can't make any software that people will pay me money to use without taking months/years of development, research, expiramentation and iteration. Just because the original people who invented compilers had to be genius, doesn't mean anyone has to spend much time or thought in copying that work now.
  - deleted 5mo ago
    [deleted]
  - YZF 5mo ago
    I built a compiler for a simpler language as part of my compilers course in a CS degree. It was a non-trivial exercise well beyond the majority of software applications. What open source libraries did you have in mind and what are you copying? If you can truly write a C compiler in weeks then kudos to you. How many compilers have you written so far for how many languages? I work for big tech and I would say a large % of developers are incapable of producing a working C compiler on any reasonable time scale, certainly not weeks, even with looking at open source. I'm sure they can download one and run it. Most developers today don't even know C or assembler. They don't know how to approach the C language spec. The top 5-10% of developers/engineers can do it but even for them it's non-trivial.
    - pron 5mo ago
      > It was a non-trivial exercise well beyond the majority of software applications That depends on how you count. By number of programs that may well be right, but that's not what matters in terms of impact on the industry, as software value roughly corresponds to the number of people working on a particular piece of software (or lines of code, if you wish). By number of people/LOC most software is not in the "simpler than a C compiler" category.
    - binary0010 5mo ago
      I'd copy and paste from all the thousands of open source ones, what do you mean? There are plenty of open source compilers that I can copy and paste whatever I need to. I don't get why you think this would have any level of difficulty? Of course I couldn't make a brand new compiler that was better than what's out there... Just like a game engine, I could clone one of the thousands of engines out there pretty easily - making something better or novel would be difficult. Just making a bare bones clone of what already exists by referencing documentation and pre-existing code is relatively easy now. Yeah, when I made a mediocre 3d game engine 20 years ago, it was brain breaking difficult work. I can make one infinitely better in a micro fraction of the time now because most of the hard stuff is done and can just be looked up now. Do you not agree?
ianbutler 5mo ago
Sorry where are we seeing that it failed? It compiled multiple projects successfully albeit less optimized. " It lacks the 16-bit x86 compiler that is necessary to boot Linux out of real mode. For this, it calls out to GCC (the x86_32 and x86_64 compilers are its own). It does not have its own assembler and linker; these are the very last bits that Claude started automating and are still somewhat buggy. The demo video was produced with a GCC assembler and linker. The compiler successfully builds many projects, but not all. It's not yet a drop-in replacement for a real compiler. The generated code is not very efficient. Even with all optimizations enabled, it outputs less efficient code than GCC with all optimizations disabled. The Rust code quality is reasonable, but is nowhere near the quality of what an expert Rust programmer might produce. " For faffing about with a multi agent system that seems like a pretty successful experiment to me. Source: https://www.anthropic.com/engineering/building-c-compiler https://www.anthropic.com/engineering/building-c-compiler Edit: Like I think people don't realize not even 7 months ago it wasn't writing this at all.
- dnautics 5mo ago
  Yeah I think people are really underestimating what LLMs can do even without specs. As an example, I did an exploratory attempt to add custom software over some genuinely awful windows software for a scientific imaging station with a proprietary industrial camera. Five days later Claude and I had figured out how to USB-pcap sample images and it's operationalized and smoothly running for months now. 100% of the code written by Claude, it's all clean (reviewed it myself) pretty much all I did was unstuck it at a few places, "hey based on the file sizes it looks like the images are being sent as a 16-bit format") For day to day work, I'll often identify a bug, "hey, when I shift click on this graphical component, it's not doing the right thing". I go tell Claude to write a RED (failing) integration test, then make it pass. Zero lines of code manually written. Only occasionally do I have to intervene and rearchitect. Usually thus involves me writing about ten lines of scaffold code, explaining the architectural concept, and telling it to just go
  - pron 5mo ago
    People both underestimate and overestimate what LLMs can do. LLMs have shown very different results when autonomously writing a small program for personal use and autonomously writing production software that needs to be evolved for years.
virgilp 5mo ago
I wonder how knowledgeable in compilation was the engineer that attempted this. I'm pretty confident that I could produce a decent C compiler in a few weeks (or less), if given Opus 4.7 + unlimited tokens + a good test suite. (and this is not blind unsubstantiated belief in AI, I've recently rewritten a somewhat sophisticated interpreter in a week with AI; and have worked on several C++ compilers in the past, including a GCC port to a custom DSP, so I have a bit of an idea about what this would take). But yeah, this is not a "one shot" project, none of it is. One shot doesn't work even with humans - after all, this is exactly what killed waterfall as a methodology.
- zem 5mo ago
  yeah, the key part is that there be a human in the loop, directing and course-correcting the ai while it produces code in reasonably small and well defined stages.
- pron 5mo ago
  > I'm pretty confident that I could produce a decent C compiler in a few weeks (or less), if given Opus 4.7 + unlimited tokens + a good test suite. Of course. The point is that a full, detailed spec isn't enough (even in the rare situations it does exist, like for a C compiler). At least for the moment, you need expert humans to supervise and direct the agents. Vibe coders usually also let the agents write the tests, which mean that the only independent human validation of the software is some cursory manual inspection. That also obviously isn't enough to validate software. > One shot doesn't work even with humans - after all, this is exactly what killed waterfall as a methodology. You can one-shot a C compiler with humans. LLMs' software development ability is impressive and helpful, but it is not human-level yet, even if at some tasks the agents are better than most human programmers. And while many waterfall projects failed, many succeeded (although perhaps not as efficiently as they could have). So far I don't believe agents have been able to produce any non-trivial production software autonomously.

stingraycharles 5mo ago

Yeah I agree, such a fundamental aspect of software engineering is translating ambiguous “asks” into specific requirements. We now have a tool to convert those requirements directly into code. And yes, architecture and how to actually implement the designs are also part of the requirements. The code is just the implementation, the actual problem that needs solving is one abstraction level higher.

Cthulhu_ 5mo ago

I'm seeing decision-makers / people who write requirements starting to use AI as well in my day to day. As before, my job is to read, understand and test those requirements against the real world as I understand it. But same with code. Software engineering for the past (at least) 20 years has had a core focus of "don't trust anyone", this hasn't changed and this takes a lot of time and effort still.

Terr_ 5mo ago
The problem is that instead of trying to figure out what they really want/need, now we're trying to figure out what they really wanted or needed before it got obfuscated by the babble-machine.

harrall 5mo ago

Trying to figure out the best way to solve vague requirements is why I got into engineering. If I got detailed specs, I’d just be a coding robot. I push that work off onto juniors.

hnthrow0287345 5mo ago
If they can't at least imagine the golden path themselves and write it down, they shouldn't be in charge of the product because they will be unlikely to understand any other in depth conversations about it. And I have no idea how they'd be having coherent conversations with anyone above them either. They're also unlikely to use AI well or not identify bad-out-of-the-gate solutions. It is of course different if they're just gathering opinions or want a PoC or exploratory work done, but those aren't requirements to me. Developers are unlikely only doing development these days. There's ops and support to do as well, so more back and forth is less time doing those things and development. We need to meet in the middle about requirements otherwise developers will end up doing someone else's job for them.

ModernMech 5mo ago

It's UML and outsourcing all over again: If only we can write the perfect UML diagrams representing the ideal class hierarchy, we can just put that in an email, send it to India, then we'll get back exactly the program we wanted, no mistakes!

mmcnl 5mo ago

This is true, but funny thing is: it was also true before AI.

juanre 5mo ago

I completely agree. It's more than 40 years since I wrote my first program, and I've never seen software that was first specified and then written and all was good. The most difficult part of any non-trivial engineering is understanding the problem, and the first versions of a piece of software are how you reach that understanding. That's why I do not think that AI-powered "software factories" will ever work. It's waterfall development all over again. An architect writing UML diagrams and handing them off to the team of programmers to do the essentially mundane task of implementing... the wrong thing. AI is, however, very good at helping you go fast from the wrong first version to the less wrong second one. But you need to remember that your main task is to understand the problem that you are trying to solve.

daxfohl 5mo ago
Yeah and any detailed design is still likely to skip over "obvious" things like "only admin users can use admin features". Both the PM and the engineering team will understand this implicitly. But with AI, you never can tell if it's going to make that inference, or just create admin users and admin APIs with no relation between them. These are also the bugs that can most easily slip through, because the reviewer wouldn't even think to look for it.
selicos 5mo ago
> AI is, however, very good at helping you go fast from the wrong first version to the less wrong second one. For me using AI as an entry point into developing my own small tools for personal use, this is incredibly useful. I can start with a concept, fail miserably, then scrap it and try again a few times in an evening. And instead of brute forcing something over that time I can hand off analysis to an LLM and read about better solutions or learn more about what I'm trying to implement.

gedy 5mo ago

> Trying to figure out what a vague, title only, feature request actually means. > My take is that to accelerate processes we should reduce coordination overhead and empower individuals and teams to make decisions and execute on them. This is funny because it's exactly what the agile/scrum training taught me 20 years ago.

Philip-J-Fry 5mo ago

I don't agree. I regularly get pieces of work someone product guy has thought up in an afternoon. They only care about the happy path, and sometimes only part of the happy path. I work for a global company that has to abide by rules and regulations in each country we operate in. The product guy thinks up some feature, we implement the feature, then we're told "actually, we legally aren't allowed to do this in 90% of the markets we operate in". Cool, so we add an ability to disable it in those markets. Then they come back "We can do this in some of those markets if it's implemented with [regulatory bureaucracy], so can you do that please". Then we have to hack away at the solution because the deadline is right around the corner. This is not software engineering! None of this is related to the software. The job of a software engineer is to take a list of requirements and figure out the way we accomplish those requirements. Requirements gathering is NOT a software engineering problem. Software is implementation, product is behaviour. That's the split. The behaviour of the thing we're building needs to be known before we even try to seriously build it. If someone just held back for week and did their due diligence, we would been able to architect a solution that is scaleable, extensible, easy to maintain and can make the future easier.

nuancebydefault 5mo ago
> Requirements gathering is NOT a software engineering problem. Software is implementation, product is behaviour. That's the split. That's a theory but I've never seen this work in practice. A piece of software is unique. If it weren't, we'd just use the cp command. What usually happens is you get a set of requirements that looks simple. Then you start thinking about a design and see 10 different possibilities, each corresponding to a slightly different interpretation of the requirements set. You iterate a few times reviewing the designs with who set the requirements and a few peers and see more possible variations to the requirements. You need to double check its parent requirements up to the master requirements. Then you need to take time/feature/quality tradeoffs, affecting the fulfillment of requirements. Once starting to implement, you see dependencies to other software (framework, sdk, drivers, language features,...) and understand that other software is not what you thought, or has bugs. Or you see an issue with performance or see that one particular feature becomes unfeasible. That's where all the complexity goes. AI doesn't change that, but can make prototyping iterations and bug hunting faster, as long as someone holds it on a leash and understands its decisions.

thisisit 5mo ago

> we should reduce coordination overhead and empower individuals and teams to make decisions and execute on them. Improved collaboration. Says every new CEO and manager. The notion that this is ever going to be solved especially with different experience, views, agendas etc needs to die too. AI is surely not going to help and with that roadblock iterating faster doesn’t help because then people want to try just for trying.

BloondAndDoom 5mo ago

We will see smaller and smaller teams where all this overhead is minimized by handful of people, and more than ever we will see 2-5 people teams creating great software

thisisnotmyname 5mo ago

Not to mention that ai lets the domain experts create and test proof of concept implementations themselves. This alone has been a revelation for us and saves a tremendous number of design cycles.

getnormality 5mo ago

> Now most of the friction comes from alignment and coordination with other teams. Then I see a solution! Why don't we simply put the entire company on one big team?

jimbokun 5mo ago
Putting everyone responsible for some function of a product on one team, instead of having separate departments for separate functions, can do wonders for actually shipping and iterating on software.
- hank9 5mo ago
  I don't remember who said it first but "software teams always ship their org chart" has always stuck with me. How a team is organized has such an outsized hand it what they build.
necovek 5mo ago
You joke but that's pretty much what cross functional teams are. The only other observation is that as you grow teams, communication channels multiply exponentially and at over 6-8 people communication starts breaking down. So instead you make small "companies" and set a few ground rules which software they build needs to follow, and you are back at a working org producing complex software.

jmalicki 5mo ago

This is also the part that AI speeds up the most for me, maybe 100x productivity. I start with something like this prompt: "This is a research project around <vague statement>. What do competitors, like <x>, <y>, <z> do around this, are there any blog posts or tech talks? Are there any academic approaches or recent papers around the topic? Can you survey any related open source projects? I know of <x> and <y>. Please include analysis of activity, github stars, number of downloads on npm/pypi/crates, and search the web for reviews or complaints or positive or negative blog posts from developers. All claims should have links to the original sources, preferably with quoted text where appropriate. We are going to write a research plan for how to produce this report. The implementation of the plan will spawn subagents to survey breadth, then spawn subagents for each depth topic in detail"

jimbokun 5mo ago

A previous iteration of my company had a CEO with a simple management idea that I believe worked really well: treat each product team as a mini startup. That means EVERY role needed to develop the product was in that team. No separate corporate wide QA function, infrastructure and operations function, sales function, project management function, or domain expertise function. All the people performing those functions for that project were part of the project team. Now this is somewhat hyperbole as if there is no sharing of resources whatsoever you don’t really have a single corporation. But the idea is clarifying and helps to eliminate silos and tighten communication and feedback loops. I miss that style of working. Although I try to break those barriers where I can as an individual contributor by just figuring out who needs to talk to who to make things happen and opening those channels of communication.

rerdavies 5mo ago

Nailed it. I suspect the OP is a waterfall guy (despite the token references to agile). All the references to documentation is a big clue. When I see "documentation" and "development" running in parallel, as if that's an extraordinary thing, I mentally cross out "documentation" and replace it with "input from stakeholders", which, in an agile world is... YES!! Of course those run in parallel. That's the whole point of agile. How do you translate "send an email to users" as a feature without a Document? ... also an incredibly waterfall thing. We Don't Do That Anymore. Thank goodness. Because it is incredibly inefficient (and not any less error-prone). And the chances that Some Guy who wrote the Document six months ago really understood the actual problem is...practically zero. One of my favorite waterfall stories. A friend of mine who does contract programming for <big company>, who said that her projects were always delivered exactly on time, so you never had to apply the "double the estimates rule". "So your projects always finish exactly on the delivery date original given?!" Incredulity! "Oh no. They usually take twice as long, but the difference is that, first we deliver what they asked for (which arrives exactly on the original schedule date, but is completely unusable); and then we charge them 3 times as much to deliver what they actually wanted (which takes twice as long)."

jwilliams 5mo ago

Agree. We have a bunch of tools - specs, code, tests. All of these really are models of the end outcome we're trying to capture. You could just build something, see if you're right and then build it again. If that seems ridiculous, what makes a spec special that it can work first time? Why we've not done this historically is code is annoying and (was) relatively expensive. You can rough out a spec document and get feedback from a wide variety of stakeholders -- after all, they can all read a document. If you can use AI to explore a problem space and get feedback directly, that's definitely a whole new tool in the kit.

nullsanity 5mo ago

[dead]

xnx 5mo ago

If you want to go fast, go alone. If you want to go far ... and fast go alone with AI.

_the_inflator 5mo ago

It is called product development. There is a reason why startups succeeded because the founders actually ate their own dog food as well as solved a problem that many felt. Products are no projects. Product development is so incredible hard. The reason why I like Steve Jobs is his nerdy personality, best shown in the iTunes key note. For two hours the guy plays around with the app, demonstrating what to do with it, how and why. Remember, this is the CEO. No wonder lower ranks had a hard time when they came up with ignorance about a product. Today’s so called Product Owners know next to nothing about their product nor craft. Design by committee. And this is what will will AI as bloated bottleneck: piling on a mess of documents only ever increasing context windows can barely make sense of. The lost art of lines deleted is amplified within the AI age.

jknoepfler 5mo ago

If by "much faster" you mean ~10% faster, then sure. Having actually measured gains across a few teams, I haven't found anything faster than that. And that's making no effort to amortize across the slowdowns introduced by shipping more bugs (which I also measurable, but is not necessarily damning. Yes, I'm well aware of the knee-jerk "then you need to use GenAI to write better tests!" - which, I will add, is more undiscovered, non-free work). Writing "80% of the code" is not saving "80% of the time". Code is usually trivial and writes itself when other problems are solved.

wek 5mo ago

The article is correct to emphasize the importance of definition and that this can be a bottleneck. But it is incorrect to show the layered documentation and development lines taking just as long as they did pre-coding-agents. Our team is coding much much faster at high quality then we were before coding agents. As this commenter says, if you empower individuals and small teams to define, document and they set up good practices and harnesses, they can accelerate dramatically.