10 ms·

The economics of software teams: Why most engineering orgs are flying blind

• SpicyLemonZest 6mo ago
  > The obvious objection is that code produced at that speed becomes unmanageable, a liability in itself. That is a reasonable concern, but it largely applies when agents produce code that humans then maintain. Agentic platforms are being iterated upon quickly, and for established patterns and non-business-critical code, which is the majority of what most engineering organizations actually maintain, detailed human familiarity with the codebase matters less than it once did. A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves. I keep seeing this assumption that "unmanageable" caps out at "kinda hard to reason about", and anyone with experience in large codebases can tell you that's not so. There are software components I own today which require me to routinely explain to junior engineers (and indeed to my own instances of Claude) why their PR is unsound and I won't let them merge it no matter how many tests they add.
  • snowe2010 6mo ago
    Yeah this really breaks down when you put the logic up against ANY sort of compliance testing. Ok you don’t meet compliance, your agents have spent weeks on it and they’re just adding more bugs. Now what are you going to do? You have to go into the code yourself. Uh oh.
• leokennis 6mo ago
  > The obvious objection is that code produced at that speed becomes unmanageable, a liability in itself. That is a reasonable concern, but it largely applies when agents produce code that humans then maintain. Agentic platforms are being iterated upon quickly, and for established patterns and non-business-critical code, which is the majority of what most engineering organizations actually maintain, detailed human familiarity with the codebase matters less than it once did. A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves. Then I'd wager it's the same for the courses and workshop this guy is selling...an LLM can probably give me at least 75% of the financial insights for not even .1% of what this "agile coach" is asking for his workshops and courses. Maybe the "agile coach LLM" can explain to the "coding LLM's" why they're too expensive, and then the "coding LLM's" can tell the "agile coach LLM" to take the next standby shift then, if he knows so much about code? And then we actual humans can have a day off and relax at the pool.
  • pydry 6mo ago
    Exactly. I think it's been a while since I've read an LLM hot take which couldnt have been written by an LLM and this one is no exception. There's a 99% chance that the training materials on sale are equally replaceable with a prompt.
    • kaon_2 6mo ago
      True. And yet, as an organization when you buy OP's training, you don't buy the material. You buy the feeling that you make your organization becomes more productive. You buy the signal to your boss that you are innovative and working to make your organization more productive. And you buy the time and headspace from your engineers that they are thinking if at least for 2 hours about making the organization more productive. The latter can be well worth the cost, and the former surely too.
      • pydry 6mo ago
        They're buying a defensible (or laudable) justification when the training company's fee appears as a line item in the company budget. This doesnt mean the training has to be good, useful or original in the slightest but the provider does need to have credentials which arent just "some dev with a hot take" that a fellow executive would recognize.
  • bonesss 6mo ago
    Ceding the premise that the AGI is gonna eat my job, my job involves reading the spec to be able verify the code and output so the there’s a human to fire and sue. There are five layers of fluffy management and corporate BS before we get to that part, and the AGI is more competent at those fungible skills. With the annoying process people out of the picture, even reviewing vibeslop full time sounds kinda nice… Feet up, warm coffee, just me and my agents so I can swear whenever I need to. No meetings, no problems.
    • taurath 6mo ago
      There’s gonna be one guy in charge of you, and he’s going to expect you to be putting out 20x output while thanking him for the privilege of being employed, assuming all goes the way every management team seems to want I dont think this will happen because AI has become a straight up cult and things that are going well don’t need so many people performatively telling each other how well things are going.
• jiusanzhou 6mo ago
  The 3-5x return threshold is the part most eng leaders never internalize. I've seen teams spend entire quarters on internal tooling that saves maybe 20 minutes per developer per week — nowhere near break-even, let alone a healthy return. The uncomfortable truth is that most prioritization frameworks (RICE, WSJF, etc.) deliberately avoid dollar amounts because nobody wants to see the math on their pet project. Once you attach real costs to sprint decisions, half the roadmap becomes indefensible.
  • cpinto 6mo ago
    You’re absolutely right, but just to a point. It should be easy to clearly quantify the desired financial outcome of a sprint, but not of its components. I don’t want to spend a single minute figuring out the financial outcome of a single ticket.
  • olsondv 6mo ago
    On the other hand, I’ve also seen single developers create a tool or dashboard off-the-books that had widespread adoption. Things that would never have breached the top 100 features list since they are entirely internal. The irony is then they are expected to maintain it indefinitely without official effort allocation.
• lynx97 6mo ago
  Using ‘blind’ to mean ‘ignorant’ is like using any disability label as a synonym for ‘bad’—it turns a real condition into an insult.
  • Smaug123 6mo ago
    "Flying blind" is a completely standard idiom originating from flying while blinded by e.g. cloud or darkness. Its meaning is a figurative transplant of a literal description.
    • lynx97 6mo ago
      I know it’s an idiom. The point is that it still uses blindness as a stand-in for incompetence/unsafe guessing. Being common doesn’t make it harmless. Common just means we’ve normalized it. And you defending it shows that weve normalized it to a point where the double-meaning is seemingly only apparent to blind people.
      • anonymous908213 6mo ago
        It absolutely does not use blindness as a stand-in for incompetence, that is your own outrage-seeking interpretation of it. A neutral interpretation would be that "flying blind" is to "operate without perfect information". It is a simple description of operating conditions, not a derogatory term in any way. Your reply is worded in such a way as to indicate that you think the person you're replying to deserves to be shamed for 'defending' it, but having a disability does not entitle you to browbeat the world into submission and regulate all usage of any words associated with your disability as you see fit. This is quite benign and people are perfectly well within their right to object to somebody trying to police plainly descriptive language.
        • tom_ 6mo ago
          Your reply would be much improved if it were just this part. > A neutral interpretation would be that "flying blind" is to "operate without perfect information". It is a simple description of operating conditions, not a derogatory term in any way. Entering it would also have put less wear and tear on the input device.
• consp 6mo ago
  The estimate cost number is for very large companies with massive overhead bulk. Dump the management overhead, the HR machine and other things smaller companies do not have and this number comes down massively.
• InfinityByTen 6mo ago
  When I see someone just throwing a lot of numbers and graphs at me, I see that there are in to win an argument, and not propose an idea. Of late, I've come across a lot of ideas from Rory Sutherland and my conclusion from listening to his ideas is that there are some people, who're obsessed with numbers, because to them it's a way to find certainty and win arguments. He calls them "Finance People" (him being a Marketing one). Here's an example "Finance people don’t really want to make the company money over time. They just thrive on certainty and predictability. They try to make the world resemble their fantasy of perfect certainty, perfect quantification, perfect measurement. Here’s the problem. A cost is really quantifiable and really visible. And if you cut a cost, it delivers predictable gains almost instantaneously." > Choosing to spend three weeks on a feature that serves 2% of users is a €60,000 decision. I'd really want to hire the Oracle of a PM/ Analyst that can give me that 2% accurately even 75% of the time, and promise nothing non-linear can come from an exercise.
  • tweetle_beetle 6mo ago
    As with most things, isn't the truth somewhere in the middle? True cost/value is very hard to calculate, but we could all benefit by trying a bit harder to get closer to it. It's all too common to frame the tension as binary: bean counters vs pampered artistes. I've seen it many times and it doesn't lead anywhere useful.
    • SpicyLemonZest 6mo ago
      Here I think the truth is pretty far to one side. Most engineering teams work at a level of abstraction where revenue attribution is too vague and approximate to produce meaningful numbers. The company shipped 10 major features last quarter and ARR went up $1m across 4 new contracts using all of them; what is the dollar value of Feature #7? Well, each team is going to internally attribute the entire new revenue to themselves, and I don’t know what any other answer could possibly look like.
      • Fripplebubby 6mo ago
        Even if you could do attribution correctly (I think you can do this partially if you are really diligent about A/B testing), that is still only one input to the equation. The other fact worth considering is the scale factor - if a team develops a widget which has some ARR value today, that same widget has a future ARR value that scales with more product adoption - no additional capital required to capture more marginal value. How do you quantify this? Because it is hard and recursive (knowing how valuable a feature will be in the future means knowing how many users you have in the future which depends on how valuable your features are as well as 100 other factors), we just factor this out and don't attempt to quantify things in dollars and euros.
• lknuth 6mo ago
  Making it solely about the extraction of dollars is a great recipe to make something mediocre. See Hollywood or Microslop. Its like min-maxing a Diablo build where you want the quality of the product to be _just_ above the "acceptable" threshold but no higher because that's wasting money. Then, you're free to use all remaining points to spec into revenue.
  • cmarot 6mo ago
    Exactly. In addition, sometimes a good software "only" makes you save 1% of your time, but that 1% was a terrible burden that induced mental fatigue, made you take bad decisions, etc. It can even make a great Engineer stay when he would have left with the previous version.
    • devmor 6mo ago
      While reading the article I was thinking the same thing. I can think of problems I've solved that directly affected 0% of our customers, but overloaded our customer support team.
• jaccola 6mo ago
  I think the only thing that matters is whether the people on the team care deeply about the product; whether they care more about the product than their own careers (in the short term). Without that, any metric or way of thinking can and will be gamed. Unfortunately, even with all the management techniques in the world, there are just some projects that are impossible to care about. There’s simply a significantly lower cap on productivity on these projects.
• petetnt 6mo ago
  > This does not mean that Slack’s engineering investment was wasted, because Slack also built enterprise sales infrastructure, compliance capabilities, data security practices, and organizational resilience that a fourteen-day prototype does not include. The LLM-agent team argument also misses the core point that the engineering investment (which actually encompasses business decisions, design and much more than just programming) is what actually got Slack (or any other software product) to the point where is it is now and where it's going in the future and creating a snapshot of the current status is, while maybe not absolutely trivial, still just a tiny fraction of the progress made over the years.
• ares623 6mo ago
  The "author" used someone's vibecoded Slack clone to justify his conclusions. I think he believes that the majority of Slack's value lies in the slick CSS animations. I do agree with his thesis in the middle, about how the ZIRP decade and the cultures that were born from that period were outrageous and cannot survive the current era. It's a brave new world, and it's not because of AI. It's because there's just not enough money flowing anymore, and what little is left is sucked up by the big boys (AI).
• ensocode 6mo ago
  [dead]
• tgdn 6mo ago
  I get "This site can’t be reached"
  • tonyedgecombe 6mo ago
    You haven't missed much.
• dsign 6mo ago
  [dead]
• boron1006 6mo ago
  > A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. I’ve been on 2 failed projects that have been entirely AI generated and it’s not that agents slow down and you can just send more agents to work on projects for longer, it’s that they becoming completely unable to make any progress whatsoever, and whatever progress they do make is wrong.
  • iamflimflam1 6mo ago
    Very much like humans when they drown in technical debt. I think the idea that a messy codebase can be magically fixed is laughable. What I might believe though is that agents might make rewrites a lot more easy. “Now we know what we were trying to build - let’s do it properly this time!”
    • Cthulhu_ 6mo ago
      Potentially, yes, but as with other software, you need to know AND have (automated) verifications on what it does, exactly. And of course, make the case that it actually needs a rewrite, instead of maintenance. See also second-system effect.
      • tonyedgecombe 6mo ago
        >And of course, make the case that it actually needs a rewrite, instead of maintenance. "The AI said so ..."
      • ben_w 6mo ago
        > Potentially, yes, but as with other software, you need to know AND have (automated) verifications on what it does, exactly. Yes, but even here one needs some oversight. My experiments with Codex (on Extra High, even) was that a non-zero percentage of the "tests" involved opening the source code (not running it, opening it) and regexing for a bunch of substrings.
    • 6mo ago
• deleted 6mo ago
  [deleted]
• sdevonoes 6mo ago
  Still don’t understand what regular people (like the author) gain from selling how wonderful AI is. I get that the folks at Anthropic and openai shove AI through our throats every day, but nobodies?
  • csomar 6mo ago
    He is selling consulting around AI/LLM.
    • febusravenga 6mo ago
      In other words, he's cutting branch he's sitting on.
      • marcosdumay 6mo ago
        That would only be a problem if his saw could actually cut wood.
  • deleted 6mo ago
    [deleted]
• danpalmer 6mo ago
  > even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today Citation needed. A human engineer can grok a lot in 10 days, and an agent can spend a lot of tokens in 10 days.
• ozim 6mo ago
  Then let's disregard cost of running and maintaining a system for having exact financial feedback. We do proxy measurements because having exact data is hard because there is more to any feature than just code. Feature is not only code, it is also customer training, marketing - feature might be perfectly viable from code perspective but then utterly fail in adoption for reasons beyond of Product Owner control. What I saw in comments — author is selling his consultancy/coaching and I see in comments that people who have any real world experience are also not buying it.
• barrkel 6mo ago
  The argument against platform teams needs to be balanced with the compounding nature of technical debt. The argument to always go for the biggest return works OK for the first few years of high growth (though the timeline is probably greatly compressed the more you use AI), but it turns into a kind of quicksand later.
• bob1029 6mo ago
  I don't understand the urgency around quantifying every aspect of the software process. Surely, we are in agreement that money in must at least equal money out if the company is to be viable? This is a simple quickbooks report, is it not? Why don't we instead focus our energies on the customer and then work our way backward into the technology. There are a lot of ways to solve problems these days. But first you want to make sure you are solving the right problem. Whether or not your solution represents a "liability" or an "asset" is irrelevant if the customer doesn't even care about it.
  • augustk 6mo ago
    Why don't we instead focus our energies on the user. For some very important software applications the customer is not the user. Let the sales department focus on the customer.
    • bob1029 6mo ago
      The user is always a customer of the product in my mind. I don't use the term to mean a purely financial relationship.
      • augustk 6mo ago
        The hospital information system Millenium from Oracle was bought by two regions in south of Sweden for around 400 million USD. It turned out to be unusable for Swedish healthcare and had to be shut down after just three days. Actual users of the software (doctors and nurses) were not involved in the procurement.
• jillesvangurp 6mo ago
  If you want to understand economics, I recommend watching some of Don Reinertsen's videos on Lean 2.0. He goes into a few concepts quite deeply that are quite intuitive. Cost of delay: calculating the cost of delaying by a few weeks in terms of lost revenue (you aren't shipping whatever it is you are building), total life value of the product (your feature won't be delivering value forever), extra cost in staffing. You can slap a number on it. It doesn't have to be a very accurate number. But it will give you a handle on being mindful that you are delaying the moment where revenue is made and taking on team cost at the cost of other stuff on your backlog. Option value: calculating the payoff for some feature you add to your software as having a non linear payoff. It costs you n when it doesn't work out and might deliver 10*n in value if it does. Lean 1.0 would have you stay focused and toss out the option for that potential 10x payoff. But if you do a bit of math, there probably is a lot of low hanging fruit that you might want to think about picking because it has a low cost and a potential high payoff. In the same way variability is a good thing because it gives you the option to do something with it later. A little bit of overengineering can buy you a lot of option value. Whereas having tunnel vision and only doing what was asked might opt you out of all that extra value. A bad estimation is better than no estimation: even if you are off by 3x, at least you'll have a number and you can learn and adapt over time. Getting wildly varying estimates from different people means you have very different ideas about what is being estimated. Do your estimates in time. Because that allows you to slap a dollar value on that time and do some cost calculations. How many product owners do you know that actually do that or even know how to do that? Don't run teams at 100% capacity. Work piles up in queues and causes delays when teams are pushed hard. The more work you pile on the worse it gets. Worse, teams start cutting corners and take on technical debt in order to clear the queue faster. Any manufacturing plant manager knows not to plan for more than 90% capacity. It doesn't work. You just end up with a lot of unfinished work blocking other work. Most software managers will happily go to 110%. This causes more issues than it solves. Whenever you hear some manager talking about crunch time, they've messed up their planning. Stretching a team like that will just cause cycle times to increase when you do that. Also, see cost of delay. Queues aren't actually free. If you have a lot of work in progress with inter dependencies, any issues will cause your plans to derail and cause costly delays. It's actually very risky to do that if you think about it like that. If you've ever been on a team that seemingly doesn't get anything done anymore, this might be what is going on. I like this back of the envelope math; it's hard to argue with. I used to be a salaried software engineer in a big multinational. None of us had any notion of cost. We were doing stuff that we were paid to do. It probably cost millions. Most decision making did not have $ values on them. I've since been in a few startups. One where we got funded and subsequently ran out of money without ever bringing in meaningful revenue. And another one that I helped bootstrap where I'm getting paid (a little) out of revenue we make. There's a very direct connection between stuff I do and money coming in.
• DeathArrow 6mo ago
  >Given that software teams are expensive In many companies there are 3 to 5 other people per developer (QA, agile masters, PO, PM, BA, marketing, sales, customer support etc.). The costs aren't driven just by the developer salaries. A CEO can cost as much as 10 developers, sometimes more.
  • dude250711 6mo ago
    That is why I respect Zuckerberg: he did not participate in Google's and Apple's salary fixing and he is willing to pay new tech hires insane money. There is something different about CEOs that came from tech.
• TheLudd 6mo ago
  One interesting factor that I rarely see discussed is this: Let's say a DevOps person does some improvement to internal tooling and a task that devs had to oversee manually now is automated. Every dev spent about 2 hours per week doing this task and now they don't have to anymore. Now, have we saved 2 hours of salary per dev per week? Not sure. Because it totally depends on what they do instead. Are they utilizing two hours more every week now doing meaningful work? Or are they just taking things a bit more easy? Very hard to determine and it just makes it harder to reason about the costs and wins in these cases.
  • radiator 6mo ago
    In such a clear-cut example, I think we have saved the two hours.
    • TheLudd 6mo ago
      Yes. You work 2 hours less, but what do you produce in those two extra hours? Can you say that your company now spends X dollars less or earns X dollars more? I don't think it can be that clear.
      • brabel 6mo ago
        And what is your theory? That it’s better to not save those 2 hours since they will just go to waste anyway? Or that there is diminishing returns to saving work as people will tend to just spend longer on other things they were already doing? How can you be sure those 2 hours will not actually be used by most to do very productive things that in the end look like +4 hours in return??
        • TheLudd 6mo ago
          No. I am not saying that it is a bad idea to do this. I am saying: Given you have saved two hours per person per week Then the value for the company is _not_ equal to two hourly salaries per week. The consequences are just not that simple.
  • 6mo ago
• gtsop 6mo ago
  [dead]
• mlazos 6mo ago
  Look! A guy built 95% of slack in 2 weeks! Very skeptical of that btw, but also an organization that justifies every single team by exactly how much $ value they’re generating sounds like hell. How would you ever innovate or try out new ideas? It’s important to quantify what impact your team is generating but there are some cases (e.g. UX) which are really hard to quantify in $ but are still very important for the product
• willvarfar 6mo ago
  With a long time in the industry and seeing how so many big software companies work, this really really chimed with me. Many/most teams and projects and busy work are not actually moving the bottom line, at massive opportunity cost! And there's so little awareness that most people in squads and their managers will think they are the exception. Whereas Whatsapp with its 30 software engineers was the exception etc. A chat with friends showed how there are parallels with how LLMs will happen in the short-term future - say the next 5 years - and the whole MapReduce mess. Back when Hadoop came along you built operators and these operators communicated through disk. It took years even after Spark was about for the hadoop userbase as a whole to realise that it is orders of magnitude more efficient to only communicate through disk when two operators are not colocatable on the same machine and that most operators in most pipelines can be fused together. So for a while LLMs will be in the Hadoop phase where they are acting like junior devs and making more islands that communicate in bigger bloated codebases and then there might be a realisation in about 2030 that actually the LLMs could have been used to clean up and streamline and fuse software and approach the Whatsapp style of business impact.
• groby_b 6mo ago
  I see we're once again missing the existence of indirect impact. There's a reason organizations look at revenue/engineer overall instead of trying to attribute it directly to specific teams. I guess his students get to relearn that on their own. Also, any post talking about building software and then contains the suggestion that "cost per unit" is an efficiency metric needs to come to the red courtesy phone, Taylorism would like to have a chat about times gone by.
• watsonjs 6mo ago
  I've been a software engineer for more than ten years and never cared about these kinds of topics. But lately, I've found them genuinely interesting. Could someone recommend books on the economics of software businesses? I can't take this author's content seriously.
  • brabel 6mo ago
    I have been interested in this topic for a long time and to be honest, there is no better book on the topic than The Mythical Man Month. Yes from the 70s I think, but still the best I have read.
• kcexn 6mo ago
  I feel like there is a lot of nuance around this topic that is getting lost in the noise. The direct and indirect financial impact of technical decisions are indeed hard to measure. But some technical decisions definitely have greater financial impact than others. Even if it's hard to precisely quantify the financial costs/benefits of every decision. It is possible to order them relatively. X is likely to make more money than Y. So we do X first and Y later. There is a significant amount of chance involved in whether a product/feature will even make money at all. So even good plans with measurably positive expected value could end up losing money. Just because it's impossible to be 100% certain of the outcome of any decision. Doesn't mean we should throw the baby out with the bathwater.
• jwpapi 6mo ago
  I thought it was a good article, till I saw the Slack example. The copy doesn’t even remotely grasp the scale of what the actual Slack sofware does in terms of scale, relaiability, observability, monitorability, maintability and pretty sure also functionality. Author only writes about the non-dev work as difference, which seems like he doesn’t know what he’s talking about in all, and what running an application at that scale actually means. This "clone" doesn’t get you any closer to an actualy Slack copy than a white piece of paper
  • camgunz 6mo ago
    I had the same experience (though I agree with other comments that the numbers are a little optimistic in terms of variance; I think there's a huge amount of variance in product work, you can't know what's a good investment until it's too late, many companies fail because of this, and there's huge survivorship bias in the ones that get lucky and don't initially fail). Slack spent tons of money in terms of product and engineering hours finding out what works and what doesn't. It's easy to copy/paste the thing after all that effort. Copy/paste doesn't get you to the next Slack though--it can get you to Microsoft's Slack-killing Teams strategy, but we obviously don't want more of that. And, obviously I agree with you about all the infra/maintenance costs, costs in stewarding API usage and extensions, etc. LLMs won't do any of that for you.
  • ekidd 6mo ago
    Yeah, I can build a Slack "clone" in a couple of weeks with my own two hands, no AI required. But it's not going to actually be competitive with Slack. Just to pick an incredibly, unbelievably basic enterprise feature, my two-week Slack clone is not going to properly support legal holds. This requires having a hard override for all deletion and expiration options anywhere in the product, that must work reliably, in order to avoid accidental destruction of evidence during litigation, which comes with potentially catastrophic penalties. If you don't get this right, you don't sell to large corporations. And there are hundred other features like this. Engineering wants an easy-to-use API for Slack bots. Users want reaction GIFs. You need mobile apps. You need Single Sign-On. And so on. These are all table stakes. It was a cliche for many years that Microsoft Word had "too many features." So people would start companies to sell "lightweight word processors" that only implemented "the most used 20% of features." And most of these companies sank without a trace (with a couple of admirable exceptions that hyperfocused on specific niches). Google finally made progress against the monopoly, but to it, they actually invested in a huge number of features. Believe me, I wish that "simple, clean" reimplementations were actually directly competitive with major products. That version of our industry would be more fun. But anyone who thinks that an LLM can quickly reimplement Slack is an utter fool who has never seriously tried to sell software to actual customers.
• dfhvneoieno 6mo ago
  [dead]
• christkv 6mo ago
  I expect engineering departments to be flattened and reduced in people. Corporate silos of responsibility around apps will probably disappear as a senior developer with tools can be pretty effective across platforms and technologies because the value of architectural and design thinking becomes more valuable
• tedggh 6mo ago
  This is a very reductionist way to calculate the value of a software team or any team within an organization. That’s because many times the value delivered by a team is not necessarily monetary but strategic.
• EastLondonCoder 6mo ago
  I think the article make some great points, however this part is not even wrong: "The obvious objection is that code produced at that speed becomes unmanageable, a liability in itself. That is a reasonable concern, but it largely applies when agents produce code that humans then maintain. Agentic platforms are being iterated upon quickly, and for established patterns and non-business-critical code, which is the majority of what most engineering organizations actually maintain, detailed human familiarity with the codebase matters less than it once did. A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves." LLMs are not conscious, that means left on their own devices they will drift. I think the single most important issue when working with LLMs is that they write text without a layer that are aware what's actually being written. That state can be present in humans as well, like for example in sleepwalking. Everyone who's tried to to complete vibe coding a somewhat larger project knows that you only get to a certain level of complexity until the model stops being able to reason about the code effectively. It starts to guess why something is not working and cannot get out of that state until guided by a human. That is not new state in the field, I believe all programmers has at points in their career come across code that's been written with developers needing to get over a hard deadline with the result of a codebase that cannot effectively be modified. I think for a certain subsets of programming projects some projects could possibly be vibe coded as in that code can be merged without human understanding. But it has to be very straightforward crud apps. In almost everything else you will get stopped by slop. I suspect that the future of our profession will shift from writing code to reading code and to apply continuous judgement on architecture working together with LLMs. Its also worth keeping in mind that you cannot assign responsibility to an LLM and most human organization requires that to work.
• boh 6mo ago
  This is some aggressive consultant fluff. Few companies have such distinctive "profit" measures. If "the financial logic is rarely examined carefully" than maybe there's a reason, since analysis like this is mostly fantastical and brittle. This is the sort of argument that is both rational and implausible. A manager might use this logic to rationalize firing an engineering team (which is mostly why guys like this get hired) but they won't use it to manage an engineering team.
• pron 6mo ago
  > A messy codebase is still cheaper to send ten agents through than to staff a team around People who say that haven't used today's agents enough or haven't looked closely at what they produce. The code they write isn't messy at all. It's more like asking the agent to build a building from floorplans and spec, and it produces everything in the right measurements and right colours and passes all tests. Except then you find out that the walls and beams are made of foam and the art is load-bearing. The entire construction is just wrong, hidden behind a nice exterior. And when you need to add a couple more floors, the agents can't "get through it" and neither can people. The codebase is bricked. Today's agents are simply not capable enough - without very close and labour-intensive human supervision - to produce code that can last through evolution over any substantial period of time.
  • MattGaiser 6mo ago
    A lot of that can be overcome by including the need to be able to put more floors on top as part of the spec. Whether it be humans or agents, people rarely specify that one explicitly but treat it as an assumed bit of knowledge. It goes the other way quite often with people. How often do you see K8s for small projects?
    • pron 6mo ago
      > A lot of that can be overcome by including the need to be able to put more floors on top as part of the spec I wish it could, but in practice, today's agents just can't do that. About once a week I reach some architectural bifurcation where one path is stable and the other leads to an inevitable total-loss catastrophe from which the codebase will not recover. The agent's success rate (I mostly use Codex with gpt5.4) is about 50-50. No matter what you explain to them, they just make catastrophic mistakes far too often.
  • cyanydeez 6mo ago
    The problem is, the MBAs running the ship are convinced AI will solve all that with more datacenters. The fact that they talk about gigawatts of compute tells you how delusional they are. Further, the collateral damage this delusion will occur as these models sigmoid their way into agents, and harnesses and expert models and fine tuned derivatives, and cascading manifold intelligent word salad excercises shouldn't be under concerned.
• clbrmbr 6mo ago
  > There is no cohort of senior product leaders who developed their judgment in conditions where their teams were expected to demonstrate financial return, because those conditions did not exist during the years when that cohort was learning the craft. There totally is such a cohort. There are plenty of bootstrapped companies or startups that took only an angel round and did not benefit from the low rate environment, in fact they suffered because of the very high price of SWE labor. But those engineering managers exist and are out there right now still building efficiently, quietly growing, passionately serving customers, and keeping a close eye on the bottom line and risks because that’s their livelihood.
• sarchertech 6mo ago
  > A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves. What experience is this guy basing this on? My guess is absolutely none at all. Maybe this will be the case in the future, but as of right now if I cut 10 agents loose for 10 days one of our repos at work and tell them to clean it up but but keep the tests passing, we’d be drowning in support tickets. Tests don’t cover all observable behavior. Every single production bug we’ve had made it through the test suite. Also this guy only had a vague idea of how platform engineering teams work in large organizations. Platform teams are the engineering org’s immune system. They’re how we fight back against the tech debt accumulated by the relentless march of features of the week. If anything the extra code people are cranking out with AI make them more necessary.
• joshwillik 6mo ago
  > The obvious objection is that code produced at that speed becomes unmanageable, a liability in itself. […] The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves. Maybe there’s some new paradigm that makes this true. But it doesn’t seem obviously true to me. Humans make the best code long term when everything orbits a vision of the underlying problem space. LLMs seem to only consider the deeper problem space when I explicitly flag it for them, otherwise they write “good enough for this situation” type code. And that stack of patches type code is exactly how the code becomes messy and complicated in the first place.
• bsenftner 6mo ago
  Yet another essay completely missing the point, and an audience that misses it as well. All these organizations fly blind because nowhere in any technology or science education is there any emphasis on effective communications, conveying understanding, solving disagreements with analysis and the best of both perspectives... none of these critical communication skills are taught to the very people that most need them. It's a wonder our civilization functions at all.
• carlm42 6mo ago
  Measuring a platform team's productivity in pure "hours saved" is missing a huge point: reliability. If your platform prevents even one outage every month, how much business value and capital are saved? That analysis is utterly absent from this article. It also seems to focus on "LLMs make code cheap" which is a half truth: LLMs make (so far) easy or messy code cheap. I'd bet that there too the analysis on reliability/stability is missing from the author's perspective.
• bdunks 6mo ago
  > Software development is one of the most capital-intensive activities a modern company undertakes The article is definitely written from a "high tech" industry lens. A mid-sized utility might spend $80-$150 million USD on IT capital projects in a year, but $2b on power pole maintenance. Utilities are a strong example, but any large enterprise manufacturing company is spending more on factory upgrades that programming. > [...] built a functional replica of approximately 95% of Slack’s core product in fourteen days using LLM agents. IT and Finance leadership and asset heavy companies are currently trying to wrap their head around the current economics of their 100+ SaaS contracts, and if it still makes sense with LLM powered developers. Can they hire developers in house to build the fraction of the tool they use from many of these companies, save on total cost and Opex? I work with these companies a lot, and won't weigh in on the right decision. Bottom line "it depends" on many factors, some of which are not immediately obvious. The article still holds weight regardless of industries, but there is some nuance (talent availability, internal change cost, etc.) that also have to be considered.
  • GorbachevyChase 6mo ago
    Yeah, that line came across as a little out of touch. I work for US DOTs, and a yearly allotment from a STIP of a small DOT is still measured in billions. Software spend is negligible. In fact, I would say software was always costly in terms of labor, but hasn’t been capital intensive until recently. But I would like to agree with what you said with respect to SaaS spending coming under scrutiny. Our technical experts are becoming aware that we spend 5 or 6-figure sums on software with barely any users that we can clone with a coding agent in an afternoon. Eventually management will find out too and we’re going to cut a lot of dead weight.
  • compiler-guy 6mo ago
    100%. A modern pharmaceutical manufacturing plant costs two-billion dollars just to build, and that doesn't include developing a drug to actually manufacture there, or a distribution network to sell what you make inside it.
  • red-iron-pine 6mo ago
    the SaaS isn't going anywhere. you're going to AI yourself a Tier 4 data center and all of the requisite generators and UPS systems. And how do you put liability on an LLM agent? I outsource to SaaS and consultants because 1) they're good at what they do, and 2) if they do it wrong I can sue them, escalate, berate them in social media, etc. and get things fixed; the AI pulls from so many places who is responsible? How do I validate that? I blackbox that I can't audit is a lot of risk compared to expensive consultants with shortcomings.
• sanghyunp 6mo ago
  [dead]
• bearjaws 6mo ago
  Thank you so much for putting into words what I have been saying for years to leadership teams. Everyone wanted to copy Meta/Google/Oracle and have internal teams, and to me internal teams have been accountability vacuums. People want an internal team so they can go "well if we had better tooling!" when instead they should make best with what they have.
• visiohex 6mo ago
  The core reason most orgs are "flying blind" is that we still don't have a reliable metric for technical debt. Management only tracks shipped features and velocity because they are easy to measure. They completely ignore the hidden liability of a rushed, messy codebase until productivity eventually grinds to a halt. You can't measure the economics of a team if you ignore the balance sheet.
  • judahmeek 6mo ago
    Time to ship, change failure, rework rates, mean time to resolve, code complexity, code churn, average age of dependencies - there's a ton of reliable metrics for technical debt, but they have to actually be looked at to do any good. The problem is that technical debt is a more complex concept & thus requires more metrics to properly measure than a simple concept like velocity.
• cmiles74 6mo ago
  There's a lot here, mixed some marketing and some dubious LLM claims. That being said, I think there could be real benefit in pushing detail on how features effect finances down to individual teams. Right now I have two features on my desk that both seem reasonable; if I knew which one would generate more income (i.e. increase customer retention, lead to more sales, etc.) that would make this choice a lot easier.
• GorbachevyChase 6mo ago
  I’m a little surprised that fundamental concepts like burn rates are not expected to be understood in software. In other professional services, this is often top of mind, at least for managers.
• kevin_nisbet 6mo ago
  I'm not commenting too much on the details of the article, but the premise does resonate with me. I would argue all the engineering teams I've been on do not spend enough time thinking about how much a piece of work will cost to execute, and whether it will generate a return. I suspect this is most apparent on things like meeting culture. Something happens and all of a sudden there is another recurring meeting on the calendar, with 15 attendee's, costing x dollars in wages, that produces no value for the customers because the lesson was already learned. Or when reacting to an incident of some sort, it's so easy to have a long list of action items that may theoretically improve the situation, but in reality are incredibly expensive for the value they produce (or the risks they reduce). It's too easy to say, we'll totally redesign the system to avoid said problem. And what worries me, is often those very expansive actions, then cause you to overlook realistic but small investments that move the needle more than you would think. And as a hot topic I also think the costs are an input into taking on tech debt. I know we all hate tech debt with a passion, but honestly, I think of it as a tool that can be wielded responsibly or irresponsibly. But if we don't know what our attention costs, we're going to have difficulty making the responsible choices about when and where to take on this debt. And then if we're not conscious about the debt, when it comes do it stings so much harder to pay down.
  • RamblingCTO 6mo ago
    Meetings aren't even the worst resource wasters. Wrong initiatives, features, apps/platforms/services are. They capture future resources in form of maintenance and complexity with them.
    • kevin_nisbet 6mo ago
      Agreed, and this is where I think some more nuanced and conscious use of tech debt can be used when applicable. It might be OK to place some bets on an initiative or feature, but if we all understand we're placing a bet, this is an area to load up on debt and really minimize the investment. This also requires an org that is mature about cutting the feature if the bet doesn't materialize, and if the market signal is generated will reinvest in paying down the debt. And also has the mega-danger territory of a weak market signal, where it's not clear if there is market signal or not, so the company doubles down into the weak signal. Also these bets shouldn't be done in isolation in my view, well executed product and market discovery should also provide lots of relevant context on the ROI.
• philsnow 6mo ago
  > Most engineers do not know this number. How could they not? When I penciled this out ~18 years ago, I included the amortized cost of all the interviews it took to hire a given engineer as well. It's not rocket surgery, as they say. Money can be exchanged for goods and services.
• rossdavidh 6mo ago
  All of this article, both the good (critique of the status quo ante) and the bad (entirely too believing of LLM boosterism) are missing (or not stressing enough) the most important point, which is that the actual programming is not the hard part. Figuring out what exactly needs programmed is the hard part. For reasons which it would take a while to unpack, if is often the case that the best (or sometimes only) way to find out what programming actually needs to be done, is to program something that's not it, and then replace it. This may need to be done multiple times. Programming is only occasionally the final product, it is much more often the means of working through what it is that is actually needed. This is very difficult for the people who ask for the software, to understand, and it is quite often very difficult for the people doing the programming to understand. Most of what is being done, during programming, is working through the problem space in a way which will make it more obvious what your mistakes are, in your understanding of the problem and what a solution would look like. Once you have arrived at that understanding, then there are a variety of ways to make what you need, but that is not the rate-limiting step.
  • Sevii 6mo ago
    While it's true that 'figuring out what exactly needs to be programmed' was always the hard part. It's not the part that the most money was spent on. Actually programming the thing always took up the most time and money.
    • rossdavidh 6mo ago
      True enough, but I think that a lot of "actually programming the thing" turned out to be "figuring out what exactly needs to be programmed". Afterwards, people did not want to admit that this was the case, perhaps even to themselves, because it seemed like a failure to plan. However, in most (nearly all?) cases, spending more time prior to programming would not have resulted in a better result. Usually, the best way to figure out what needs to be programmed, is to start doing it, and occasionally take a step back to evaluate what you've learned about the problem space and how that changes what you want to actually program. In other words "figuring out what needs to be programmed" and "actually programming the thing" look the same while they're happening. Afterwards, one could say that the first 90% was figuring out, and only the last 10% was actually doing it. The reason the distinction matters, is that if you do something that makes programming happen faster, but figuring out happen slower, then it can have the surprising affect of making it take longer to get the whole thing done.
• htrp 6mo ago
  Most orgs don't understand this. If you've ever been in a meeting with multiple L8's arguing over features, you should be able to estimate how much each hour of that meeting is costing the org.
• dasil003 6mo ago
  This article is not bad overall, but it does over-index on the cost of making software development costs and tradeoffs legible. Of course leadership does need to make decisions, and so the quest for better data and better cost modeling will continue, and rightly so, Goodhart's law notwithstanding. I do like this bit though: > A large codebase also carries maintenance costs that grow over time as the system becomes more complex, more interconnected, and more difficult to change safely. Every engineer added to maintain it increases coordination costs, introduces new dependencies, and adds to the organizational weight that slows decision-making. The asset and the liability exist simultaneously, and for most of the past twenty years, the financial environment masked the liability side of that equation. And the insight that LLMs are exposing this reality is absolutely true. The funny thing is they are exposing it by accelerating both good and bad engineering practices. Teams with good engineering judgement will move faster than ever with fewer people, and teams with bad engineering judgment will bury themselves in technical debt so fast the wheels will come off. For me, running an engineering org is primarily about talent acquisition and empowering those ICs with judgment to move quickly. How well systems and teams scale depends on the domain, product, and how it allows you to decouple things. With the right talent and empowerment there are often creative ways to make product and system tradeoffs and iterate quickly to change the shape of ROI. Any mapping to financial metrics is a hugely lossy operation that can't account for such changes. It might work in mature companies that are ossified and in the second half of their lifecycle, but in growing companies I think it's fundamentally misguided would amount to empowering the wrong people.
• rockemsockem 6mo ago
  Wow that article made a hard right turn about halfway through. "Most organizations improperly account for engineering teams and incorrectly consider both code and team growth to be assets when in fact they increase complexity..... but LLMs can fix all of this" Wtf? Measuring things that actually matter is a great way to improve clarity on a team, you can probably just stop reading this article at the halfway point. EDIT: Specifically this paragraph is insane "The obvious objection is that code produced at that speed becomes unmanageable, a liability in itself. That is a reasonable concern, but it largely applies when agents produce code that humans then maintain. Agentic platforms are being iterated upon quickly, and for established patterns and non-business-critical code, which is the majority of what most engineering organizations actually maintain, detailed human familiarity with the codebase matters less than it once did. A messy codebase is still cheaper to send ten agents through than to staff a team around. And even if the agents need ten days to reason through an unfamiliar system, that is still faster and cheaper than most development teams operating today. The liability argument holds in a human-to-human or agent-to-human world. In an agent-to-agent world, it largely dissolves."
• rektomatic 6mo ago
  Does anyone really believe that Slawk is a "replica of approximately 95% of Slack’s core product"??
• TheGRS 6mo ago
  The points brought up are all great. I'm in a lower management position and I've wondered for a decade why the budget, cost, and return on work (i.e. revenue) were never divulged or connected to the work at hand. So kudos for facing that problem bluntly in language that's easy to follow. The place I'm at currently, its much more about automating away processes and making back office operations easier, so there's likely a lot of direct cost savings that we could measure, but don't. Here's the problem I see with how this particular article is moving though: the context of these projects are often highly technical connecting back to the human problem space. Developers sit on the technical end but they also usually have a mental model for how it connects back to the non-technical. A product manager is another addition to compensate for the user connection. Between all of these folks they can only hold so much in their head about the problem space on a day-to-day basis. And that headspace for the problem is what is critical. Management wants to try a new idea for sales? They need to take it to the team with that problem space to translate it into working code. Even with the assistance of agents, one needs to hold the important patterns in their head. And my company certainly isn't going to vibe code its way through anything regulatory, mistakes there might cost us a ton in fees and bad PR. Hell I've seen product managers sweat over the possibility of getting a few 1 star reviews on the app store. Anyway, you still need people with context to break things down and get them out the door, the agents can just assist with the speed of the In Progress stage. And clever teams can figure out how to automate their validation (but they could already do that). Rockstar developers often seem to be the ones who can parachute in, gain context, make changes, and leave to find another problem space. They get bogged down when they've visited 10 or more problem spaces and then they start getting called back into service. Again the agents don't change any of that, the human involved has a finite capacity for context. Teams who structure around maintaining context might be best suited for the new world of code.
• RamblingCTO 6mo ago
  I do tech dd, exit readiness and post merger integration in tech companies and this is my daily bread. The biggest lever I have: connecting initiatives to ROI/bottom line impact. It's incredible how blind product/software teams run. So much to do but most of it won't make any money and just feels productive. Connecting activities and work directly towards revenue is very important. If your company runs well: won't hurt you much that you're not doing this. Otherwise this will be your end. And that really hurts because you lose the economical impact of the product and the jobs.
• juancn 6mo ago
  The over-simplification rubs me the wrong way, for example: Consider a team of eight engineers whose mission is to build and maintain an internal developer platform serving one hundred other engineers. This is a common organizational structure, and it is one where the financial logic is rarely examined carefully. The team costs €87,000 per month. To justify that cost, the platform they build needs to generate at least €87,000 per month in value for the engineers who use it. The most direct way to measure that value is through time saved, since the platform’s purpose is to make other engineers more productive. At a cost of €130,000 per year, one engineer costs approximately €10,800 per month, or around €65 per working hour. For the platform team to break even, their platform needs to save the hundred engineers they serve a combined total of 1,340 hours per month. That is 13.4 hours per engineer per month, or roughly three hours per week per person. There's a fungibility assumption which is pervasive here. In most cases, a platform team is there not "to save time". It's there to deal with cross concerns that would be not only time consuming but could be business threatening, and in some cases, you keep there more expensive engineers that ensure that certain critical things are done right. Too much snake oil for my taste.
  • nonameiguess 6mo ago
    It's missing the point of a platform team. Regardless of how you choose to organize your larger business, each individual product needs similar things. Whatever those may be. Code forge. Binary artifact repositories. Build, test, and deployment pipelines. Development and staging environments. Workstations with a network connection to your business systems. You can do pay for SaaS enterprise stuff, build it yourself team by team, or have a platform team. The point at which it makes economic sense to have a platform team isn't when they can directly save money by saving time. It's when you have enough product teams that having them all devote effort to doing the same thing in parallel when it can be centralized is counterproductive, and your needs are sufficiently niche or the seat licensing is high enough that SaaS solutions no longer work, either. It's effectively a form of administrative overhead. The question isn't whether the cost is justified. You have to do it. The question is whether you're better off centralizing the function in house or not. You see a similar dynamic with any other administrative function. Should you hire a payroll company or have your own payroll department? The difference is software developers tend to believe, and in many cases might even be right, that they can individually role their own platform functions, introducing a secondary consideration, whereas the idea of having software developers simply handling their own payroll clearly makes no sense. Of course, it's worth another of the more obvious reasons it doesn't make sense. If you give your engineers direct access to the company bank account, there's a pretty clear risk they're going to steal from you. Similarly, there's a non-economic argument to be made for having a platform team in terms of separation of duties for security. Every business decision isn't simply made by answering what does this cost and how much will it earn me? Try running a software business where the developers are responsible for physical site security and see how long you make it before getting robbed. Try not hiring any janitors, letting your developers clean their own restrooms, and see how long you retain any staff.
• decancode 6mo ago
  But in reality, the real cost of engineering teams grow as the sub organizations and teams continue to make short term decisions, optimizing for the next immediate win. More common so in larger organizations than smaller ones
• balderdash 6mo ago
  i think the thing that hits home for me here is that when you go back and do the after action report on where the time was spent last year and what it cost its terrifying. of course hindsight is 20/20 and predicting how difficult something is going to be is hard, but when you say we spent $x million on this version update that does y and $a hundred thousand to implement this feature - you think to yourself we would have never made that cost / benefit decision if we had known.
• toniantunovi 6mo ago
  [dead]
• Dr4k3_ 6mo ago
  “Day 14: A functional replica of Slack’s core product, built by Nathan using LLM agents.” What does that even mean? What are “core features”? Just a chat room? That’s trivial compared to what a real system like Slack actually does: handling millions of messages at scale, ensuring consistency across clients, maintaining high availability, storing and serving media (videos, audio, images), managing backups, and more. Building a simple chat app that mimics Slack’s UI isn’t impressive. That was a few hours of work even 20 years ago, long before LLMs. Calling that a “replica of Slack’s core product” is, at best, misleading.
• strife25 6mo ago
  I like this angle. I didn't read much of the article, but I do think this is a topic that people should consider, at least when it comes to estimating software projects. I don't think we really think enough about the monetary costs of our projects. When I raise the topic in my discussions, it really changes and focuses people, particularly when folks are advocating for tech debt projects that don't really have a return because they're not a focus for the company at this time. When we say that it's going to be a $60,000 project, it quickly becomes deprioritized in the conversation.
• deleted 6mo ago
  [deleted]
• thornewolf 6mo ago
  more ai articles on my front page :(
• rupayanc 6mo ago
  The Anthropic C compiler experiment keeps coming up for a reason. Two weeks, 100KLOC, completely unmaintainable. Faster generation doesn't help if nobody in your org can reason about what was shipped. The bottleneck was never typing speed.
• ericyd 6mo ago
  > their value is, at least in principle, calculable I feel this article does not spend enough time investigating the challenges of directly measuring the financial output of an engineering team. I agree it is theoretically possible, but I don't think the full answer is that people got lazy on cheap capital and didn't care enough to measure. I think it would be exceedingly difficult to put a dollar amount on the monthly output of most engineering teams due to the variety of tasks they cover, and the extreme challenge of knowing exactly why your customers are behaving a certain way. If you get 1000 more signups in a month, is that directly attributable to the engineering team's output? If anyone could have been concretely answering that question this whole time, I don't think they would have been ignoring the metrics.