8 ms·

The unique thing about estimates in software engineering is that if you do it right, projects should be impossible to estimate! Tasks that are easiest to estim

• seviu 11mo ago
  I am in a project where we have to give estimates in hours and days. Needless to say we always underestimate. Or overestimate. Best case we use the underestimated task as buffer for the more complex ones. And it has been years. Giving estimations based on complexity would at least give a clear picture. I honestly don’t know what the PO and TL gains with this absurd obscenity.
  • 121789 11mo ago
    Hours is insane. But ultimately time is money and opportunity cost. Software engineering can’t be the only engineering where you ask the engineers how much something will cost or how much time it will take and the answer is “it’s impossible to know”. Even very inaccurate estimates can be helpful for decision making if they are on the right order of magnitude
    • yetihehe 11mo ago
      I think software is one of those VERY rare things, where inaccurate estimates can actually be inaccurate by "orders of magnitude". After 20 years in the field, I still managed to use 2 months of time on a task that I estimated as 10 days.
      • camel_gopher 11mo ago
        A rule that has suited me well is to take an estimate, double it, and increase by an order of magnitude for inexperienced developers. So a task the say would take two weeks ends up being 4 months. For experienced developers, halve the estimate and increase by an order of magnitude. So your 10 days estimate would be 5 weeks.
        • yetihehe 11mo ago
          10 days was already after I used this algorithm. Previous several tasks on that codebase were estimated pretty good. Problem with this is that some tasks can indeed take SEVERAL orders of magnitude more time that you thought. One of the hardest problems with estimating for me is that I mostly do really new tasks that either no one wants to do because they are arduous, or no one knows how to do yet. Then I go and do them anyway. Sometimes on time, mostly not. But everybody working with me already knows, that it may be long, but I will achieve the result. And in rare instances other developers ask me how did I managed to find the bug so fast. This time I was doing something I have never before done in my life and I missed some code dependencies that needed changing when I was revieving how to do that task.
        • QuercusMax 11mo ago
          The biggest estimation effort I ever took part in was a whole-system rewrite[1] where we had a very detailed functional test plan describing everything the system should do. The other lead and I went down the list, estimated how long everything would take to re-implement, and came up with an estimate of 2 people, 9 months. We knew that couldn't possibly be right, so we doubled the estimate and tripled the team, and ended up with 6 people for 18 months - which ended up being almost exactly right. [1]: We were moving from a dead/dying framework and language onto a modern language and well-supported platform; I think we started out with about 1MLOC on the old system and ended up with about 700K on the rewrite.
        • bdangubic 11mo ago
          I’ll send my friend that has a construction company to build your next 3500 sq ft house for $13.6 million dollars :)
    • njovin 11mo ago
      The next natural progression of this line of discussion between "the business" and engineering is for them to agree on a time range as an estimate. Engineering doesn't want to say it'll be done in 6 weeks, but they feel okay saying it will take between 4 and 20 weeks so this estimate is accepted. You can guess what happens next, which is that around week 8 the business is getting pretty angry that their 4-week project is taking twice as much time as they thought, while the engineering team has encountered some really nasty surprises and is worried they'll have to push to 24 weeks.
      • skeeter2020 11mo ago
        it is still better to give a range though because 1. it explicitly states the degree of unknown and 2. no boss is going to accept 4-20 weeks, which means you start talking about how you can estimate with better accuracy and the work required to do so, which is a major goal of planning & estimation.
      • shiroiuma 11mo ago
        You're forgetting the part where the business expects the engineers to pull all-nighters so they can meet the deadline.
    • zdragnar 11mo ago
      There's two things here that get overlooked. First, people asking for estimates know they aren't going to get everything they want, and they are trying to prioritize which features to put on a roadmap based on the effort-to-business-value ratio. High impact with low effort wins over high impact high effort almost every time. Second, there's a long tail of things that have to be coordinated in meat space as soon as possible after the software launches, but can take weeks or months to coordinate. Therefore, they need a reasonable date to pick- think ad spend, customer training, internal training, compliance paperwork etc. "It is impossible to know" is only ever acceptable in pure science, and that is only for the outcome of the hypothesis, not the procedure of conducting the experiment.
      • skeeter2020 11mo ago
        My take: if they have not done the work to get to at least some degree of a spec, and they won't give you time to review and investigate, they get nothing more than a vague, relative t-shirt size.
      • collingreen 11mo ago
        > "as soon as possible after the software launches" This isn't true, just desired, and is one of the main roots of the conflict here. OF COURSE you would like to start selling in advance and then have billing start with customers the instant the "last" pr is merged. That isn't a realistic view of the software world though and pretending it is while everyone knows otherwise starts to feel like bad faith. Making software that works, then having time to deploy it, make changes from early feedback, and fix bugs is important. THEN all the other business functions should start the cant-take-back parts of their work that need to coordinate with the rest of the world. Trying to squeeze some extra days from the schedule is a business bet you can make but it would be nice if the people taking this risk were the ones who had to crunch or stay up all night or answer the page. Trying to force complicated and creative work into a fake box just so you can make a gantt chart slightly narrower only works on people a couple times before they start to resent it. 10x that if management punishes someone when that fantasy gantt chart isn't accurate and 100x that if the one punished is the person who said "it's impossible to know" and then was forced into pretending to know instead of the person doing the forcing.
    • lmm 11mo ago
      > Software engineering can’t be the only engineering where you ask the engineers how much something will cost or how much time it will take and the answer is “it’s impossible to know”. Because it's not engineering at all. But even if it was, plenty of engineering projects are impossible to estimate - the ones that are doing something novel - and disliking that fact doesn't make it go away. > Even very inaccurate estimates can be helpful for decision making if they are on the right order of magnitude If what the business wants is an order-of-magnitude, they should ask for that; often (not always!) that's a lot easier.
  • SoftTalker 11mo ago
    The last director I had would ask "is it a day, a week, a month, or a year" he understood that's about as granular as it's possible to be. And he really only used them in comparison to estimates for other tasks, not to set hard deadlines for anything.
    • skeeter2020 11mo ago
      This is essentially t-shirt sizing without all the baggage that comes from time. Your boss is trying to use the relative magnitude but it's inevitable that people will (at least internally) do math like "7 day tasks is the same as one week task", or worse over-rotate on the precision you get from day/week/month, or even worse immediately map to the calendar. Suggestion: don't use time.
      • cnity 11mo ago
        If you pretend not to use time, everyone will do an implicit time mapping in their head anyway. I've never seen it go any other way.
        • seviu 11mo ago
          Surprisingly prob yes But still we are much better at estimating complexity Time estimations usually tends to be overly optimistic. I don’t know why. Maybe the desire to please the PO. Or the fact that we never seem to take into account factors such as having a bad day, interruptions, context switch. T-shirt sizes or even story points are way more effective. The PO can later translate it to time after the team reaches certain velocity. I have been developing software for over twenty years, I still suck at giving time estimates.
          • jrs235 11mo ago
            Time estimations, or conversations to days or other units, typically fail because if a developer says 1 day they might mean 8 focused uninterrupted development hours while someone else hears 1 calendar day so it can be done by tomorrow, regardless of if a developer spends 8 or 16 hours on it.
  • Scarblac 11mo ago
    > I honestly don’t know what the PO and TL gains with this absurd obscenity There are marketing campaigns that need to be set up, users informed, manuals written. Sales people want to sell the new feature. People thinking about road maps need to know how many new features to can fit in a quarter. Development isn't the only thing that exists.
• Scarblac 11mo ago
  Another reason is that figuring out what the software to be written should actually do, and how it should work, is work that is part of the project and the time it will take needs to be estimated. As well as the actual development work that will result, which isn't known yet at the time of estimation.
• IanCal 11mo ago
  Repetitive and easy or worth automating are not the same. The fundamental problem is that a setup capable of solving any of the requests that come to it is as complex as just a whole programming language and then you’re back to square one.
• wpietri 11mo ago
  And I'd add that the need for them is a sign they aren't worth doing. As you say, worthwhile software is usually novel. And to justify our expense, it needs to be valuable. So to decide whether a project is worth doing, we're looking at some sort of estimate of return on investment. That estimate will also, at least implicitly, have a range. That range is determined by both the I and the R. If you don't have a precise estimate of return, making your estimate of investment more precise doesn't help anything. And I've never seen an estimate of return both precise and accurate; business is even less certain than software. In my opinion, effort put into careful estimates is almost always better put into early, iterative delivery and product management that maximizes the information gained. Shipping early and often buys much clearer information on both I and R than you can ever get in a conference room. Of course all of this only matters if running an effective business is more important than managerial soap opera and office politics. Those often require estimates in much the same way they're required from Star Trek's engineers: so the people with main character syndrome have something to dramatically ignore or override to prove their dominance over the NPCs and material reality.
  • bpt3 11mo ago
    The solution you described is basically agile, and that definitely includes estimates and deadlines.
    • Mtinie 11mo ago
      It’s Agile philosophically, and how it should be. But that is rarely how it works. In the dozens of different projects across ten or twelve companies I’ve had insight into, “doing Agile” is analogous with “we have a scrum master, hold stand ups, and schedule iterations” while the simple reality is “Agilefall.”
      • bpt3 11mo ago
        Agreed, but the parent poster said that estimates shouldn't be done at all, which is not a legitimate argument to make in any scenario.
• bpt3 11mo ago
  If software developers want to be then seriously as a profession, they need to be able to provide and justify estimates for their work. Everything you said could apply to a new bridge, building, pharmaceutical compound, or anything else that is the result of a process with some known and some unknown steps.
  • XorNot 11mo ago
    Pharmaceutical compounds frequently don't make it to market after significant investment. No one in that industry is giving estimates based on developing brand new drugs - they're giving estimates related to manufacturing lead times, unalterable physics time lines, and typical time to navigate administrative tasks which are well known and generally predictable (but also negotiable: regulations have a human on the other end). All of this after they have a candidate drug in hand. Same story with bridge building basically: no one puts an estimate on coming up with a brand new bridge design: they're a well understood, scalable engineering constructions which are the mostly gated by your ability to collect the data needed to use them - i.e. a field survey team etc. - and also once again, regulatory processes and accountability.
    • bpt3 11mo ago
      Yes, that's my point. There's way more uncertainty in trying to bring a new drug to market or build a new bridge than creating yet another CRUD app, yet somehow they are any able to break these efforts into tasks that can be estimated and tracked and many software engineers think they should be exempt from any accountability to schedule or budget.
      • Aeolun 11mo ago
        And do you think those things are delivered on schedule any more often than software projects?
        • bpt3 11mo ago
          Take a look at the top of this thread and see what we're talking about. The fact that people in many industries are not good at estimating doesn't mean that it's impossible in software development specifically and uniquely, as was originally claimed.
• raw_anon_1111 11mo ago
  There is very little novel about most B2B CRUD and internal bespoke apps that most developers are doing. The novel part if any is implementing the business vertical logic
• vegetablepotpie 11mo ago
  As a project manager, it sounds like you're making excuses. Just give me a number, trust your gut! We have a fundamental failure to communicate, what we're doing. The game project managers and finance believe we're all playing is a regression towards the mean, where everything is additive and rounds up to nice consistent formulaic sums. Whereas software development follows power law distributions. A good piece of software can deliver 10x, 100x, or 1000x the cost to produce it (ex: distribution, cost of delivering another copy of software is near 0 once written). You don't get that sort of upside with many other investments. Finance is happy with an NPV 8% above what they invest in. This means that when software people talk, everything they say sounds foreign, and everyone assumes it's because of jargon. It's not. The fish don't know they're swimming in water. When the fisherman comes, everyone is caught off guard. So we get what the author talks about > The estimates stopped being estimates. They became safety railings against being held accountable for unreasonable expectations. We. Pad. Like. Crazy. Yes this is inefficient. Some project managers recognize this. We get theory of constraints. But rather than cull the layers of hierarchy that lead to the padding in the first place, all the blame for failure goes back to developers. Get hit on the head enough and you will stop acting in good faith and pad to save your ability to feed and cloth yourself.
  • joombaga 11mo ago
    It's not obvious to me that we should avoid padding, or why it's seen as undesirable.
    • vegetablepotpie 11mo ago
      Padding ties up capital, it reduces credibility, it delays deployment, it adds costs through delay. It is bad for organizations. However, it is a great solution if you're a worker in a bureaucratic environment that can tolerate large costs, but is intolerant of 1-day of schedule slips. It's a great solution for complacent management, who are confused about the game they're playing and wants to report that they're "on track", which means "not late". The agile solution of incremental value delivery is a compromise, and can produce good outcomes for functional changes. But agile has unacceptable failure modes when working on infrastructure and satisfying system constraints. Agile can work okay for programmers, but it's not a solution for engineers. Acknowledging, owning, and managing risk is more scalable, but you have to have leaders who acknowledge that they exist and have the maturity to take on that responsibility.
• deleted 11mo ago
  [deleted]
• lloeki 11mo ago
  From another thought-experiment-y perspective: Say you have problem A to solve. Then either one of those is true: 1) it has been solved before, ergo by virtue of software having a zero cost of copying (contrary to, say, a nail, a car, or a bridge), so there is no actual problem to be solved. 2) it hasn't been solved before, ergo it is a new problem, and thus at any moment you may turn a stone and discover something that was not foreseen (whether they are rabbits, yaks, bikesheds, dragons, or what have you eldritch horrors) and thus of unknown cost. Any task that cannot be obviously fit into one or the other can nonetheless be split into an assembly of both. Thus any attempt at estimates is as futile as gambling to win, tasks are only ever done when they're done, and "successful estimators" are kings of retconning. It's all make-believe.
  • zffr 11mo ago
    I was with you until this part: > Thus any attempt at estimates is as futile as gambling to win, tasks are only ever done when they're done, and "successful estimators" are kings of retconning. > It's all make-believe. Software estimates are not futile or make believe. They are useful even if they are not always precise. That’s why the industry continues to use them.
    • kragen 11mo ago
      This argument proves too much: "Bloodletting is not futile or make believe. It is useful even if the patient does not always survive. That's why physicians continue to use it." "Trial by ordeal is not futile or make believe. It is useful even if sometimes Inquisitors reach mistaken conclusions. That's why Inquisitors continue to use it." "Lottery-number-picking systems are not futile or make believe. They are useful even if some players never win. That's why players continue to use them." It is a fully general argument which, if correct, would demonstrate that no practice that had continued for a period of time could ever be ineffective or counterproductive.
    • bonesss 11mo ago
      The amount of tap dancing and philosophizing some developers are willing to do to dodge estimates is hilarious. It’s a skill… a basic part and critical part of engineering. IME the common thread between objectors is that they haven’t made a consistent effort to improve — developing, iterating, and refining their estimation process over time. Yeah, every line of code is a unique snowflake piece of undefinable research the universe has never seen, equally unknowable and inscrutably enigmatic. But the workers at EngiCorp building EngiCorp products using EngiCorp project routines and resources first, second, and third quarter of 2025 are literal world experts at EngiCorp outcomes. They very reasonably should be able to estimate EngiCorp work in Q4, and account for EngiCorp realities, providing maps of future costs that can drive EngiCorp process improvement and investment. If I ask for a decking estimate and get back sophistry and smug incompetence, I’m not talking with a super skilled professional deck builder. Doesn’t matter how they hammer, saw, or draw.
• analog31 11mo ago
  "... anybody with any brains has already left town..." -- Bob Dylan Anybody with any business experience has already isolated themselves from the certainty of software project failure, where "failure" is a euphemism for "late." So it doesn't matter if software can't be estimated. This can be nerve-wracking to a beginner, but one gets used to it over time.
• eb0la 11mo ago
  That's the point: how can you tell WHEN you are going to reach a place you've never been before traveling an uncertain path? Making mistakes over and over again. And adding a lot of time buffers just to be safe
• harrall 11mo ago
  As I’ve gotten older, I find this to be untrue. Estimating the unknown is itself a skill. It’s like choosing to crash random parties everyday — at first, exceptionally novel but facing the unknown becomes itself mundane. Humans adapt. They build mental modals to approach the unknown and these mental modals are predictable. I just don’t think most people get much practice.
• yodsanklai 11mo ago
  I don't know how typical teams work, but in my case, new projects always come on top of other obligations. It may take 1 day to add the field, but how many meetings, fires, or other disturbances will happen during that day?
• GMoromisato 11mo ago
  Very insightful. This is also my go-to argument for why software engineering is real engineering.
• tbrownaw 11mo ago
  > If I ask you how long it'll take to add a new database field, and you've added a new database field 100s of times in the past and each time they take 1 day, your estimate for it is going to be very spot-on. But in the software world, predictable and repetitive tasks are also the kinds of tasks that are most easily automated The slow part isn't following a written "log in and run ALTER TABLE" runbook, it's the review of "does this solution make sense given what we're trying to do".
• austin-cheney 11mo ago
  This is going to be immediately downvoted, but whenever I see software developers mention that estimating is uniquely challenging for software it immediately calls out a bunch of red flags. Estimates in software are challenging but no more or less challenging than other industries. Typically the people that single out software as a unique snowflake in the world of delivery and estimation really just say out loud they have no experience in project management and no experience working outside of authoring code. The things that make estimations most challenging in software are the same factors that make estimations challenging everywhere else.
  • ctenb 11mo ago
    Can you give an actual counterargument?
  • Capricorn2481 11mo ago
    I don't think it's unique, but many software projects are inherently prone to big surprises you can't catch until you start the work. I would love to keep estimating and estimating and get an accurate number, but it costs money and, eventually, you're just working on the project.
• deepsun 11mo ago
  Spot on! Now add one more dimension (from machine learning projects and pretty much other research fields) -- you also never know if the result are going to even work. Maybe the data just doesn't have the correlation you want. Maybe you just didn't try harder, never know for sure. But owners reasonably demand some estimations.
• gilbetron 11mo ago
  I call it the "Persistent Incompetence of Software Development", which is another perspective on estimation, focused more on expertise. A chef that cooks pizzas, cooks the same pizza over and over again and becomes amazing at it. If you are a developer that writes the same code over and over, you are terrible at software development. A good software developer should always be solving new problems, as by the nature of software, once they solve a problem, they never solve that (exact) problem again. So we are persistently incompenent. Which is why software development can't be estimated, as well. Because it is all, as you say, novel. With infinite error bars. At this point, I can't take anyone seriously that believes software dev can be estimated.
• NathanaelRea 11mo ago
  This is such an interesting observation, makes a lot of sense.