10 ms·

Ask HN: Why do software companies still insist on Estimation?

• giantg2 4y ago
  Estimating what? If we're talking about effort, timelines, etc, then it does work. You are right that it generally doesn't work on the surface. However, even if the estimates are wrong, the benefit is that the artificial dates and such provide a mental push. In some cases dates are important, to meet a business deadline, beat a competitor, etc. Breaking that up and pushing it down to smaller stories means more feedback which provides more of that 11th hour push more often. And then of course things like costs and staffing are crucial to any business.
• superchroma 4y ago
  Because you can't go to money people with "it'll take as long as it takes". It may be true, but that doesn't fit in Microsoft Project, and as long as there is another reputable group out there providing estimates, even junk ones, people will expect them.
  • 121789 4y ago
    well, it isn't really true. you should be able to estimate a project within an order of magnitude of time in the vast majority of instances. Think of 1 day 1 week 1 month 3 months 6 months 1 year 3 years if you had a random project, do you think you wouldn't be able to pick the most likely estimate and be within one level above or below? so "i have no idea, it'll take as long as it takes" is not really true.
    • 3vidence 4y ago
      Even this can be really deceiving, maybe core APIs to crucial libraries get removed. As a response maybe the team either needs to migrate to a new library, or rebuild the APIs from scratch. A situation like this can turn a "X days" task into an "X months" task. I know it is not super realistic but it represents a type of situation.
      • 121789 4y ago
        there's a big difference between "we can make a reasonable estimate based on X assumptions with Y dependencies, but there is a risk we are wrong" and "it will take an undefined amount of time, estimation is useless"
• dyingkneepad 4y ago
  Where I work they ask me how much time a certain task will take, but they still go with whatever time the PM/Manager thinks it will actually take, then they complain when things get late. I once had a huge discussion with the Architect because I said something would take an year to finish and the Architect was claiming it would take 2 months. He would not understand that the remaining 10% of the task would take 90% of the time (yes, it was partially complete and I had been following its development from the start, although never involved in it directly). In the end they went with his estimation but it took 2 years to finish the thing. Still, I think estimation is important, once you actually listen to the estimations and understand it really depends on who is doing it.
• crazygringo 4y ago
  I mean, what's the alternative? Even if it's nowhere near perfect, it's far, far better than no information at all. Surely you'll concede that there's an objective difference between something that will take one person 1–5 days, vs. a team 6–12 months? Also, the process of estimation very often reveals difficulties or unknowns that nobody was even thinking about previously. That's the whole purpose of estimation happening via unanimous planning poker -- the discussion as to whether a feature is a 3 or a 10 is incredibly valuable in getting on the same page as to what the feature even is.
• mixmastamyk 4y ago
  Expecting a blank check, eh? Steve Mconnell dispelled many of the misconceptions around project planning and estimation around twenty years ago. Worth a read.
• patrogizmo 4y ago
  Let's talk about the reality of software development. Some projects are easy to estimate, especially in domains a developer has tackled before. As an example, consider a developer who has written five static pages who's asked to estimate how long it'll take them to write a sixth. Think of these projects as a well-defined path through the meadow to success. However, some projects are closer to crossing glacier for the first time; you never know when that fluffy bit of snow on the ground is hiding a crevasse of challenges. While it may be true, saying "it'll be done when it's done" isn't useful. All tasks involve some aspect of uncertainty. Coming up with an at least half-way decent estimate requires a team to break down the problems they're facing into smaller chunks that can be estimated. This is a useful exercise when it comes to prioritizing projects, budgeting, and go/no-go decisions. There are many ways in which estimation as an exercise can be corrupted. Some people, particularly those who haven't written software themselves, treat all estimates as accurate. In their mind, that "I guess 3 months" estimate a developer gave quickly turns into "I can 100% deliver this to you fully tested, with all of the additions you added along the way in 3 months". This over-indexing often leads to the situation where a developer is disciplined for not meeting expectations, even though they tried their hardest to do so. Some people don't like the estimates they are presented with, so they seek out a lower estimate. They are then shocked when the project takes longer.
  • malux85 4y ago
    > In their mind, that "I guess 3 months" estimate a developer gave quickly turns into "I can 100% deliver this to you fully tested, with all of the additions you added along the way in 3 months". This is my biggest trigger, when a vague, hand-wavey estimate is turned into a signed guarantee, then the developer gets disciplined or treated as a problem creator when more detailed information comes in. I found this was the problem with working at agencies - No matter how warm touchy feely they are, the fact is the agency is economically incentivised to burn out their developers (in the short term, which sadly, is the most frequently perceived time horizon). This is why I only work at venture funded startups now with a preference for default-alive ones, yes it's a bit chaotic because resources are not infinite everywhere, but it's much better than working at an agency who are just middle-man reselling your time and trying to keep their margins up
• shagymoe 4y ago
  OP: If you were spending your hard-earned money to pay a team of engineers to build your vision, would you care about how long it might take and how much it might cost?
• drewbeck 4y ago
  There's a certain class of engineer that wants the freedom to build whatever, take however long it takes, and have the broader organization react. "You can market the feature whenever I'm done" is, unfortunately, not going to lead to success for the org as a whole.
• 3vidence 4y ago
  I actually agree with this take, but I think the opposite can work. Create delivery dates based on runway / business priorities, re-evaluate what you think you can do in that amount of time. Every few weeks try to determine what you might need to cut to meet that date or see if the business can push back or delay that date. I don't think starting from I think X feature will take Y days ever works. By contrast, we need to launch by Z date is technically a valid statement, it doesn't need to be estimated. I think for this to work the business actually needs to be willing to admit that either more people need to be hired, features need to get cut or dates need to be pushed back.
• lesuorac 4y ago
  I dunno, using previous projects as a similar baseline seems to work out very well for estimating within ~10% of future projects. Although not all future projects are of a similar scope with similar people as a previous project but when it applies it seems to work well.
• mcluck 4y ago
  Accurately estimating work is possible and I have done it plenty of times. Precisely estimating work is nearly impossible. I'm of the opinion that everyone who has given up on estimates has some combination of not enough practice and inefficient communication of their estimates. If I ask an average web developer if they can create a basic landing page in 3 months, they will always say 100% yes. Now shrink it to 1 month: 100%. 3 weeks: 90%. 1 week: 50%. Now as the person paying for the work, you can make a choice. Do you plan to start advertising on the odds of a coin flip or do you wait until one of the 100% points?
• potamic 4y ago
  When you hire a contractor to redo your kitchen, do you ask how long it may take? When you order food online, do you expect to know when it will arrive? When you go to your mechanic for a repair, do you want to know when you may get your car back? When you go to a lawyer to verify a contract, do you need to know when they will be done? Why should building a software solution be any different?
  • yonixw 4y ago
    > When you go to your mechanic for a repair, do you want to know when you may get your car back? I want! However, for my old 2000 car, repairs exceeded the original estimate 80% of the time, forcing me to change my day-to-day plan. This could be a hint: good/new things breaks less, giving you false confidence when estimating for old/bad stuff.
    • jinmo 4y ago
      100% complexity beats assumptions
  • soulofmischief 4y ago
    Each of the services you mentioned have much better constraints (distance to travel, pages to read, components to replace, etc) They also all suffer from estimation issues in practice. Luckily these constraints set a limit on how much a competent professional can underestimate the project. If you're asking someone to install or update some software they've installed before on a similar system, they should be able to give you a solid estimate. This has parallels with many of the services you mentioned. If you're asking them to research, develop and test entirely new components, which is the basis of software engineering (an engineer does more than install and update packages), you should be able to understand why this is different than delivering food.
    • potamic 4y ago
      I think you're not appreciating the complexity involved in these. Except for delivery, other tasks do involve significant discovery, research and problem solving. Delivery on the other hand is also hard to estimate. Especially when you have multiple deliveries lined up, need to plan your route, find parking, register with building security, manage obscure addresses etc.
• belfalas 4y ago
  Why do software engineers insist on being paid when they never deliver any working software? Imagine a world in which programmers were only paid once their code hit production. My hypothesis is that estimation skills would improve at a geometric rate. Said differently: good programmers are not afraid of making estimates. This classic article says it all better than I could: https://www.kitchensoap.com/2012/10/25/on-being-a-senior-engineer/ https://www.kitchensoap.com/2012/10/25/on-being-a-senior-eng... From The Unwritten Laws of Engineering, quoted in the article: Promises, schedules, and estimates are necessary and important instruments in a well-ordered business. Many engineers fail to realize this, or habitually try to dodge the irksome responsibility for making commitments. You must make promises based upon your own estimates for the part of the job for which you are responsible, together with estimates obtained from contributing departments for their parts. No one should be allowed to avoid the issue by the old formula, “I can’t give a promise because it depends upon so many uncertain factors." Also from the article: Avoiding responsibility for estimates is another way of saying, “I’m not ready to be relied upon for building critical pieces of infrastructure.”
  • pydry 4y ago
    If you deliver incremental improvements frequently enough people stop asking for promises, estimates and schedules. The reverse is also just as true, of course.
  • UK-Al05 4y ago
    If you follow CD practices you will delivering working software all time. It doesn't mean you know what feature is going to be delivered in 32 weeks in the future though.
  • mfuzzey 4y ago
    No one is talking about "never delivering any working software". You definitely do need to deliver working software and should do it in increments. That both shows the customer you are making progress and (more importantly) lets them adjust their requirements as they go along. Software is different from most other branches of engineering in that it is far less limited by physics and that the requirements are far harder to define. For example the requirements for a bridge or a building are relatively straight forward (size, load capacity, climatic resistance etc etc). That's not to say that designing the bridge or the building is necessarilly simple (though it probably is for small bridges and houses because we have centuries of experience with them) but even for complex construction projects the requirements are relatively simple even if the implementation may be very complex (think channel tunnel or mileau viaduc). In software there are much flexibility that most customers don't know what they want until they see it. They generally aren't capable of saying up front what they really want (and when they try to they are often wrong and end up saying "yes, that's what I said I wanted but now I can see it it's not good".
• idonotexist911 4y ago
  Because they're not talking to only Developers but others who think in traditional project terms. Timelines, or at least tentative timelines. Or at least a range. Just something that someone ELSE can plan around. Like Marketing teams who need to know when something will likely work so they can do their own piece of work.
• chrsig 4y ago
  Estimation can be a useful tool for a team to do to ensure that everyone has a shared understanding of a task. Two drastically different estimations can highlight that there may be a lack of understanding or a high difference in skill / experience on the team. Knowing that allows empowers the team to break a task down smaller, make the task more clear, or spend some time educating team members. I don't think those estimates should be shared outside the team.
• hnaccount2001 4y ago
  I’m not sure what the alternative is, in any field, unless your work is so boutique and specialized that nobody else does it (auteur filmmakers, maybe). I used to work in a design firm where estimates were even more meaningless than in software. We all hated estimates but we all knew nothing would get done without them.
• joeyrosztoczy1 4y ago
  Estimates or scientifically wild ** guesses (SWAGs) can be an incredible feedback loop to peers, a team, or an organization. For peers, far apart estimates might indicate differences in understanding of the problem/solution. For a team, if they are wildly diverging or low-certainty it can indicate the right next step is experimentation or exploration, rather than committing on delivering something to production. For an organization, they can provide a barometer for investment decisions, such as what the appetite to solve a given problem space is, or how to evaluate sunken costs. I particularly like a progressive framework on estimates, i.e. 10% certainty when a project is just taking shape, 20% when scope gets tightened up, 50%+ after spikes / rfc drafting / design etc. Often they are used for rote accountability, which can be detrimental and/or naive - they should be a stake in the ground to have discussions around risks, assumptions, and trade-offs!
• errantmind 4y ago
  This take lacks nuance, better to ask why software estimation is hard. To name a few reasons: 1. Business pressures to 'accelerate' timelines without understanding the tradeoffs 2. Frequent developer turnover / ramp-up time 3. Developer + engineering manager level of competence, mostly due to people going where the money is and it being difficult to assess skill level 4. 4-year degrees teaching very few applicable real-world skills and bootcamps teaching even less
• Kkoala 4y ago
  How would rest of the organization work if every new feature and bugfix would never have expected delivery date and everything was done "when it's ready"? I understand your point and estimating is never easy, but software engineering alone won't keep the company a float
• beagle3 4y ago
  A trick I learned a long time ago, that is very helpful in old structured code style (e.g. C, mostly-class-free Python, some subsets of C++ -- but rarely Java) is to estimate the number of non-comment lines for each task, rather than the time needed. This turns out to be much easier to reason about, discuss with colleagues, calibrate one's intuition etc; If one person thinks it will take 100 lines at the end, and another thinks it will take 1200 lines at the end - it provides a more concrete discussion when trying to resolve the difference than "10 days vs. 3 months". And the number of bug-free structured-programming-style lines a programmer produces per day is surprisingly low variance, when measured at the end of a project. (And ... the mean is often in the 1-10 range for good projects and good programmers for places that require high quality like medical devices or aviation). If that sounds too low to you - consider a project 6 years in the making (~2000 days) worked on by 50 programmers. That's 100,000 programmer days. If it has 1M non-comment lines, it's only 10 per days on average. Somehow, I've never been able to apply this to Java code or classful Python code, and have never met anyone who was - I suspect it's because these projects are dominated by boilerplate and huge bureaucracies, which weren't prevalent in the old days when this trick was devised. (Which is not to say that object oriented programming and classful Python and Java are bad; Just that they seem to be resistant to this form of estimation)
• mfuzzey 4y ago
  There are estimates and estimates. As a rough ballpark figure (1 day, 1 week, 1 month, 3 months, 6 months, 1 year, 2 years, 3 years, 5 years) estimates are doable and reasonable. It's also reasonbable and doable to let scope be the factor rather than time. "Tell us the most important thing and we'll do that and deliver the results every couple of weeks, tell us when what you have is good enough". You can also do fairly accurate estimates when you're basically just doing the same thing as you've already done many times or when the customer doesn't have to give much input. Most of the construction industry is like this because it's older, goverend by physics and building codes and we have a few standard models for most things. Most of the software industry isn't like that. However for something new, where the customer doesn't really know what he wants in much detail accurate estimates aren't really possible. In this case anyone who purports to give you an accurate estimate is either lying or using extensive padding. Customers, and non techinical people in general, have a hard time understanding what things are difficult / important from a complexity perspecitve. Often seemingly "simple" (to them) changes are actually very complex, and sometimes the other way.