22 ms·

Why software projects take longer than you think – a statistical model

• mekane8 8y ago
  I used to work for a software company that did projects for clients and charged by the hour (a consultancy) and we frequently had to estimate projects with incomplete information in order to event get the contract in the first place, so they often turned into the actual final budget. Since I was the head of engineering I ended up doing a lot of sales and estimation for new projects. Besides just doing it a lot and gaining experience from many projects, there were a few other things that really helped me: 1) Software Estimation: Demystifying the Black Art by Steve McConnell (already mentioned by others). 2) His short video on "Targets vs. Estimates" was super helpful - we watched it with engineering, sales, and project management all together and had a good discussion afterwards. https://www.youtube.com/watch?v=FY9X21HA02w https://www.youtube.com/watch?v=FY9X21HA02w 3) Keeping a running list of "Things to Remember". Every time a project went astray or we encountered something during a project that I had failed to estimate (but potentially could have) it went on the list. That was useful to share with other too, when they did estimates. I really like the discussion of standing up to those who ask for estimates. A clear understanding of what an estimate is and what it's for is important, as is a strong sense of professionalism. I would recommend "The Clean Coder" by Robert Martin for this. It's more about professional behavior than software practices. Especially his chapters on "Saying No" and "Saying Yes". I read it and discussed it with my team often. It helped us realize when we had to refuse to give an estimate because we lacked the necessary information to do so, rather than just guess or make something up.
• bigred100 8y ago
  What I want to know is why the software developer is expected to give estimates. Estimating how long common tasks take seems like something I would expect a software manager to be able to do effectively and care about. If you’re just a monkey who is asking everyone to do the planning and estimation job for you, I’m frankly unsure why the company allows you to collect a paycheck.
• edejong 8y ago
  Yes, this is a thesis I've brought up quite some times in relation to sprint planning. The odd misjudgment having a strong influence on the total estimate biases upwards, not downwards.
• robbick 8y ago
  One take away is for sprint planning - if there is a reasonable amount of uncertainty on a task, take it out, either break it down or do some investigation, and bring it back next time. You don't want a σ=2 task messing up your sprint!
• iraldir 8y ago
  I believe this is the reason why scrum uses story points instead of time estimate. By putting uncertainty on the same level as effort, you give it more weight. And using a fibonacci sequence rather than a continuous amount with the rule you should round up if unsure tend to correct those defects.
  • js8 8y ago
    Yeah, that's the same reason to use pseudoscience, it's a longer word with "science" in it, which means - more science!
  • IshKebab 8y ago
    Yeah in my experience everyone just mentally translates to time anyway so it is pointless. "11 points, that's about 2 days right?" If you want to make the uncertainty clear it's better to allow ranges, like 2h-5d instead of trying to play stupid mind games. Story points are like setting your watch 5 mins early so you are on time to things. It doesn't work.
    • ricardobeat 8y ago
      That happens indeed, but the point (!) stands: the fibonacci scale ensures estimates are rounded up, and that small abstraction merges time and complexity into a single number, whereas it feels more difficult to bump a time estimate based on unknowns/complexity.
    • GrumpyNl 8y ago
      That's what happens, its all tranlated to hours and cost instantly. I have seen that at several places during poker sessions. The fun starts when management hears this and starts to calculate with it.
  • jfoutz 8y ago
    there is an uncharitable response to this thread, which is marked dead. I do think there is a kernel of truth in that response, which is nothing more than mocking. why fibonacci? I think it's reasonable to say time includes an exponential error. an estimated 1 hour task is very different than an estimated 1 week task. i see that 1 hour, 1 day and 1 week estimates are progressively, and likely exponentially, worse. Is it just the ease of doing the math? (totally reasonable answer, in my humble opinion). or is there something specific about fibonacci that's actually relevant? I think it's the former not the latter. but if you have any evidence to the contrary, i'd love to hear it.
• hnzix 8y ago
  My rule of thumb: take your estimate, double it, then add 20%. I'm not joking.
  • PeterisP 8y ago
    An approach that gives very similar results to yours but is more scientific is to multiply the initial estimates by e (2.718). Or, if you're conservative, then by pi.
    • leethargo 8y ago
      The above approach uses the factor 2.2. Why would your factors be any better?
      • ajuc 8y ago
        Nitpick: x * 2 * 1.2 == x * 2.4, not 2.2
        • apaatsio 8y ago
          Extended nitpick: Technically, they only said "then add 20%". They didn't specify of what that 20% is. It's up to interpretation whether they mean 20% of the original value or the doubled value.
      • alexgmcm 8y ago
        Wouldn't it be 2.4 as you are adding 20% of the already doubled estimate?
        • spurgu 8y ago
          Yes.
      • anoncake 8y ago
        2.2 is just an arbitrary number. e is also just an arbitrary number, but it looks like someone calculated it using sophisticated math.
• onion2k 8y ago
  This is why I like 3 point estimation[1] - if you have optimistic, expected and pessimistic estimates for each task you can pull out which points are high risk. Using a single estimate can't give you that insight. [1] https://en.wikipedia.org/wiki/Three-point_estimation https://en.wikipedia.org/wiki/Three-point_estimation
  • vbuwivbiu 8y ago
    manager: "thanks for the optimistic estimate!"
    • onion2k 8y ago
      Sure, in the same way a bad manager will say "No, that's too high, I'm going to reduce your estimate" if you use a single number. Bad managers are a thing. Make sure you get a good one.
      • nicoburns 8y ago
        Yep. I still remember my first manager, who doubled every estimate I gave him. He was great.
        • onion2k 8y ago
          Definitely a good manager, but Hofstadter's Law says that's still too low. :) https://en.wikipedia.org/wiki/Hofstadter%27s_law https://en.wikipedia.org/wiki/Hofstadter%27s_law
  • jon-wood 8y ago
    I’ve had a lot of success using this method, but it does take having management that understand how software development works. A really good PM will look at an estimate like this as a range, with work complete somewhere between the lower and upper bounds. A bad one will take the optimistic estimate as gospel and build everything else around that.
• boffinism 8y ago
  I've always thought that typically overoptimistic estimates tend to be more based on the mode. I.e. 'this is the sort of task that normally takes 1 day, so I'll estimate 1 day'. The high point on the probability curve is the most noticeable, but it's also way further to the left than either the mean or the median. I have no data to back that up though.
• andybak 8y ago
  I once had a really convoluted metaphor for estimation which involved opening boxes that sometimes contained other boxes which sometimes contained other boxes... I wonder how that models mathematically.
  • joe_the_user 8y ago
    Well, in the most extreme form, you'd wind-up with something like the poisson distribution [1], where your chance of finishing each week would be the same. This is not entirely implausible but it produces effects even more paradoxical/pathological than log-normal. Here, you could give a correct estimate of the expected time for a project to complete as 50 weeks. You could then reach week fifty and again give a correct estimate of 50 more weeks required. And then you could finish the next week. [1] https://en.wikipedia.org/wiki/Poisson_distribution https://en.wikipedia.org/wiki/Poisson_distribution
  • andy_ppp 8y ago
    The problem with this analogy is that the boxes do not obey the laws of physics and fit inside eachother...
    • andybak 8y ago
      Did I mention that they were magic boxes?
• acd 8y ago
  US navy has developed something similar with beta statistical distribution. You estimate "Optimistic", "Most likely" and "Pessimistic" time estimates for each task in the project and then use beta distribution on it. Some tasks take way longer than estimated. Here is the link to the time estimation described above with Beta distribution. https://www.isixsigma.com/methodology/project-management/better-project-management-through-beta-distribution/ https://www.isixsigma.com/methodology/project-management/bet...
  • mattsfrey 8y ago
    I've seen a similar model that I've adopted for my own projections which is 30/60/90 - the 30% chance scenario which is highly successful but less likely, 60% chance scenario that is good with a relatively decent chance of success, and the 90% chance scenario that is the lowest outcome but will almost certainly happen.
  • rorykoehler 8y ago
    I do this but only with best and worst case. Any estimate with a worst case of more than 5 days need to be broken down again unless it's a known quantity. Thanks for the link, I had no idea this was part of 6 sigma
  • natorion 8y ago
    You might also know this as PERT (https://de.wikipedia.org/wiki/Program_evaluation_and_review_technique https://de.wikipedia.org/wiki/Program_evaluation_and_review_...) I used it in the past and I think it is a good framework to discover uncertainty and make it more visible, because it makes you talk about optimistic and pessimistic cases. The estimates are also good enough to come up with a draft schedule.
  • maltalex 8y ago
    I find this approach very interesting, but it hinges on the assumption that project completion times follow a beta distribution. What's the basis for that?
    • kqr 8y ago
• danpalmer 8y ago
  Something I’ve been trying is estimating for the time in which I’m 80% sure I can finish something. I end up racing ahead of estimates most of the time because they are too high, but then some times I’ll find something that is a bit more complicated than expected and it takes a little longer. Overall this seems to balance out, but also have a lot more predictability, it’s easier to predict at any given time what I might be working on. This has been pretty important on my team where I’ve been doing API work for some iOS developers to use.
• ACow_Adonis 8y ago
  Honestly, after doing this whole data science thing for a while now, I'm going to be blunt: I can estimate quite a lot of tasks with quite a lot of accuracy. including software and IT tasks. What I can't do is make bad management hear what they don't want to hear. Nor can I stop people from accepting an estimate because its closer to what they want to be true, because they've already made a promise that conflicts with the actual estimate, or because a certain process requires or necessitates inaccurate estimates. I think the whole "software is hard to estimate" myth stems from 2 fundamental causes: - not controlling for human biases or referencing actual real world data - processes that don't punish/ reward people who provide inaccurate/accurate estimates respectively.
• afarrell 8y ago
  An important aspect of being a professional software engineer is having the backbone to sometimes say things like: - “I don’t know yet enough about the problem to give you even a rough estimate. If you’d like, I can take a day to dig into it and then report back.” - “This first part should take 2-3 days. 5 on the outside. But the second part relies heavily on an API whose documentation and error messages are in Chinese and Google Translate isn’t good enough. I’d need to insist on professional translation in order to even estimate the second part.” - “The problem is tracking down a bug rather than building something, so I don’t have a good way of estimating this. However, I can timebox my investigation and if I’ve not found the cause at the end of the timebox, can work on a plan to work around the bug.” You need to be willing to endure the discomfort of looking someone in the face, saying “I don’t know”, and then standing your ground when they pruessure you to lie to them. They probably don’t want you to lie, but there is a small chance that they pruessure you to. If you don’t resist this pruessure, you can end up continually giving estimates that are 10x off-target, blowing past them as you lose credibility, and your running your brain ragged with sleep-deprivation against a problem you haven’t given it the time to break down and understand. But when you advocate clearly for your needs as a professional, people are generally reasonable.
  • taneq 8y ago
    100% this. It's not your job to say something that will make your client happy. It's your job to tell them your honest assessment of the task.
    • chii 8y ago
      Imagine a tradie saying it'd take 1week to finish remodelling your kitchen, and then it turns out they couldn't. As a professional, it's your responsibility to educate the client on the issue - most technical problem could be explained to a layman if the context is set up properly. The problem occurs when the client doesn't want to hear it.
      • moccachino 8y ago
        From around here that's everyone's experience of tradespeople in every non-emergency task. "Sure I can do this in 3 weeks." Then shows up once every two weeks and finishes around the four month mark.
• StreamBright 8y ago
  There would be great to have predictions that use ML to estimate how long something will take.
  • harias 8y ago
    COCOMO and Function point models use regression. That's ML too.
  • ska 8y ago
    If you are thinking this as opposed to statistical modelling, what is the benefit you imagine?
• usgroup 8y ago
  TLDR anyone?
  • markwkw 8y ago
    Thesis: Developers are good at estimating median time to finish tasks. But the tasks that take longer, in fact take much, much longer than estimated. E.g. Dev estimates that time to complete each of A, B, C tasks will be 2 days. In reality, A will take 1 day, B will take 2 days, but C will take 8 days. Dev was right about the median time to complete each task (2 days) but average was much higher. Article goes into how to statistically model the distribution of actual time to complete tasks.
    • afarrell 8y ago
      That was a really concise but faithful summary. Welcome to HN.
• oli5679 8y ago
  In the UK, bookmakers offer 'accumulator' bets, where a punter can select many outcomes, getting a big prize if 100% correct. This takes advantage punters' failure to accurately multiply probability - 10 events with 80% probability have joint probability <11%. Something similar happens with planning, where people fail to compound many possible delay sources accurately. Dani Khaneman covered this in Thinking Fast and Slow, also showing that people overestimate their ability, thinking they will outperform a reference class of similar projects because they are more competent. https://en.m.wikipedia.org/wiki/Planning_fallacy https://en.m.wikipedia.org/wiki/Planning_fallacy
  • Sahhaese 8y ago
    It doesn't take advantage in the way you say because you get paid along the same lines. If you bet an accumulator with 2 selections at evens, you get paid at 3/1. (4.0 decimal odds) so that is "fair". It's profitable for bookies because they have a house edge, and that edge is increased the more subsequent bets you make. The house has more edge with an accumulator than a single bet. People like to do accumulators because it's more meaningful to win large amounts occassionally than more regularly win less meaningful amounts. So it's a "trick" to simply increase gambling. If you had to pick out 8 evens shots in sequence and intended to roll over your bet each time it would have the same effect / outcome, but starting with a pound by the last bet you're effectively placing a 128 quid bet on an evens money. It's not that the player thinks that they have a better chance of winning than 1/256 it's that it effectively forces them to gamble a lot larger amount in the situations where only 7 out of 8 of their picks come in. And that's before considering the edge. If we consider that these are probably events that happen more like only 45% of the time (at best) then instead of a 255/1 shot we're looking at 600/1 shot.
• mikekchar 8y ago
  The interesting thing is that by the central limit theorem, the mean of a mean is normally distributed. This is extremely helpful. Here's what I suggest you do: Same size your stories to small values. Do 30 stories in a sprint and take the mean. Do 30 sprints and take the mean of the sprint. What you get is the mean amount of time to do a sprint of 30 stories. What's amazing is that this estimate will be normally distributed. You can measure the variance to get error bars. Of course 900 stories to get good estimates ;-) However, imagine that your stories averaged 2 days each. Imagine as well that you have a team of 10 people. That means that you will finish a "sprint" of 30 stories in 6 days (on average). 30 sprints is 180 days -- the better part of a year, but you probably don't need a 95% confidence interval. You will find that after a few sprints, you'll be able to predict the sprint length pretty well (or if you set your sprints to be a certain size, then you will predict the number of stories that will fit in it, with error bars). The other cool thing is that by doing this, you will be able to see when stories are outliers. This is a highly undervalued ability IMHO. Once a story passes the mean plus the variance, you know you've got a problem. Probably time to replan. If you have a group of stories that are exceeding that time, then you may have a systemic estimation problem (often occurs when personnel change or some kind of pressure is being applied to the team). This kind of early warning system allows you to start trying to find potential problems. This is really the secret behind "velocity" or "load factor" in XP. Now, does it work on a normal team? In my experience, it doesn't because groups of people are crap at calmly using statistics to help them. I've had teams where they were awesome at doing it, but that was the minority, unfortunately.
  • piccolbo 8y ago
    The central limit theorem is in the limit for the number of variables in the sum approaching infinity. In the finite world, the article explains how it's done. The article is saying, the sum of lognormals is not normal. You are saying: take enough of them and it is normal. The article is still more accurate than your reasoning for 30 stories. From the wikipedia entry for Central limit theorem " As an approximation for a finite number of observations, it provides a reasonable approximation only when close to the peak of the normal distribution; it requires a very large number of observations to stretch into the tails". To prduce a 95% confidence intervals, you have to upper-bound the tails. All methodologies that are based on sum of subtasks estimates are not evidence based. But we knew already sw methodologies are not evidence-based, did we?
• scandox 8y ago
  In the world of small medium projects often the major issue is that software engineers give estimates for writing the software but customers take that to mean time to actual delivery in production and a lot of the time have no idea how big a task deployment and integration are...or don't even have a plan for that.
  • maxxxxx 8y ago
    in medical devices it usually takes five times as long to really finish the project vs. finishing development.
• teddyh 8y ago
  According to Joel Spolsky¹, programmers are generally bad at estimating, but they are consistently bad, with the exact factor depending on the individual. So by measuring each person’s estimate and comparing it to the actual time takes after the fact, you can determine each person’s estimation factor, and then when they estimate again, you can get a pretty reliable figure. 1. https://www.joelonsoftware.com/2007/10/26/evidence-based-scheduling/ https://www.joelonsoftware.com/2007/10/26/evidence-based-sch...
  • tonyedgecombe 8y ago
    You know that article was written to sell a feature in their bug tracker. I like Joel's writing but I'd take that piece with a pinch of salt.
    • teddyh 8y ago
      If we try to be a bit charitable, we could assume that they implemented the feature in the bug tracker because of this observed property of estimates.
    • arsenico 8y ago
      True, but you should always put up estimated effort and real spent time in the task, whichever bug tracker, or issue tracker you're using. I don't know of any other way to improve the estimation "skills".
    • madhadron 8y ago
      I tracked this myself for a few months and found that I was fairly consistently estimating around twice the actual required time. That was for fully scoped out and planned work, mind you (and that I was the one who scoped and planned).
  • rightbyte 8y ago
    "As estimators gain more experience, their estimating skills improve. So throw away any velocities older than, say, six months." I don't know how Spolsky is (was) as a boss, but it's is way easier to just adjust your time reports according to the estimates and make your boss happy than to actually try for no benefits to either you or the project. As long as the estimation errors are white noise-ish and in linear relation to actual task length, which is the assumption for his model to work, manipulated time reporting and not manipulated make no difference. I've noticed that when programmers are introduced to Agile they initially are bad at time estimates until they learn that you actually can report time according to the estimates and no-one will ever notice. Extra points for looking on the burndown chart before reporting time to make it smooth. "Some developers (like Milton in this picture) may be causing problems because their ship dates are so uncertain: they need to work on learning to estimate better. Other developers (like Jane) have very precise ship dates that are just too late: they need to have some of their work taken off their plate." Milton is probably doing fair estimates. When planning four months into the future you probably can say where you are within 2 months with 50% probability (or what ever boxplot he uses) of where you think you will be at that time. Janes reporting is obviously manipulated to match the estimates.
• settsu 8y ago
  I’ve been coding professionally for 20 years and I’m largely no better at estimating completion than day 1, maybe worse since I’m more likely to feign confidence in a figure I’ve essentially pulled out of my ass. At some point, I came to the realization that my fundamentally poor concept of time was always going to be an insurmountable obstacle to my career advancement.
• arendtio 8y ago
  Well, I think this is a complex topic. Not because of the math, but because the key to an accurate estimate is to understand who has done the estimate and on what basis. As stated in the summary, the core driver for inaccurate estimates is uncertainty: > Tasks with the most uncertainty (rather the biggest size) can often dominate the mean time it takes to complete all tasks. There are different sources of certainty: - Experience: If someone has done a task 20 times he probably knows how much time he will require. Someone who hasn't done the task yet, probably underestimate the time he requires (e.g. because of the median vs. mean conflict). But don't be fooled just because you have 20 years of work experience in the field but never actually done a specific task doesn't mean you can estimate it better than someone who just started the job but had done that specific task 10 times. However, most of the time, projects are doing something new. So you have to find out which tasks have been done before by a project member and which are completely new ground. If something is completely new, remember to plan time for getting familiar with a problem space plus a handful of complications (together this will be more than the actual task would take someone who is trained for that specific task). - Detail: The smaller the tasks the larger the overall estimate... or so. Planning on a top-level is rarely going to be accurate. We do it a lot because it doesn't take much time. But if you want an accurate estimate, you have to plan on small, specific tasks. - Risk management: Every project has risks. Some don't really have an impact and others blow up the whole project. Know your risks and what you are going to do if something should go in the wrong direction. It is not like you wouldn't have time to figure out what to do when the problem occurs but to understand how it would impact your timing and to take preventive actions (e.g. include stakeholders). If you have people who have done the exact same task a few times, made a detailed plan of every step and know how to handle the most likely or impactful risks you are in a good position to deliver on time. Most of the time you won't have that luxury and have to compensate the resulting uncertainty with a prolonged time to project completion, but that should be just fine when being communicated in the beginning. With all that said, remember, that some projects don't require an accurate estimate. Sometimes it is enough to deliver just as soon as possible.
  • helloindia 8y ago
    On the experience part, this happened to me few times, when the project manager asked me for an estimation, i give it from my perspective and experience, and then he gives the work to someone else with no experience, and obviously takes more than estimated. Now, i always give two estimates: An estimation if the work is done by me, and an estimation if the work is done by someone else.
• zcanann 8y ago
  I always think of project tasks as flow charts, where every item either takes 1 day, or 1 week. There's no way of really knowing in advance. Complications happen. It makes it really hard to calculate the "expected value" of 5-10 tasks.
  • dTal 8y ago
    The more tasks you have to do, the more certain about duration you should be - some of the uncertainty will cancel out and you will get a gaussian distribution. For your example, I expect 10 tasks of between 1 day and 1 week each (with flat probability) to take about 6 weeks in total, with a 95% chance of completion within 7 weeks.
• Michielvv 8y ago
  I think the most important issue why projects take longer is not because the time it takes to complete a task is uncertain, but because at the start it will always be unknown which tasks will prove critical. The further you get, the more tasks will reveal itself that were not part of the original scope, but critical nonetheless.
• af1991 8y ago
  In my opinion, the reason that software projects take longer than we think and estimate is due to the fact that companies don't know how to handle Underemployment properly...
• chiefalchemist 8y ago
  The why is typically: 1) The person(s) doing the estimate aren't qualified to do so. 2) There is a disconnect between wishful thinking and reality. 3) There is some arbitrary future milestone (i.e., "We need to ship by ____ because ____ is happening the following week.") that is independent of software development. 4) Most importantly, when the deadline is missed the estimate / estimators are not questioned, the software team is. I've been at this a long time - too long? - and the narrative that IT is __always__ at fault is a myth that needs to be buried.
• juskrey 8y ago
  Because they can't take negative time.
• revskill 8y ago
  To me, software projects take longer than i think because customers don't know what they actually need until there's a runnable version of what they want.
• nocturnial 8y ago
  Wouldn't it be more sensible to give a range instead of a fixed date? I know it's not going to happen, but I think it would be more informative and honest. That way you could communicate better of how certain you are. There's a difference between telling something will be completed in, for example, 6 month +- 2 weeks. Or something will be finished in 6 months +- 1.5 months. The estimated time is the same but communicates the level of certainty more clearly. Or just use between, for example, 4-6 months if you don't want to use +- notation.
  • DougWebb 8y ago
    In my experience, developers understand and appreciate range-based estimates. But when those numbers start moving up the communication chain, some non-developer is going to either not like or not understand the point of the range, and will convert it to a single number: either the first one, the last one, or the average. They might even be honest about it, thinking "I need to know the earliest possible date, so I'll use the low end". But then the next person, who doesn't see the range, thinks that low end is THE committed date and will plan accordingly. Now your deadline is 99% likely to be missed.
  • trustfundbaby 8y ago
    Never works. Management will just use your lowest estimate. So you pad that and realize its the same as your high estimate, so you just give them that instead ... then they complain its too high ... rinse repeat. Unless you stand you ground.
• SiempreViernes 8y ago
  Fits symmetric function to clearly asymmetric distribution Author: Decent fit, in my opinion! This bad fit makes me genuinely sad (；∩；)
  • ben509 8y ago
    He could probably tweak a skew normal distribution to make it fit nicely, but it's pretty close to normal.
• basetop 8y ago
  The mythical man month ( required reading for most CS programs ) goes into a historical and production aspect of why software projects take longer than what you think and what you expected. Also, there is a law named after author called the Brooks's law : "adding human resources to a late software project makes it later" https://en.wikipedia.org/wiki/Brooks's_law https://en.wikipedia.org/wiki/Brooks's_law In most industries, if you are running behind schedule, you throw more workers at the problem to catch up. For example, laying railroad tracks, digging ditches, deliverying packages, harvesting crops, etc. By adding more workers, you shorten the time it takes to complete the project. But which software engineering, the reverse tends to happen. If you are falling behind, just throwing more developers at the problem worsens the problem. Most likely because you need the new developers to get "caught up" with the code/project/tools but if you rush that process, then they won't have a full understanding of the project/code/tools and introduce bugs/problems themselves which exacerbates the problem. It's a fun read if you have the time.
  • v3gas 8y ago
    > required reading for most CS programs Really?
  • rainhacker 8y ago
    Given most software project estimations are off, wonder if a corollary of Brook's law can be - don't add resources in later stages of 'any' software project.
    • bostonvaulter2 8y ago
      Ah, but how do you know what stage of the software project you're in? Are you 3 months into a 4 month project or are you 3 months into a 5 year project?
      • rainhacker 8y ago
        Maybe, instead of asking - if I'm x months into a y months project (assuming y months is the initial estimate) - ask if the project is x% feature complete. Based on the remaining features identify how far the project has progressed. Though, this approach has the problem of scope creep. As the requirements are fluid, especially in a long-running project.
• temp269601 8y ago
  How about making the manager estimate the project, that way if the deadline is not met, the manager receives the blame? It's the manager's job to manage resources, and if the deadline is not hit, then they can hire/bring on more resources. If an engineer works as hard as they can for 40 hours a week, why is it the engineer's fault if the arbitrary deadline is not met? If the engineer estimate's time for a project, the engineer will always have to work more than 40 hours a week because some estimates will be too optimistic.
  • jcon321 8y ago
    Maybe if your PM has any technical experience
  • ben509 8y ago
    The manager does receive the blame, and then stuff rolls downhill.
• pikzel 8y ago
  I've been in a company where all estimates were multipied by pi.
• deleted 8y ago
  [deleted]
• Chris2048 8y ago
  I'm sorry to contribute to the dogpile effect (long thread probably says same thing I'm about to say, but I didn't see it..), but.. devs estimate known risks. Ideal path + predictable delays. The further reaches of the long tail are the unknown risks. known risks are estimated based on knowledge (hence, a question for a dev), unknown risks are just an adjustment parameter on top of that estimate, possibly based on historical evidence (there is no reason a dev could estimate any better). It should be managements job to adjust a dev estimate. Let's be real here - I've never heard a real life example of management using stats for this kind of thing, or being enthusiastic about devs doing the same. Perhaps if management is taken seriously as a science, things will change, but I doubt it. <strong_opinion type="enterprise_software_methodology_cynicism"> Bizness is all about koolaid-methodology-guru management right now, very much the bad old ways - a cutting example of workable analytical management would be needed for things to change, but this is unlikely as all the stats people are getting high pay cool ML jobs, and aren't likely to want to rub shoulders with kool-aider middle managers for middle-management pay.. </strong_opinion>
  • ska 8y ago
    It's actually not super uncommon for management to use statistical tools for this, although they may not realize what exactly is going on. For example there are a couple of tools whose names escape me at the moment that extend or wrap MS Project but have a statistical time estimate piece based on quintile time estimates (e.g. your engineers give you 50% and 90% conf estimates, it applies simple modeling). I've also been at one shop where we used a more sophisticated modeling approach, but built in house. Reception was warm in at least most of management. So my experience isn't that these tools don't exist, or that management is unwilling to use them - but rather that management has trouble accepting the implications of using them properly. Specifically, if any of the inputs of the model change, you should update those and re-run the model. With linear (e.g bullshit) GANTT, you just say "oh, we missed this week of work, move the endpoint forward a week". With more careful modelling adding an unexpected dependency or a couple days work in the wrong place can suddenly add 3 months to your worst-case model. Exec's really don't like that happening without a whole song and dance about why, so there is a tendency to "freeze" the models in ways that they become progressively less useful. Worst case they are asymptotic on meaningless.
• bunderbunder 8y ago
  I've been one place that I thought was really good at software estimation. Their system was: Everything gets a T-shirt size. Roughly, "small" is no more than a couple person-days, "medium" is no more than a couple person-weeks, "large" is no more than a couple person-months. Anything beyond that, assume the schedule could be unbounded. Figure out how to carve those into a series of no-larger-than-large projects that have independent value. If they form a series of iterations, don't make any assumptions about whether you'll ever even get around to anything but the first one or two. That just compromises your ability to treat them as independent projects, and that creates risk that you find yourself having to worry about sunk costs and writing down effort already expended when it eventually (and inevitably) turns out that you need to be shifting your attention in order to address some unforeseen business development. At the start of every quarter, the team would commit to what it would get done during that quarter. There were some guidelines on how many small, medium or large projects they can take on, but the overriding principle was that you should under-promise and over-deliver. Lots of slack (1/3 - 1/2) was left in everyone's schedule, in order to ensure ample time for all the small urgent things that inevitably pop up. There was also a log of technical debt items. If the team finished all their commitments before the end of the quarter, their reward was time to knock things off that list. Best reward ever, IMO.
  • SketchySeaBeast 8y ago
    > Everything gets a T-shirt size. Roughly, "small" is no more than a couple person-days, "medium" is no more than a couple person-weeks, "large" is no more than a couple person-months. That's pretty much exactly what I've ended up using on past projects - a little more fine grained (start at a half day, went up to months), but that was my approach as well, and if I didn't know everything it went up a size.
    • karthikb 8y ago
      Did you find much differentiation between half day and couple of days? Especially because some things that might take a couple days end up taking 30 mins (some efficient package already exists), and some half day things end up taking a couple days, so it comes out in a wash?
• mlthoughts2018 8y ago
  I’d also add that the reason why the mean will be larger than the median is usually hard to discern for any specific situation. This foments bikeshedding debates that are a mix of business pressure to favor uninformed estimates and disagreements about which possible sources of surprises or hitting a wall are most likely. As soon as you mix this with a formal estimation system like Agile (yes, even the platonic ideal ‘agile’ too), it creates a snowball effect of time wasting because overhead is so frequently required to resolve debates and placate business pressures for estimates.
• wellpast 8y ago
  > Instead, figure out which tasks have the highest uncertainty – those tasks are basically going to dominate the mean time to completion. From the technical side of things, uncertainty can mean a few things here: (A) I've never done this kind of task (or I don't remember or didn't write down how long this task took in the past) (B) I don't know how to leverage my historic experience (e.g., implementing an XYZWidget in React and implementing the same widget in Vue or Elm for some reason take different amounts of time) Considering (A)... Rarely does a seasoned developer in the typical business situation encounter technical tasks that are fundamentally different than what has been encountered before. Even your bleeding-edge business idea using modern JS' + GraphQL' is still going to be built from the same fundamental pieces as your 1999 CRUD app using SOAP and the estimates are going to be the same. If you disagree with this you are in the (B) camp or you haven't done the work to track your estimates over time and see how ridiculously accurate estimates can be for an experienced practitioner. Even "soft tasks" like "design the widget" are estimable/repeatable. This whole you-can't-estimate-software accuracy position is entirely a position of inexperience. And of course all bets are off there. You are talking about estimating learning in this case, not doing. And the bets are especially off if you aren't modeling that these are two different activities: learning and doing.
  • afarrell 8y ago
    > is entirely a position of inexperience There are a lot of inexperienced software engineers and very little good guidance written for them. What is a new CS grad to do when asked for an estimate? How can a new grad learn to produce accurate estimates within 3 months?
    • wellpast 8y ago
      A problem with our industry in this regard is that we don't understand the difference between learning and doing. A new grad entering industry is going to be doing a lot more learning than doing -- or rather learning while doing. I know from experience that explicit doing is highly predictable/estimate-able for the experienced software practitioner. I have a suspicion that the learning side would be predictable, too, if our industry could do a better job of articulating what the practitioner skill set actually is -> then a pedagogy could develop and it could be said that learning X-Y-Z takes on average this long for a set of students. Etc. But we as an industry do not seem to be near this level of clarity -- in large part because we don't even have the vocabulary to frame it this way... in terms of learning vs doing... Now what this means for the new CS grad is not the best story. You'll rather have to play the chaotic game a little bit, which includes a mess of things like doubling estimates and working weekends or what-have-you depending on the work culture you find yourself within. That ^^ in the short term. In the long term what you should do is practice on your own: 1) ALWAYS privately plan and estimate your tasks to the best of your ability--on your own, you may not get benefit by exposing your "practice" to the higher-ups 1a) hint: your tasks should be as scoped/small as you can make them and they will look pretty simple, like this: "design widget X", "design API interface", "implement API interface", "test API interface", "implement widget Y", "learn how framework X lifecycle works" (yes! even learning tasks can be estimated!), and so on. The key is that you try to keep them under a day and ideally even smaller on the order of hours. 2) RECORD the time it takes and compare to your estimates. --> LEARN from this. 3) REPEAT If you do this conscientiously you will find that your estimates improve dramatically over time and you'll be able to out-estimate your peers to their own wild confusion. This skill set will pay off in your ability to deliver AND have weekends and hours with your family. Because you will be able to "see time" and protect yourself. You'll have protection and ammo even in the worst pressure-cooker environments because you will be able to say "No" with confidence. Or rather you will learn how to say "Yes" and what to say "Yes" to. (Ie. you will get really good at priority negotiation etc.) And you'll cultivate delivery trust with superiors and everyone will be happy. The main reason you get these claims that "business just doesn't understand software" and "they put too much pressure on us" is because "business" side doesn't trust us. Once you get good at estimates and delivering on them, that trust is restored and everyone can be made happy. BUT -- and here's the rub -- it takes time and conscientious effort and variety of experience to reach this level of confidence. My advice: Ignore all the philistines who say it can't be done....because they'll just try to talk you out of the effort that they weren't or aren't willing to do themself.
• yawz 8y ago
  Even the language is not correct: We all call it "estimate", but stakeholders behave like it's a "commitment". Passage from uncertainty to certainty happens in the language, and all the responsibility is on the engineering team's shoulders.
  • adrianmonk 8y ago
    Even if you choose the right words, people don't necessarily pay close attention. And if they want a commitment, they may assume that the numbers you give them are a commitment regardless of how you phrase it. But even if they did listen closely to what you said, "estimate" is not even a great word. If your car is in a wreck, a body shop gives you an estimate to fix it, and they may treat that estimate as a commitment. It's pretty common practice that people are held to an estimate, or to not going over by a certain small percentage. Maybe we need a word like "forecast" or "prediction".
    • Bjartr 8y ago
      > Maybe we need a word like "forecast" or "prediction". Scrum (The org behind that flavor of agile) actually changed the language used in their guide back in 2011 from developers making a "spring commitment" to making a "spring forecast" for exactly these reasons. https://www.scrum.org/resources/commitment-vs-forecast https://www.scrum.org/resources/commitment-vs-forecast
  • c0vfefe 8y ago
    > stakeholders behave like it's a "commitment". Many places treat budgets the same way.
• cryptokernel 8y ago
  Because it is costly and full of bugs Just leaving this here: Do tasks, get paid. Join us in the CoinCircle website to earn your crypto https://coincircle.com/l/50VVxbObg3 https://coincircle.com/l/50VVxbObg3
• adrianmonk 8y ago
  One strategy for dealing with risk is to order your tasks so the highly variable ones come first. That way, as you progress through the project, you eliminate a lot of unpredictability, and it's smoother sailing toward the end. However, as the tables in this article show so clearly, one task can dominate others. This serves as a great illustration of a perception problem the above strategy can create. Outside observers watch your progress and will probably evaluate it in terms of number of tasks completed. You have (say) 10 tasks to complete, and from their point of view, all they know is a really long time has passed and you haven't even completed 1 of them! Since they are further removed from the situation than you are, they are almost guaranteed to not appreciate why the variable task is that way. They're likely to just think your team is performing badly. So, maybe it's better to order tasks so that risky stuff is spread more evenly across the timeline of the project. It could create less confusion. Putting risk at the beginning is a strategy that requires a whole lot of trust and buy-in.
• unityByFreedom 8y ago
  Eh don't worry, self driving cars are coming this year, Elon said so.
• diiq 8y ago
  Vistimo.com, the tool I built and use when I run estimates for my clients, uses log-normals and monte-carlo simulation to conquer this exact problem. Really exciting to see other people beginning to recognize and use the same statistical tools -- they've served me really well.
• achenatx 8y ago
  The assumption in the article is the amount of work is a constant. Instead what happens is that a manager has decided how much time it should take (always too little). If you accept that time, then you will go over. If you push for more time and get it, then the manager will eventually add more scope and you will go over. The vast majority of the time we see projects go over deadlines because scope was added under the guise of clarifying scope. Starting with a business value actually works pretty well to enable people to control scope. We also use actual velocity on tasks to re-forecast daily. That has been the most effective way to get a good date. Over a large number of tasks this works very well.
• pjungwir 8y ago
  Here is a story about the woes of interpreting statistical distributions: I have two habits when I estimate: First, I like to give a range or even better a triple estimate of "optimistic", "expected", "worst case". (And btw, you should expect things to weight toward the pessimistic side, because you always find new requirements/problems, but you rarely discover a supposed requirement is not really one.) Second: I like to break down a project into tasks of just a few hours and then add everything up. I usually don't share that spreadsheet with the customer, but it helps me a lot. Pretty much always it gives me a number higher than I'd like, but which is almost always very accurate. A couple times I've ignored the result and given a lower estimate because I really wanted some project, and it has always turned out the spreadsheet was right. Well, one time I combined these two approaches, so that my very finely-chopped estimates all had best/expected/worst values, and I shared that with the customer. Of course they took one look at the worst-case total and said, "How can a little thing like this possibly take 2 years??" I didn't get the work. :-) EDIT: Btw it feels like there is a "coastline paradox" here, where the more finely you estimate, the higher the max possible, so that you can make your estimate grow without bound as long as you keep splitting items into greater detail. It'd be interesting to see the math for that.
  • pjungwir 8y ago
    EDIT2: In spite of my personal experience I do think the author makes a strong case for this: "Adding up task estimates is a really misleading picture of how long something will take." Perhaps I've had better results because I try to give myself a little padding in every task, just considering that everything requires not just typing code but getting on some calls, having back-and-forth in emails and the project management tool, testing (automated or manual), fixing a thing or two, etc. So my individual estimates are probably a bit higher than median. When I work with other developers they consistently estimate lower than me. But my numbers are deliberately not "best case", because then you know you'll go over on the total.
• SilasX 8y ago
  Pet theory: this is entirely explained by unknown systems not behaving as expected. As developers, and unlike e.g. carpenters, we are constantly using new tools with effects we haven't yet experienced. Then we have to yak-shave to get around their heretofore unknown kinks. Then the time blows up. If and when you're using known features of a known framework, and that's all you're doing, the estimates are accurate and the work is completed quickly.
  • maltalex 8y ago
    I disagree. Estimations tend to be just as wrong even when the tools are well known. There's always that one edge case you haven't considered, that one algorithm that doesn't work as well as you expected, that small change to the requirements the requires a completely different approach.
    • SilasX 8y ago
      "Just as" wrong? I don't know what to point to to resolve disagreement here since it's just anecdotal, but if you're just using the same feature you've used hundreds of times before, there is nowhere near the potential for yak-shaving snags.
      • maltalex 8y ago
        Okay, sure. Not "just as" wrong since a certain class of pitfalls are eliminated by deep knowledge of your tools. But I'd argue that that's not where the bulk of the uncertainty comes from. I think that in the software industry as a whole, most of the problems are solved by people with at least a decent understanding of their tools, and estimates still suck. The problems is that the tasks we deal with are always new, unsolved problems. That's the nature of software. Unsolved problems come with uncertainty. That's the nature of unsolved problems. Carpenters on the other hand deal mostly with solved problems. They just need to execute.
• jermaustin1 8y ago
  In my experience software takes longer to build than original estimates because no one will get out of the way of the development team and let them work. This is an extreme example, but one I now live in daily. My current full-time-ish gig is working on a pretty enterprisy system for law enforcement. To this date there hasn't been a single feature request, or bug fix that took more than 16 hours of development time. And so I know that I can typically finish something within a few hours to a day of receiving the task. UNLESS my manager wants to discuss ad nauseam what he means when he says "intersect an array". Or get stuck in 2 day long code reviews where my manager makes me sit behind him while he goes over every single line of code that changed, then gets side tracked and starts checking emails, chat messages, text messages, calling other developers in to check on their statuses, and even watching youtube... while I'm stuck in his office waiting on my code review to be done so I can go back to my 5th day of trying to complete a task that would have taken only a couple of uninterrupted hours. /rant And this is why I pay $120 a week for therapy.
  • stronglikedan 8y ago
    I give estimates in hours. They're really by days for large projects and half days for smaller projects, but I express them in hours. I then provide a breakdown of the hours the team has been able to spend to date, as well as the hours that each forced interruption took. That way, when my manager asks why he perceives us as being behind on the project, I can tell him exactly how long his pointless meetings and changing priorities have delayed the team. It's also a good CYA technique for when he tries to blame the team when his manager asks why the project is delayed.
  • pysxul 8y ago
    Sorry but why would a manager even do a code review?
    • jermaustin1 8y ago
      He was originally hired as the sole developer 10 years ago, but the project grew too big, and instead of hiring a manager to oversee the project and hire more devs, they moved him into a management position, and put him in charge of hiring new developers.
• mbesto 8y ago
  As always, my favorite article on this subject: https://www.lesswrong.com/posts/CPm5LTwHrvBJCa9h5/planning-fallacy https://www.lesswrong.com/posts/CPm5LTwHrvBJCa9h5/planning-f... > A clue to the underlying problem with the planning algorithm was uncovered by Newby-Clark et al., who found that Asking subjects for their predictions based on realistic “best guess” scenarios; and Asking subjects for their hoped-for “best case” scenarios . . . . . . produced indistinguishable results. > So there is a fairly reliable way to fix the planning fallacy, if you’re doing something broadly similar to a reference class of previous projects. Just ask how long similar projects have taken in the past, without considering any of the special properties of this project. Better yet, ask an experienced outsider how long similar projects have taken.
  • jfehr 8y ago
    Daniel Kahnemann calls this the "inside view" and "outside view", from his book Thinking Fast and Slow. The relevant excerpt (mostly an anecdote that serves as an introduction to a whole chapter about it) can be found here: https://www.mckinsey.com/business-functions/strategy-and-corporate-finance/our-insights/daniel-kahneman-beware-the-inside-view https://www.mckinsey.com/business-functions/strategy-and-cor...
• maltalex 8y ago
  My theory on why software estimates suck is tied to Rosenberg’s Law: > Software is easy to make, except when you want it to do something new. The corollary is, The only software that’s worth making is software that does something new. It's hard to estimate the difficulty of something you've never done. And the nature of software development is to always do new things.
• gilbetron 8y ago
  It takes longer than we think because writing software is solving a math problem, and you can't know how long solving a math problem will take until after you solve it. http://www.warhound.org/kcsest.pdf http://www.warhound.org/kcsest.pdf
• OnlyRepliesToBS 8y ago
  estimates are stupidity codified in language
• jrochkind1 8y ago
  1. Do the tasks with, as far as you can tell, the most uncertainty as soon as possible in the process. 2. This is why "agile" tries to avoid estimating more than a few weeks. The more tasks you have in there, the more you estimate is likely to be really off.
• dre85 8y ago
  I'm blown away on a regular basis by how long it takes to write software that in my opinion is super easy. Usually it just comes down to fuzzy/missing/changing requirements. A lot of times people know that they want something, but they don't know exactly what. Or they're absolutely sure they need feature x, but then later they realize they don't, but they missed out on developing other more fundamental features.
  • adrianmonk 8y ago
    I have developed a belief about this: people don't know what they want until you show them what they said they want. Then it's immediately obvious to them what they wanted instead. This suggests that demos and mock-ups might be a valuable tool. The sooner you can get someone to try something, the sooner they can tell you what direction they really wanted you to go in instead, and the less time you waste.
    • scruple 8y ago
      They want everything but they can't even define a starting point. It's insanity. I've been going back and forth with a customer about this (through my product and sales team) since August of last year. At first the priority was high and it seemed like the scope was reasonable. But no one was making moves or delivering on my requests / asks / concerns. I suggested a phased approach, so that we could _be_ agile and get _something_ in front of the customer for feedback. But before I even got there the requirements changed again and again and again and again and again... Thankfully all of this specific work is easy to segregate away from the rest of my team. It's proven to be very toxic and morale draining. If my entire team was involved, I've no doubt that we'd have lost people by now. And the real kicker: It turns out that right now we can't even deliver on the first pass that I had originally suggested, despite me being basically done with my teams piece, because some other team in the company was loose with their language and convinced a bunch of product and sales folks that, yes, of course they had what the customer was asking for. They didn't. They still don't. I'm convinced today that they never will. Bunch of fucking sycophants. I have no idea how these things happen but recently I've become convinced that this sort of confusion-on-all-fronts is just par for the course in this industry today. The only work that I've been able to do in the past couple of years that was well understood and easy to articulate, and as a result capable of being completed mostly on time and within scope, was born out of a select few individuals being able to identify a real problem and a real solution and grinding away at it in a controlled fashion. But that seems to be rare and not at all "how things are done."
• twothamendment 8y ago
  They don't take longer than I think. The problem is nobody listens! True story - They handed me an RFP, asked for an estimate. I gave it and they cut it in half and got the job. I didn't take any pleasure in being right, but I was right. It was an expensive mistake.
• Farradfahren 8y ago
  Frankly, where does this obsession to plan to force others to guess about there time originate? From a honst wish to know project time? No, for that you simply would have to take the jobs apart, looking for stuff never done before. After that you know how much your estimate can be right and wrong. But you want your developers to feel bad and put under pressure by themselves, thats why it has to be them, who guesstimate the unknowable.
• JabavuAdams 8y ago
  My empirically-confirmed heuristic is that the time to deliver a feature set that someone would actually want to use is 2.5x-3x of the time I think of when asked for an off-the-cuff estimate. Basically, multiply initial estimate by a number between e and pi -- no joke! It's a bit of a problem given that PM's think they're being generous with a 20% pad.
  • rainhacker 8y ago
    I can relate to your heuristic. The current project I've been working on is close to completion and somewhere between 2.5-3x of its initial estimate.
• bloorp 8y ago
  One big factor that I don't THINK the article touches on is how we are using 'space' metaphors (speed and distance) for our work that's very 'time' based (working productivity and duration). And I think when we estimate, we try to estimate distance/duration but we forget that we're really trying to estimate our speed/productivity. Where that gets painful is, say you estimate you can get something done in two days. In reality, you're twice as fast (there's that speed metaphor), so you actually get it done in one day. Yay, you saved a day! Now assume you're twice as slow as your two day estimate. Boo, you spent TWO days longer. So in terms of duration, getting it incorrect in the painful direction seems like a bigger mistake. I don't think this is the same phenomenon as the author's mean vs. median dilemma. I'll bet both the mean vs. median and the productivity vs. duration dilemmas are real factors though.