6 ms·

Project delays: why good software estimates are impossible

165 points by SarasaNews 10y ago

paulsutter 10y ago
Evolution. 100,000 years ago, there were humans who could accurately estimate the difficulty of, say, catching a deer. And since they knew it was too much trouble, they just stayed home in the cave. Meanwhile some crazy optimist went out and to catch a deer. It was many times more difficult than expected, but they returned to the cave with the deer and eventually reproduced more successfully than the accurate estimator. Thus our ancestors were the crazy optimists.
- pliny 10y ago
  Or the estimators only hunted deer when they were well positioned to do so (e.g. few predators of humans and many deer around), and the optimists were eaten by bears.
- zby 10y ago
  That does not make sense. If eventually it turned out that it was worth it, then their estimate - i.e. 'it is worth it' was accurate - not too optimistic.
- smallnamespace 10y ago
  > And since they knew it was too much trouble More trouble than starving?
- redler 10y ago
  Another way to look at this is that the crazy optimists who are the worst estimators never return (themselves eaten by predators, run out of water, hypothermia) or return empty handed, lowering their reproductive success. The optimists who succeed quickly adjust their understanding of the estimating process ("Zog died when he went after deer at night during winter, and I barely survived being attacked by wolves when the deer were most plentiful, so I should go on cool midday hunts when the deer are already slightly down in population").
- Rexxar 10y ago
  That's the one who has domesticated the cows instead of hunting it who has win.
- ryanmarsh 10y ago
  Or the ones that stayed back and saved their energy to kill the weakend hunter and take the meat.
maxaf 10y ago
I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe. My criticism might seem nitpicky until it's time to measure this article's message against the possible reaction of an engineering or business manager. Launching into a systematic dismantling of the pervasive estimates culture is best done with humble, verifiable assertions and helpful suggestions in tow. Attempting to recruit the long-dead Einstein is unhelpful and hand-wavy. There must be a better way to package the no-estimates message, and it's not this.
- ktRolster 10y ago
  I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe. Indeed, most projects are remarkably (and sadly) similar to other projects we've done before.
  - markbnj 10y ago
    It doesn't much matter how "similar" they are. Unless you can copy and paste the code you're building new stuff and running the risk of encountering all the issues inherent in building new stuff.
- woodchuck64 10y ago
  How about: the reliability of software estimates are inversely proportional to creativity requirements, with a unified theory of physics establishing an upper bound on creativity with a corresponding 0 for the estimate reliability.
- manyxcxi 10y ago
  The thing is that for many companies no estimates isn't possible. You have to be able to tell a client roughly how much they are going to get charged or how long it will take on order for them to even be able to with the options. Saying no estimates is akin to saying "this is hard, we suck at it so we quit." The problem isn't solved, you still have no way of measuring how complete you are and when you'll be able to turn new things on. We use a combination of story points and historical reference where we can during our estimating phase. So if we are talking about adding a new feature and wet estimate it out we go look at the actuals from a similar change that we did before to see how is base we might be. We also like story points because they are a ballpark and help us judge if we are estimating reasonably well. If we are continually having things signed low points that take data to finish, we can address that. Fast feedback and lots of iterations. The thing teams fail to do is to measure properly. If you measure too much no one fills out their stuff accurately and then you've just got garbage. Measure the wrong things and you're blind. Measure nothing (no estimates) and you're just giving yo. We don't try to measure to the minute or hour. Things are basically 'quick' 1hr or less, 2hrs, 1/2 day, 1 day, or a 1/2 week for a task we know is big and hairy that has a lot of sub tasks that we don't really want to dive into the minutiae of it because what's relevant is that X does A and B, not that X requires T, U, and V to be done first. That's the hard part for us, enough detail that its meaningful without spending all your time planning to plan.
vermooten 10y ago
“What? Two days?”, “But our previous developer could do that in a couple of hours! No way it will take you that long.” Answer: Well get the previous developer back to do it then. Otherwise STFU, and let me get started.
- bunderbunder 10y ago
  If I had $1 for every time I saw one teammate get roped into spending a week building some goofy little low value feature because another teammate - never a guy with any skin in that particular game game - said, "I could have that done in two hours" during a meeting. . .
  - dpark 10y ago
    You have bad managers if they let that crap fly. The appropriate response to that kind of thing is "then you should do it, because you clearly have the domain expertise to do the work efficiently".
- tortilla 10y ago
  During negotiations with a large corporation for customizations to our SaaS product, a business analyst asked: "Why is this so expensive? Can't you just copy and paste some of the code from other parts of the system?" I couldn't even respond right away, I had to censor myself before saying anything. Background: They wanted pre-populated values/logic very specific to their workflow. It wasn't just an easy cut and paste because the software would behave differently for them.
Klockan 10y ago
Software estimates will be optimistic as long as optimism is rewarded.
- HeyLaughingBoy 10y ago
  I would give you every upvote in the universe if only I had it...
- amai 10y ago
  I would even say: Software estimates will be optimistic as long as wrong estimates aren't punished.
dpweb 10y ago
When in doubt, add time
_Codemonkeyism 10y ago
1. People need to look into PST from Simon Wardley. 2. Some of my clients do excellent work with estimations delivering on time and on budget and satisfied customer (In the Wardley Six Sigma zone - low uncertainty) 3. Estimation does not work in the Genesis or Product zones (high uncertainty) We need to grow up beyond the "software estimation not possible".
- BatFastard 10y ago
  It depends on the type of product. If the product space is well known, then a mature organization can predict how long it should take. If the product is in a new or unknown area, or the product uses an existing unreliable/unknown code base, reliability of estimates are much lower.
  - fragsworth 10y ago
    In general, software is re-usable, and when done right, it is unnecessary to need software engineering for projects that are very similar to previous ones. The part you're working on (and trying to estimate) in almost all cases should be "new or unknown area".
    - BatFastard 10y ago
      Both parts of a project need estimates. If you are not estimating for the known parts how can you project how long the project will take? Its the unknown parts that can have a lot more variability to them.
- wpietri 10y ago
  In Wardley's model, almost no software development should take place in the "six sigma" zone. Consider figure 1 here: http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sigma.html http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sig... It says "Six Sigma/Outsource" for a reason. The product is "ordered, known, measured, standard, obvious, low margin". If at all possible, this should be an off-the-shelf choice. Previously, that would have meant packaged software; now it mostly means getting something as a library or service. Six Sigma is fine for manufacturing low-margin items, because you are trying to get high repeatability in the world of atoms. If you have one widget and need to make one million more, you use Six Sigma. But with bits, we already have near-perfect repeatability. If I have one integer and need a million more integers, the computer will do that at a reliability level well beyond six sigmas. If software is being made in the Six Sigma/Outsource zone, it mostly means that one is reinventing the wheel. Sometimes that's necessary for historical reasons. E.g., if I'm working in a new language but need a library that doesn't exist yet, sometimes I really can just take the old spec and implement it anew. Or if I want developers to use my cloud service, maybe I need to just reimplement chunks of the AWS API, because Amazon won't give me that code. But the great majority of software development should take place in the higher-novelty zones, where reliable project-scale estimation is somewhere between impossible and nonsensical. (As an aside, casting people you disagree with as children is insulting.)
FollowSteph3 10y ago
How long would it take to walk from San Francisco to Los Angeles. Even that almost impossible to estimate and not be off be several multiples, if you're lucky! http://www.michaelrwolfe.com/2013/10/19/50/ http://www.michaelrwolfe.com/2013/10/19/50/
- ktRolster 10y ago
  The guy in your link did lousy routefinding, and estimated that he could walk 40 miles a day. Anyone with experience backpacking can tell you that's a ridiculous pace (for a group, one mile an hour is a good pace). Plenty of people plan out long hikes with accurate estimations.
  - FollowSteph3 10y ago
    But remember you're doing something novel so you've never hiked before, and so you can only estimate how much you can do each day. You may have done other travels but never walked that distance. Even if you half the speed you'll still be way off. And this is just 400 miles, what about a walk that across the whole US? Can you predict that Susie on your team will fall ill for a few week. How many meetings will management need? What about feature changes? How much time do do you have to estimate? So you have months to plan out your walk? Most projects need estimates within minutes to hours. And so on...
    - ktRolster 10y ago
      But remember you're doing something novel so you've never hiked before You're not a novice programmer, you've done it before. And if you are a novice hiker, you shouldn't be doing it alone.
      - FollowSteph3 10y ago
        You've built a custom system with the exact same business requirements that connect to the exact same different sub systems, api, etc? It may be the same programming language and general framework but that's only a small part of most projects ;)
evoltix 10y ago
Another thing to note here that I have experienced is when management has an incompetent mentality that, for a given project, that all tasks for this project are created equal. For example, they expect that if task X took 1 hour that surely task Y will also take 1 hour despite the fact that task X and task Y vary greatly in complexity.
- tluyben2 10y ago
  Explaining the difference in complexity helps, but the problem is that often the programmer doesn't really know in detail besides a feeling that comes mostly from many years of experience. And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides.
  - evoltix 10y ago
    True. But it does no good if management is dismissive. But that's another problem outside of the topic of estimating. > And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides A grim picture to think about but I find this to be true a lot of the time.
- maxxxxx 10y ago
  I always have trouble explaining that it took a week to implement 90 percent of the whole project but then it took another 4 weeks to figure out why some little feature failed sometimes. That's my problem with scrum and velocity numbers. It gives clueless people the illusion that they can estimate everything easily.
Nokinside 10y ago
Simple equation that is just as good as any: p : number of people c : number of changes into specifications (= learning during implementation) f : number of frameworks, libraries, tools. x : number of new people, frameworks, libraries, tools e : initial time estimate in weeks correct estimate: E = e + e(0.1p + 0.2c + 0.3f + 1.1^x)
- evoltix 10y ago
  Do you have a source for this equation or did you derive this yourself?
- maxxxxx 10y ago
  My equation for the effort until the project is really finished: E=e*5 Almost always works.
perspectivep 10y ago
Even if you get better at estimating tasks, it's extremely hard to account for bugs that aren't discovered until later. I usually have a pretty accurate estimate for small tasks but never go back and add the time spent on fixing recessions caused by that task. Or the time spent on extra smoke testing due to missing automation.
- mentat 10y ago
  That why you use procedures that find bugs early. This has been solve (in various ways) since the 70s yet people are still writing about how it's "not possible".
  - perspectivep 10y ago
    We do. Unit tests, integration tests, manual smoke tests, code reviews of every commit, design reviews, self hosting... Sometimes your test matrix is too big. But I agree: sometimes you have to move slower in order to go faster overall and have better estimates. Estimates have to take into account testing and regression testing, which most people don't so it can make you look slow if you start doing that.
    - ktRolster 10y ago
      If you're having trouble, this book can help you: https://www.amazon.com/Zero-Bugs-Program-Faster-Thompson/dp/0996193308 https://www.amazon.com/Zero-Bugs-Program-Faster-Thompson/dp/...
      - jonex 10y ago
        You should probably mention that it's your own book you are plugging here.
        ktRolster 10y ago
        ok, next time I will. Edit window has ended.
wellpast 10y ago
> The best you can do is estimate based on historical metrics which ... is not worth very much. I've had significant success in using historical metrics to accurately estimate software projects. It takes diligence and professional experience to learn to do this, but it is possible in my experience. Unfortunately I find this philistine sentiment quite frequently in the software world: "I've had difficulty doing X. I am smart. I am senior. I am a professional. Therefore X must be impossible." Isn't it possible you haven't been stubborn/hard-working enough in your own personal growth?
- avasylev 10y ago
  Could you share more details on how you used historic estimates? You basically described me. I've never found historic estimates useful in practice, of course I haven't tried to hard, but over the years can't think even single case I'd find useful to use some old estimate.
  - wazoox 10y ago
    That's quite simple, really. If you've carefully tracked your past projects, by comparing tasks from the planned project to those you've done of similar complexity you can make more realistic estimations. It works best on a long-running project. When you plan a new feature, you can gauge really accurately at what speed your current team developed a feature of comparable complexity, so as you go forward you get better and better estimates. Joel Spolsky explained it there: http://www.joelonsoftware.com/items/2007/10/26.html http://www.joelonsoftware.com/items/2007/10/26.html
  - HeyLaughingBoy 10y ago
    I know, for example, that under a certain annoying development process coughPSPcough, I will produce code at a rate of 30 LOC/hour when measured from initial requirements gathering through handing code off to testers. I know this because every line of code I've written (using that company's process) for years has been measured and timed. So now the estimation process becomes a matter of figuring out how many LOC (as a proxy for effort) a given feature/project will be and then dividing that number by 30. It is scarily accurate, but like I said, it's been aggregated over thousands of lines of code for years.
ktRolster 10y ago
Think of estimation as an outer bound (with a confidence level). You are saying "I am 98% sure the project will be completed by time X." The more unknowns you have, the farther out you should push the deadline. Give yourself extra time: no one will complain if you finish early.
kyled 10y ago
Meh, i wouldn't say impossible. I worked with amazing people. We were able to hit our goals consistently. Key is to have a great team who knows what they are doing and are honest. Measure by complexity, not time. Unless you worked on a few products, your going to be bad at estimating larger pieces. A lot of software is very similar.
deleted 10y ago
[deleted]
cpitman 10y ago
If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI! If you cannot even give me probabilities, the lottery is a more guaranteed investment! Software estimation is required for real business to ever consider software projects. The key is to embrace uncertainty, and instead of estimating that task X will take exactly Y days, estimate a range of possibilities. I like the three point system, estimating a best case, worst case, and most likely case. It explicitly asks you to think about the long tail for when things go wrong. Then use statistical methods to combine estimates and create estimates for multiple confidence levels. This is all covered in a great book, "Software Estimation: Demystifying the Black Art" (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDMPOVQ&preview=newtab&linkCode=kpe&ref_=cm_sw_r_kb_dp_kE5KxbAPDNJB5 https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). It's not that long a read, and it goes over the problems covered by this blog post but doesn't throw it's hands up in the air and say estimation is impossible.
- je42 10y ago
  Estimation is not impossible. However, 100% confidence in your estimates is impossible.
  - Leszek 10y ago
    100% confidence in any estimate is impossible, that's what makes it an estimate.
    - je42 10y ago
      Exactly, but if you communicate it to a "standard" manager/boss it usually becomes a deadline.
- jsprogrammer 10y ago
  >If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI! Because the benefits of the software project would far outweigh the benefits of a new store? With the completion of the software project you could open an order of magnitude more stores? Completion of the software project will increase the profitability of existing stores such that they make more than a new store at existing (hypothetical, since the store is not open yet) profitability?
graycat 10y ago
As has long been accepted, can get good estimates of time and cost for software project A if it is quite similar to and done by the same team as successful projects B, C, D, E, F, .... IMHO, for software project A, break the work down into relatively small pieces and attack those. Make each piece small enough so that if that the work on that piece takes too long or even just fails, then the loss there is not too big. Some of the most important pieces are about the experience of the team -- have they successfully done just such a thing before? So, for such experience, start with the skills and, in particular, the documentation. So, start with the skills and experience with the operating system, the text editor, the e-mail program, the file system, the command lines, and the programming languages. Then move on to other related software, e.g., TCP/IP, HTML, CSS, JavaScript, SQL, AJAX, Web site security, collection classes, APIs, etc. So, for each of these, get the documentation, the skills, the experience, etc. E.g., have people write test programs that illustrate and test the tools, and some of the more important limitations on the tools, being learned. Then start in, say, outline form, the waterfall development process by coming up with a requirements document, that is, what the heck the software is to do. On writing this document, e.g., what level of detail to include, get some good consulting help. Then move on to the architecture document. This document is an example of the standard project approach of divide and conquer. So, that division results in pieces, e.g., components of some kind. So, get started on some of the components. And have write some code to test those. Do these pieces one at a time so that a failure is not to expensive. As long as the project is not struggling at some obstacle, is making good progress, each of the small steps looks okay, then continue on. Soon begin to get some data on how fast the work is going, at least for each of the stages. When are well into writing the code for version 1.0, then soon should have some okay estimates of the time to complete version 1.0. Here see part of the truth: Much of the work is getting relevant skills and experience. Next much of the work, especially the risky part, is getting a clean description of what the heck the system is to do. With those parts done well, we should be talking mostly a nice road downhill from there to a good finish line. Gee, in the middle ages in Rome there was a big project to move a huge stone obelisk some yards to one side. Big obelisk. Big project. Lots of timbers, ropes, horses, workers. It was successful. Impressive until ask how the heck the obelisk got there in the first place? Sure, ballpark 1000 years before some of Caligula's slaves went to the upper Nile, cut the obelisk from solid stone, floated it down the Nile, across the Mediterranean, and to Rome, and erected it. So, Caligula's slaves did some project planning, for a project they were likely mostly doing for the first time. And they got it done. And now we are struggling with some software project for some Web based order entry and inventory system or some such? Ah, come on, guys!
spotman 10y ago
The difference between a consultant with lots of real world experience and not. Yes there is slippage, but you learn to plan for it. In the past week I had an employee have a baby and an employee have a parent pass away and another one have a horrible knee injury and be unable to work for a couple days, meanwhile we lost no time on a project by we are up some of our buffer. If you don't plan to meet your deadlines even when faced with feature and scope creep ( which managing is another discussion entirely ) or things 100% out of your control there is other people that can and so your not going to be picked for projects that require real time mission critical demands. And yea, if you figure that out, you get to charge more, and sleep less.
bboreham 10y ago
The fundamental activity of software development is _learning_. Although it looks to an outsider like the main thing is typing stuff in (or pasting it in :-), the thing that takes all the time is learning what you should type, or learning why the code you already typed in isn't doing what you expected. If you've done that exact project before then sure, you can estimate how long it will take to do it again. But that isn't often the case - usually we're trying to do something new, or better, or on a different platform. Always learning.
alexnewman 10y ago
I've never had a big problem with it. Even with multi month estmates
doozy 10y ago
Not impossible, its akin to having a P vs NP problem. Properly estimating a software project takes just as long as developing the software itself. When people want estimates they somehow assume they can be given on the spot, or at most with a few hours of work. Nothing further from the truth. A good software estimate is a project in its own right. It takes time, a deadline and a budget to break apart a software project, design a solution, identify its subtasks, identify its critical path, research alternatives, evaluate frameworks, tentatively assign tasks to teammates, write down an estimation report, discuss it with the customer, etc. And then you end billing the customer for 120 hours of work to conclude the project can be done in five days, give or take a week.
lucindo 10y ago
I like this Quora answer on the subject: https://www.quora.com/Why-are-software-development-task-estimations-regularly-off-by-a-factor-of-2-3/answer/Michael-Wolfe https://www.quora.com/Why-are-software-development-task-esti...
- AdieuToLogic 10y ago
  That Quora answer is brilliant! Thank you for linking it here.
unoti 10y ago
This isn't a problem unique to software engineering. All kinds of things in business require estimating the unknowable. Investing in any business enterprise requires forecasting what the return on investment is and how long it'll take before you see those returns. These issues exist whether you're starting a hot dog stand, building an apartment complex, bringing a software product to market, or building an off shore oil rig. Business people have put a great deal of worthwhile thought into how to manage these risks in ways that make sense for the company. For some fresh, eye opening ideas on how to do project management for software and other businesses, check out the book The Critical Chain. The short section at the end on metrics for investment is worth the price of admission alone.
alex-yo 10y ago
Heh, I recall when a funny manager asked my team about an estimation. We agreed on a year (exactly something like 12-14 months). Then he went to his managers, sold the project with 6 months deadline, and was quite happy. The outcome was simple: his estimation was good, the team was lazy.
segmondy 10y ago
I must urge any serious developer who has read that article to immediately forget about it! Let's begin with the first phrase, "Software estimate". An estimate is an approximation. An approximation is an educated guess. So if someone said to you, "Please make an educated guess on when the software you are working on will be completed" To say it would be impossible would be the most absurd thing! To make an educated guess, You would have to break down the tasks into manageable chunks that you can clearly reason about. Once you have your chunks of task, you will have knows, unknowns, some with risks or not, some that are complex or easy, etc. The problem with estimates is the unknown. If you are constructing software for an entity, then you must not approach it as an R&D, but rather, as a possible Research THEN Development. All your unknowns should be figured out at the research phase, then you estimate the development, else you should fight to remove those unknown tasks from the project. Think about it, do you get asked for "Software research estimate" or "Software development estimate"? It's the latter, but most developers make the mistake of offering their estimate across both R&D. The really beneficial thing to developers about estimates if they do take it serious, is that it forces you to stop, think, design, and think again before coding. The biggest concerns for developers is that estimates TURN into deadlines. This is a legitimate concern, but nevertheless this doesn't invalidate your estimate, which is nothing but an approximation. The other concern is that feature/scope creep into the project really puts a wide gap between the estimate and completion date. This again is true, remember your estimate was for the original requirements. So what are you to do as a developer? You must teach your manager if they don't know or whomever you are working for the difference between estimates and deadlines. You must also learn to keep feature creeps at bay or estimate those new features anytime they are added and adjust your estimate accordingly. If you are a non manager or business. Please do think about this from their point of view. Projects have costs, which we can quickly approximate via cost of people working on it multiplied by the duration of the project. Part of running a business is managing your cash flow, knowing if you should spend or not. Without having an estimate of a duration of the project. How can businesses make decisions on which projects to fund? If you have a startup and $100,000 in cash. No income yet, 5 programmers that you pay $5,000 a month. Would you let them work on a project with no completion date, 1 yr estimate or 2 months estimate?
jacques_chester 10y ago
By definition any perfect knowledge of the future is impossible. Regardless of why. That's a trite observation. What matters is than an estimate can be made better than not estimating at all. It's funny how all these problems come up in other disciplines, but they don't throw their hands in the air and quit. They accept that the Nirvana Fallacy is not an acceptable reason to not estimate and they go ahead and do it anyway.
tomlu 10y ago
I find this a bit defeatist. Estimating software projects is definitely possible, especially for less novel projects, and high-effort estimates will have higher fidelity than low-effort estimates. Few projects are truly novel. You can't make business decisions without having some idea of estimates, so I find that arguments like this are used to weasel out of making any sort of commitment. Go ahead, add a big buffer, try to take uncertainty into account, pad, pad, pad - at least after all that we'll have some idea. On the other hand a very real problem I've often faced is when the target date is an input into the estimation process. I.e. we want to ship by May 2017, go estimate until the estimation fits this target date. Of course, after estimation the target date can be an input into requirements scoping, after which you can re-estimate.
ryanmarsh 10y ago
Software isn't construction it's design. If you start treating estimates for software as estimates for design work your life will suck less. The anxiety and stress around making and meeting software estimates is all rooted in a misperception of what software is. "Real businesses" budget for design work too, they just have different expectations for it than, say, a new office building.
slavik81 10y ago
The problems he talks about have noticeable effects on task completion time and estimate error. In my experience, estimation error is not normally distributed. For most tasks, estimation error is small, but perhaps one in ten estimates are off by an order of magnitude. It's usually a single bug or feature which blows the iteration. Though, by consuming so much time, it may cause other tasks to spill too. If you aim to estimate most tasks accurately, the overall project will probably be underestimated. Outliers consume a significant portion of the overall project time.
yason 10y ago
Estimating software schedules is easy. Just gauge a ballpark figure of what it might take if everything went nice and smooth, you had no distractions nor surprises, and then multiply by pi.
- dozzie 10y ago
  Estimating is always easy. Estimating something somewhat accurately is difficult. And no, your formula is not a solution either.

165 points by SarasaNews 10y ago

Evolution. 100,000 years ago, there were humans who could accurately estimate the difficulty of, say, catching a deer. And since they knew it was too much trouble, they just stayed home in the cave. Meanwhile some crazy optimist went out and to catch a deer. It was many times more difficult than expected, but they returned to the cave with the deer and eventually reproduced more successfully than the accurate estimator. Thus our ancestors were the crazy optimists.

pliny 10y ago
Or the estimators only hunted deer when they were well positioned to do so (e.g. few predators of humans and many deer around), and the optimists were eaten by bears.
zby 10y ago
That does not make sense. If eventually it turned out that it was worth it, then their estimate - i.e. 'it is worth it' was accurate - not too optimistic.
smallnamespace 10y ago
> And since they knew it was too much trouble More trouble than starving?
redler 10y ago
Another way to look at this is that the crazy optimists who are the worst estimators never return (themselves eaten by predators, run out of water, hypothermia) or return empty handed, lowering their reproductive success. The optimists who succeed quickly adjust their understanding of the estimating process ("Zog died when he went after deer at night during winter, and I barely survived being attacked by wolves when the deer were most plentiful, so I should go on cool midday hunts when the deer are already slightly down in population").
Rexxar 10y ago
That's the one who has domesticated the cows instead of hunting it who has win.
ryanmarsh 10y ago
Or the ones that stayed back and saved their energy to kill the weakend hunter and take the meat.

maxaf 10y ago

I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe. My criticism might seem nitpicky until it's time to measure this article's message against the possible reaction of an engineering or business manager. Launching into a systematic dismantling of the pervasive estimates culture is best done with humble, verifiable assertions and helpful suggestions in tow. Attempting to recruit the long-dead Einstein is unhelpful and hand-wavy. There must be a better way to package the no-estimates message, and it's not this.

ktRolster 10y ago
I find naive and pretentious the suggestion that the average programmer's work is in any way akin to Einstein's search for the truth about our universe. Indeed, most projects are remarkably (and sadly) similar to other projects we've done before.
- markbnj 10y ago
  It doesn't much matter how "similar" they are. Unless you can copy and paste the code you're building new stuff and running the risk of encountering all the issues inherent in building new stuff.
woodchuck64 10y ago
How about: the reliability of software estimates are inversely proportional to creativity requirements, with a unified theory of physics establishing an upper bound on creativity with a corresponding 0 for the estimate reliability.
manyxcxi 10y ago
The thing is that for many companies no estimates isn't possible. You have to be able to tell a client roughly how much they are going to get charged or how long it will take on order for them to even be able to with the options. Saying no estimates is akin to saying "this is hard, we suck at it so we quit." The problem isn't solved, you still have no way of measuring how complete you are and when you'll be able to turn new things on. We use a combination of story points and historical reference where we can during our estimating phase. So if we are talking about adding a new feature and wet estimate it out we go look at the actuals from a similar change that we did before to see how is base we might be. We also like story points because they are a ballpark and help us judge if we are estimating reasonably well. If we are continually having things signed low points that take data to finish, we can address that. Fast feedback and lots of iterations. The thing teams fail to do is to measure properly. If you measure too much no one fills out their stuff accurately and then you've just got garbage. Measure the wrong things and you're blind. Measure nothing (no estimates) and you're just giving yo. We don't try to measure to the minute or hour. Things are basically 'quick' 1hr or less, 2hrs, 1/2 day, 1 day, or a 1/2 week for a task we know is big and hairy that has a lot of sub tasks that we don't really want to dive into the minutiae of it because what's relevant is that X does A and B, not that X requires T, U, and V to be done first. That's the hard part for us, enough detail that its meaningful without spending all your time planning to plan.

vermooten 10y ago

“What? Two days?”, “But our previous developer could do that in a couple of hours! No way it will take you that long.” Answer: Well get the previous developer back to do it then. Otherwise STFU, and let me get started.

bunderbunder 10y ago
If I had $1 for every time I saw one teammate get roped into spending a week building some goofy little low value feature because another teammate - never a guy with any skin in that particular game game - said, "I could have that done in two hours" during a meeting. . .
- dpark 10y ago
  You have bad managers if they let that crap fly. The appropriate response to that kind of thing is "then you should do it, because you clearly have the domain expertise to do the work efficiently".
tortilla 10y ago
During negotiations with a large corporation for customizations to our SaaS product, a business analyst asked: "Why is this so expensive? Can't you just copy and paste some of the code from other parts of the system?" I couldn't even respond right away, I had to censor myself before saying anything. Background: They wanted pre-populated values/logic very specific to their workflow. It wasn't just an easy cut and paste because the software would behave differently for them.

Klockan 10y ago

Software estimates will be optimistic as long as optimism is rewarded.

HeyLaughingBoy 10y ago
I would give you every upvote in the universe if only I had it...
amai 10y ago
I would even say: Software estimates will be optimistic as long as wrong estimates aren't punished.

dpweb 10y ago

When in doubt, add time

_Codemonkeyism 10y ago

1. People need to look into PST from Simon Wardley. 2. Some of my clients do excellent work with estimations delivering on time and on budget and satisfied customer (In the Wardley Six Sigma zone - low uncertainty) 3. Estimation does not work in the Genesis or Product zones (high uncertainty) We need to grow up beyond the "software estimation not possible".

BatFastard 10y ago
It depends on the type of product. If the product space is well known, then a mature organization can predict how long it should take. If the product is in a new or unknown area, or the product uses an existing unreliable/unknown code base, reliability of estimates are much lower.
- fragsworth 10y ago
  In general, software is re-usable, and when done right, it is unnecessary to need software engineering for projects that are very similar to previous ones. The part you're working on (and trying to estimate) in almost all cases should be "new or unknown area".
  - BatFastard 10y ago
    Both parts of a project need estimates. If you are not estimating for the known parts how can you project how long the project will take? Its the unknown parts that can have a lot more variability to them.
wpietri 10y ago
In Wardley's model, almost no software development should take place in the "six sigma" zone. Consider figure 1 here: http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sigma.html http://blog.gardeviance.org/2015/10/agile-vs-lean-vs-six-sig... It says "Six Sigma/Outsource" for a reason. The product is "ordered, known, measured, standard, obvious, low margin". If at all possible, this should be an off-the-shelf choice. Previously, that would have meant packaged software; now it mostly means getting something as a library or service. Six Sigma is fine for manufacturing low-margin items, because you are trying to get high repeatability in the world of atoms. If you have one widget and need to make one million more, you use Six Sigma. But with bits, we already have near-perfect repeatability. If I have one integer and need a million more integers, the computer will do that at a reliability level well beyond six sigmas. If software is being made in the Six Sigma/Outsource zone, it mostly means that one is reinventing the wheel. Sometimes that's necessary for historical reasons. E.g., if I'm working in a new language but need a library that doesn't exist yet, sometimes I really can just take the old spec and implement it anew. Or if I want developers to use my cloud service, maybe I need to just reimplement chunks of the AWS API, because Amazon won't give me that code. But the great majority of software development should take place in the higher-novelty zones, where reliable project-scale estimation is somewhere between impossible and nonsensical. (As an aside, casting people you disagree with as children is insulting.)

FollowSteph3 10y ago

How long would it take to walk from San Francisco to Los Angeles. Even that almost impossible to estimate and not be off be several multiples, if you're lucky! http://www.michaelrwolfe.com/2013/10/19/50/ http://www.michaelrwolfe.com/2013/10/19/50/

ktRolster 10y ago
The guy in your link did lousy routefinding, and estimated that he could walk 40 miles a day. Anyone with experience backpacking can tell you that's a ridiculous pace (for a group, one mile an hour is a good pace). Plenty of people plan out long hikes with accurate estimations.
- FollowSteph3 10y ago
  But remember you're doing something novel so you've never hiked before, and so you can only estimate how much you can do each day. You may have done other travels but never walked that distance. Even if you half the speed you'll still be way off. And this is just 400 miles, what about a walk that across the whole US? Can you predict that Susie on your team will fall ill for a few week. How many meetings will management need? What about feature changes? How much time do do you have to estimate? So you have months to plan out your walk? Most projects need estimates within minutes to hours. And so on...
  - ktRolster 10y ago
    But remember you're doing something novel so you've never hiked before You're not a novice programmer, you've done it before. And if you are a novice hiker, you shouldn't be doing it alone.
    - FollowSteph3 10y ago
      You've built a custom system with the exact same business requirements that connect to the exact same different sub systems, api, etc? It may be the same programming language and general framework but that's only a small part of most projects ;)

evoltix 10y ago

Another thing to note here that I have experienced is when management has an incompetent mentality that, for a given project, that all tasks for this project are created equal. For example, they expect that if task X took 1 hour that surely task Y will also take 1 hour despite the fact that task X and task Y vary greatly in complexity.

tluyben2 10y ago
Explaining the difference in complexity helps, but the problem is that often the programmer doesn't really know in detail besides a feeling that comes mostly from many years of experience. And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides.
- evoltix 10y ago
  True. But it does no good if management is dismissive. But that's another problem outside of the topic of estimating. > And the best thing is if they then get 'their 10x code slinger' who does it indeed very fast, sends the code back to you and you are stuck with something akin to a broken toilet with diarrhea coming out the sides A grim picture to think about but I find this to be true a lot of the time.
maxxxxx 10y ago
I always have trouble explaining that it took a week to implement 90 percent of the whole project but then it took another 4 weeks to figure out why some little feature failed sometimes. That's my problem with scrum and velocity numbers. It gives clueless people the illusion that they can estimate everything easily.

Nokinside 10y ago

Simple equation that is just as good as any: p : number of people c : number of changes into specifications (= learning during implementation) f : number of frameworks, libraries, tools. x : number of new people, frameworks, libraries, tools e : initial time estimate in weeks correct estimate: E = e + e(0.1p + 0.2c + 0.3f + 1.1^x)

evoltix 10y ago
Do you have a source for this equation or did you derive this yourself?
maxxxxx 10y ago
My equation for the effort until the project is really finished: E=e*5 Almost always works.

perspectivep 10y ago

Even if you get better at estimating tasks, it's extremely hard to account for bugs that aren't discovered until later. I usually have a pretty accurate estimate for small tasks but never go back and add the time spent on fixing recessions caused by that task. Or the time spent on extra smoke testing due to missing automation.

mentat 10y ago
That why you use procedures that find bugs early. This has been solve (in various ways) since the 70s yet people are still writing about how it's "not possible".
- perspectivep 10y ago
  We do. Unit tests, integration tests, manual smoke tests, code reviews of every commit, design reviews, self hosting... Sometimes your test matrix is too big. But I agree: sometimes you have to move slower in order to go faster overall and have better estimates. Estimates have to take into account testing and regression testing, which most people don't so it can make you look slow if you start doing that.
  - ktRolster 10y ago
    If you're having trouble, this book can help you: https://www.amazon.com/Zero-Bugs-Program-Faster-Thompson/dp/0996193308 https://www.amazon.com/Zero-Bugs-Program-Faster-Thompson/dp/...
    - jonex 10y ago
      You should probably mention that it's your own book you are plugging here.
      - ktRolster 10y ago
        ok, next time I will. Edit window has ended.

wellpast 10y ago

> The best you can do is estimate based on historical metrics which ... is not worth very much. I've had significant success in using historical metrics to accurately estimate software projects. It takes diligence and professional experience to learn to do this, but it is possible in my experience. Unfortunately I find this philistine sentiment quite frequently in the software world: "I've had difficulty doing X. I am smart. I am senior. I am a professional. Therefore X must be impossible." Isn't it possible you haven't been stubborn/hard-working enough in your own personal growth?

avasylev 10y ago
Could you share more details on how you used historic estimates? You basically described me. I've never found historic estimates useful in practice, of course I haven't tried to hard, but over the years can't think even single case I'd find useful to use some old estimate.
- wazoox 10y ago
  That's quite simple, really. If you've carefully tracked your past projects, by comparing tasks from the planned project to those you've done of similar complexity you can make more realistic estimations. It works best on a long-running project. When you plan a new feature, you can gauge really accurately at what speed your current team developed a feature of comparable complexity, so as you go forward you get better and better estimates. Joel Spolsky explained it there: http://www.joelonsoftware.com/items/2007/10/26.html http://www.joelonsoftware.com/items/2007/10/26.html
- HeyLaughingBoy 10y ago
  I know, for example, that under a certain annoying development process coughPSPcough, I will produce code at a rate of 30 LOC/hour when measured from initial requirements gathering through handing code off to testers. I know this because every line of code I've written (using that company's process) for years has been measured and timed. So now the estimation process becomes a matter of figuring out how many LOC (as a proxy for effort) a given feature/project will be and then dividing that number by 30. It is scarily accurate, but like I said, it's been aggregated over thousands of lines of code for years.

ktRolster 10y ago

Think of estimation as an outer bound (with a confidence level). You are saying "I am 98% sure the project will be completed by time X." The more unknowns you have, the farther out you should push the deadline. Give yourself extra time: no one will complain if you finish early.

kyled 10y ago

Meh, i wouldn't say impossible. I worked with amazing people. We were able to hit our goals consistently. Key is to have a great team who knows what they are doing and are honest. Measure by complexity, not time. Unless you worked on a few products, your going to be bad at estimating larger pieces. A lot of software is very similar.

deleted 10y ago

[deleted]

cpitman 10y ago

If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI! If you cannot even give me probabilities, the lottery is a more guaranteed investment! Software estimation is required for real business to ever consider software projects. The key is to embrace uncertainty, and instead of estimating that task X will take exactly Y days, estimate a range of possibilities. I like the three point system, estimating a best case, worst case, and most likely case. It explicitly asks you to think about the long tail for when things go wrong. Then use statistical methods to combine estimates and create estimates for multiple confidence levels. This is all covered in a great book, "Software Estimation: Demystifying the Black Art" (https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDMPOVQ&preview=newtab&linkCode=kpe&ref_=cm_sw_r_kb_dp_kE5KxbAPDNJB5 https://read.amazon.com/kp/embed?asin=B00JDMPOVQ&asin=B00JDM...). It's not that long a read, and it goes over the problems covered by this blog post but doesn't throw it's hands up in the air and say estimation is impossible.

je42 10y ago
Estimation is not impossible. However, 100% confidence in your estimates is impossible.
- Leszek 10y ago
  100% confidence in any estimate is impossible, that's what makes it an estimate.
  - je42 10y ago
    Exactly, but if you communicate it to a "standard" manager/boss it usually becomes a deadline.
jsprogrammer 10y ago
>If I am running a business that is looking at what projects to fund for the next year, and one of those projects is a software project with an unknowable time to completion vs (for example) opening a new store, with a well known time to complete, why would I ever pick the software project? It has a completely unknown ROI! Because the benefits of the software project would far outweigh the benefits of a new store? With the completion of the software project you could open an order of magnitude more stores? Completion of the software project will increase the profitability of existing stores such that they make more than a new store at existing (hypothetical, since the store is not open yet) profitability?

graycat 10y ago

As has long been accepted, can get good estimates of time and cost for software project A if it is quite similar to and done by the same team as successful projects B, C, D, E, F, .... IMHO, for software project A, break the work down into relatively small pieces and attack those. Make each piece small enough so that if that the work on that piece takes too long or even just fails, then the loss there is not too big. Some of the most important pieces are about the experience of the team -- have they successfully done just such a thing before? So, for such experience, start with the skills and, in particular, the documentation. So, start with the skills and experience with the operating system, the text editor, the e-mail program, the file system, the command lines, and the programming languages. Then move on to other related software, e.g., TCP/IP, HTML, CSS, JavaScript, SQL, AJAX, Web site security, collection classes, APIs, etc. So, for each of these, get the documentation, the skills, the experience, etc. E.g., have people write test programs that illustrate and test the tools, and some of the more important limitations on the tools, being learned. Then start in, say, outline form, the waterfall development process by coming up with a requirements document, that is, what the heck the software is to do. On writing this document, e.g., what level of detail to include, get some good consulting help. Then move on to the architecture document. This document is an example of the standard project approach of divide and conquer. So, that division results in pieces, e.g., components of some kind. So, get started on some of the components. And have write some code to test those. Do these pieces one at a time so that a failure is not to expensive. As long as the project is not struggling at some obstacle, is making good progress, each of the small steps looks okay, then continue on. Soon begin to get some data on how fast the work is going, at least for each of the stages. When are well into writing the code for version 1.0, then soon should have some okay estimates of the time to complete version 1.0. Here see part of the truth: Much of the work is getting relevant skills and experience. Next much of the work, especially the risky part, is getting a clean description of what the heck the system is to do. With those parts done well, we should be talking mostly a nice road downhill from there to a good finish line. Gee, in the middle ages in Rome there was a big project to move a huge stone obelisk some yards to one side. Big obelisk. Big project. Lots of timbers, ropes, horses, workers. It was successful. Impressive until ask how the heck the obelisk got there in the first place? Sure, ballpark 1000 years before some of Caligula's slaves went to the upper Nile, cut the obelisk from solid stone, floated it down the Nile, across the Mediterranean, and to Rome, and erected it. So, Caligula's slaves did some project planning, for a project they were likely mostly doing for the first time. And they got it done. And now we are struggling with some software project for some Web based order entry and inventory system or some such? Ah, come on, guys!

spotman 10y ago

The difference between a consultant with lots of real world experience and not. Yes there is slippage, but you learn to plan for it. In the past week I had an employee have a baby and an employee have a parent pass away and another one have a horrible knee injury and be unable to work for a couple days, meanwhile we lost no time on a project by we are up some of our buffer. If you don't plan to meet your deadlines even when faced with feature and scope creep ( which managing is another discussion entirely ) or things 100% out of your control there is other people that can and so your not going to be picked for projects that require real time mission critical demands. And yea, if you figure that out, you get to charge more, and sleep less.

bboreham 10y ago

The fundamental activity of software development is _learning_. Although it looks to an outsider like the main thing is typing stuff in (or pasting it in :-), the thing that takes all the time is learning what you should type, or learning why the code you already typed in isn't doing what you expected. If you've done that exact project before then sure, you can estimate how long it will take to do it again. But that isn't often the case - usually we're trying to do something new, or better, or on a different platform. Always learning.

alexnewman 10y ago

I've never had a big problem with it. Even with multi month estmates

doozy 10y ago

Not impossible, its akin to having a P vs NP problem. Properly estimating a software project takes just as long as developing the software itself. When people want estimates they somehow assume they can be given on the spot, or at most with a few hours of work. Nothing further from the truth. A good software estimate is a project in its own right. It takes time, a deadline and a budget to break apart a software project, design a solution, identify its subtasks, identify its critical path, research alternatives, evaluate frameworks, tentatively assign tasks to teammates, write down an estimation report, discuss it with the customer, etc. And then you end billing the customer for 120 hours of work to conclude the project can be done in five days, give or take a week.

lucindo 10y ago

I like this Quora answer on the subject: https://www.quora.com/Why-are-software-development-task-estimations-regularly-off-by-a-factor-of-2-3/answer/Michael-Wolfe https://www.quora.com/Why-are-software-development-task-esti...

AdieuToLogic 10y ago
That Quora answer is brilliant! Thank you for linking it here.

unoti 10y ago

This isn't a problem unique to software engineering. All kinds of things in business require estimating the unknowable. Investing in any business enterprise requires forecasting what the return on investment is and how long it'll take before you see those returns. These issues exist whether you're starting a hot dog stand, building an apartment complex, bringing a software product to market, or building an off shore oil rig. Business people have put a great deal of worthwhile thought into how to manage these risks in ways that make sense for the company. For some fresh, eye opening ideas on how to do project management for software and other businesses, check out the book The Critical Chain. The short section at the end on metrics for investment is worth the price of admission alone.

alex-yo 10y ago

Heh, I recall when a funny manager asked my team about an estimation. We agreed on a year (exactly something like 12-14 months). Then he went to his managers, sold the project with 6 months deadline, and was quite happy. The outcome was simple: his estimation was good, the team was lazy.

segmondy 10y ago

I must urge any serious developer who has read that article to immediately forget about it! Let's begin with the first phrase, "Software estimate". An estimate is an approximation. An approximation is an educated guess. So if someone said to you, "Please make an educated guess on when the software you are working on will be completed" To say it would be impossible would be the most absurd thing! To make an educated guess, You would have to break down the tasks into manageable chunks that you can clearly reason about. Once you have your chunks of task, you will have knows, unknowns, some with risks or not, some that are complex or easy, etc. The problem with estimates is the unknown. If you are constructing software for an entity, then you must not approach it as an R&D, but rather, as a possible Research THEN Development. All your unknowns should be figured out at the research phase, then you estimate the development, else you should fight to remove those unknown tasks from the project. Think about it, do you get asked for "Software research estimate" or "Software development estimate"? It's the latter, but most developers make the mistake of offering their estimate across both R&D. The really beneficial thing to developers about estimates if they do take it serious, is that it forces you to stop, think, design, and think again before coding. The biggest concerns for developers is that estimates TURN into deadlines. This is a legitimate concern, but nevertheless this doesn't invalidate your estimate, which is nothing but an approximation. The other concern is that feature/scope creep into the project really puts a wide gap between the estimate and completion date. This again is true, remember your estimate was for the original requirements. So what are you to do as a developer? You must teach your manager if they don't know or whomever you are working for the difference between estimates and deadlines. You must also learn to keep feature creeps at bay or estimate those new features anytime they are added and adjust your estimate accordingly. If you are a non manager or business. Please do think about this from their point of view. Projects have costs, which we can quickly approximate via cost of people working on it multiplied by the duration of the project. Part of running a business is managing your cash flow, knowing if you should spend or not. Without having an estimate of a duration of the project. How can businesses make decisions on which projects to fund? If you have a startup and $100,000 in cash. No income yet, 5 programmers that you pay $5,000 a month. Would you let them work on a project with no completion date, 1 yr estimate or 2 months estimate?

jacques_chester 10y ago

By definition any perfect knowledge of the future is impossible. Regardless of why. That's a trite observation. What matters is than an estimate can be made better than not estimating at all. It's funny how all these problems come up in other disciplines, but they don't throw their hands in the air and quit. They accept that the Nirvana Fallacy is not an acceptable reason to not estimate and they go ahead and do it anyway.

tomlu 10y ago

I find this a bit defeatist. Estimating software projects is definitely possible, especially for less novel projects, and high-effort estimates will have higher fidelity than low-effort estimates. Few projects are truly novel. You can't make business decisions without having some idea of estimates, so I find that arguments like this are used to weasel out of making any sort of commitment. Go ahead, add a big buffer, try to take uncertainty into account, pad, pad, pad - at least after all that we'll have some idea. On the other hand a very real problem I've often faced is when the target date is an input into the estimation process. I.e. we want to ship by May 2017, go estimate until the estimation fits this target date. Of course, after estimation the target date can be an input into requirements scoping, after which you can re-estimate.

ryanmarsh 10y ago

Software isn't construction it's design. If you start treating estimates for software as estimates for design work your life will suck less. The anxiety and stress around making and meeting software estimates is all rooted in a misperception of what software is. "Real businesses" budget for design work too, they just have different expectations for it than, say, a new office building.

slavik81 10y ago

The problems he talks about have noticeable effects on task completion time and estimate error. In my experience, estimation error is not normally distributed. For most tasks, estimation error is small, but perhaps one in ten estimates are off by an order of magnitude. It's usually a single bug or feature which blows the iteration. Though, by consuming so much time, it may cause other tasks to spill too. If you aim to estimate most tasks accurately, the overall project will probably be underestimated. Outliers consume a significant portion of the overall project time.

yason 10y ago

Estimating software schedules is easy. Just gauge a ballpark figure of what it might take if everything went nice and smooth, you had no distractions nor surprises, and then multiply by pi.

dozzie 10y ago
Estimating is always easy. Estimating something somewhat accurately is difficult. And no, your formula is not a solution either.