15 ms·

Rules of thumb for software development estimations

180 points by bndr 3y ago

ochronus 3y ago
One thing I found crucial when dealing with estimation is to always, always explicitly add the level of confidence (in the estimate). This saves so many tough discussions later... either have it built in to your product development process (e.g. cone of uncertainty, or different stages with different predefined levels of confidence), or always communicate it right next to your estimates. Some stakeholders will push back initially, but you can reason and tell them that you need to invest more time into research, planning, PoCs etc. if they want a more accurate estimation. Eventually, it boils down to "are we agile yet", trust in the organization, maturity and culture. One of the questions I always ask when interviewing for a role is how this is done in the company, with actual examples when projects took longer than planned (and what happened). Tells a lot about the maturity of the org.
- jon-wood 3y ago
  I get really frustrated that despite the advice to give confidence levels being all over the place I’m aware of no project planning software which incorporates that advice. It all assumes a perfect world where you can say it will take 6.5 days to complete a task, and then cascades from there. I want project planning software which accepts lower and upper bounds, or an estimate + %age confidence, and then at the end gives me a project plan with error bars on each part of the timeline. Let me give stakeholders a diagram showing exactly what we mean by it being hard to say how long this will take, rather than what appears to be a rock solid plan timed to the day. Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several).
  - ochronus 3y ago
    True that. It's inconvenient, but you can use custom fields (e.g. in JIRA) for this, but I agree, it's not a proper solution as it doesn't display e.g. ranges :/
    - alkonaut 3y ago
      Now I'm thinking I should build a better Jira where the basic unit of everything is a range instead of a number. I wonder how long that would take me.
      - kayodelycaon 3y ago
        If you’re looking to capture all of JIRA’s features, it would take you a very long time. Duplicating their extensive automation capabilities is a massive project. If you ever get around to that, let me know. JIRA really is best in class and I want to shove it into a very tiny, very lonely box before tossing it into the deepest part of the ocean I can find.
      - kqr 3y ago
        The area of statistically sane software for regular users is so underexploited that there are many instances of X you can plug into "X but with ranges instead of numbers" and carve out a niche for yourself. I myself am hoping someone will pick up X=spreadsheets at some point soon.
        jon-wood 3y ago
        You’re looking for causal.app for that, and it’s as good as it sounds.
  - Ensorceled 3y ago
    > Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several). My favourite is the ones who not only will push back on confidence levels but will then drill down into any estimates they think are "conservative".
  - marcosdumay 3y ago
    If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. If you decide they are described by fat-tailed distributions¹, then the exact distribution and its parameters are much more important than your estimated intervals, so again collecting them adds nothing. So, yeah, collecting confidence intervals never adds anything. 1 - Like they should, because nobody does a project on anything they know well enough to describe with classical statistics.
    - kqr 3y ago
      > If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)
      - kelseyfrog 3y ago
        I believe what the parent is referring to is that the sum of two normally distributed variables can be modeled as normally distributed[1]. Ie: normal distributions are closed under addition. This fact leads to one's naive idea about adding variances working as expected. However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time. Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode. As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated. 1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra... 2. See the Fenton-Wilkinson approximation.
  - thebiss 3y ago
    Microsoft Project does support PERT estimates, though it's difficult to find and use: https://support.microsoft.com/en-us/office/expected-duration-task-field-d2d5d25f-0fe9-4ab7-8724-7a5172a02e2d https://support.microsoft.com/en-us/office/expected-duration...
    - kqr 3y ago
      Doesn't PERT prescribe simply adding together the optimistic and pessimistic cases with each other? Percentiles are not additive.
  - nicolas_t 3y ago
    Fogcreek's software fogbugz had that... It's a pity that it's mostly been abandoned.
- Too 3y ago
  Estimating in highly inaccurate resolution, eg fibonacci or powers of 2, is a way to embed the lack of confidence in single number estimates. You may think something takes 9 days, but if you are only allowed to answer either 8 or 16, choosing 16 is the only safe option and automatically includes some lack of confidence.
  - digging 3y ago
    While that's true, it doesn't work if everyone doesn't have the exact same understanding of how it works. For example, most people I've worked with will choose 8 instead of 16 for an estimate of 9. This makes it risky to overestimate and appear less capable.
  - ochronus 3y ago
    It's a suitable approach in the local context (tickets/tasks for the team), but won't work on the project level and with stakeholders. Stakeholders eventually will want a timeframe.
ohthehugemanate 3y ago
There is lots of good advice here, but I'm always amazed that these articles miss the #1 improvement to estimates recommended and validated by scientific studies: do not estimate in units of time. Everything else here applies, but don't estimate in time. Estimate in difficulty points, t shirt sizes, cups of coffee, gold stars, anything will do. Then as you track your outcomes (see TFA), measure the average relationship between your units and time. Then use that relationship to project a timeline based on your gold stars or whatever. Using the law of large numbers to average externalities and developer inaccuracy into your time estimate has been the most accurate method since Kahneman and Tversky's Nobel prize winning studies in the 1970s.
- techdragon 3y ago
  I’m always trying to gather extra ammunition for when I’m arguing for points based estimation methodology… do you (or anyone else) have any favourite scientific studies on this ?
- switch007 3y ago
  My previous manager and PM would scrutinise the time taken in Jira, then create a personal spreadsheet to roughly map points to hours/days (because senior stakeholders don't think in cups of coffee or t-shirt sizes), then refer to the spreadsheet privately during estimation. But he eventually let slip about the existence of it So we just bypassed the charade and starting giving hours/days. The whole team was really just doing hours/days -> points mapping in their heads during estimation anyway.
  - ohthehugemanate 3y ago
    > The whole team was really just doing hours/days -> points mapping in their heads during estimation anyway. This is a common story. But because of the calculation step, your PM was still mapping estimated time units to actual time consumed, and using that mapping for predictions. So you were getting some of the benefits of an abstracted unit (accuracy, averaging in unexpected events) without the stress reduction or focus on quality over time. But if it worked for your team, I would advocate to keep it. Human groups are hard enough to coordinate as it is, there's no need to hew to a dogma.
ssss11 3y ago
I liked this article. Practical, not full of bs, not full of bias, some good tips. Might check out his other articles now.
okaleniuk 3y ago
I stepped in as a project manager for my R&D group when the war started. I wasn't going to but we had nobody else at the moment so here I am. One of the perks of this arrangement is that I basically can do whatever I want. Worse case scenario - I'm back to my engineering job I wasn't going to leave to begin with. So the first thing I did when I stepped in is I made estimations paid. As a business partner, you can request an estimation for a feature, but an estimation itself then becomes a project. Alternatively, you can just tell me your deadline and we'll manage things on our side to fit your date. Do you know how many times did we do the "estimation as a project"? Once. And their project got cancelled anyway so we didn't even have to do the actual work and nobody knows if our estimate was accurate. I've heard many times that business needs estimations. As it turns out, it doesn't. In times of inherent unpredictability, the real priorities show. All that business needs is to be better than competitors. It needs to create marketable value. It needs ideas, it needs analysis, it needs expertise and hard work. Estimations don't help create value. They only create more work. As soon as you make this work visible, poof - nobody wants estimations anymore!
- regularfry 3y ago
  Yep. I've been through this cycle too. The phrase that helped me sell the idea was "estimation is design". You can't come to an estimate that has any relationship to reality without knowing what you're intending to build, and that's a design function, even if it's very high-level, coarse, and approximate. Just expecting that design to materialise from thin air is extremely unrealistic, so you have to go through some sort of intentional process to get there. Making that process cause an appropriate amount of pain does make it clear that none of it is free.
- AnimalMuppet 3y ago
  I love this, but you can't always use it. Sometimes management needs an estimate (even if a coarse one) to decide whether the project is worth doing or not. Sometimes you're doing fixed-price bids for a customer, and you'd better know how much to ask for in the bid. And so on. But your main point is absolutely right. Estimates aren't free. They're a project, even if a small one.
- marcosdumay 3y ago
Tade0 3y ago
I noticed one important point mentioned in the banking app example, but not elaborated on: Shirnk critical scope. By this I mean if you can produce a simpler version of whatever is that you're doing and have separate tasks for expanding it, do it. Your e-commerce solution can start off with just one payment provider. Your form validation can initially respond with just "nope!". You don't need that color picker shaped like a peacock in your MVP. Especially if these things prevent you from delivering something that someone, anyone can use. Also half of such things become irrelevant and ultimately get dropped.
- spongeb00b 3y ago
  I repeatedly see features being placed into a "phase 2" that appears approx 90% into a project as the deadline looms. Half of those features eventually get implemented, the rest it’s realised are not needed at all.
  - alkonaut 3y ago
    That's the correct way of going about any software development. The problem though is the customer will likely be unwilling to agree to a contract that doesn't have at least 75% of the scope or the product is "unusable". Their existing/previous product is usually better all the way until every single feature of it is available in the new system (I have not yet in my career ever seen a system that isn't replacing a previous system - at least if you count an excel sheet or a piece of paper as "previous system"). What you end up with when shipping an MVP is you also convert a minimum number of users and you end up with two systems. The "MVP" is thus often "whatever allows all existing users to switch" which is why projects snowball to be too big and too late. You won't realize what people used the old system for, until you try to replace it. And it always turns out it has 2000 features when your discovery had found 20.
sanitycheck 3y ago
The elephant in the room here is, of course, that often accurate estimates are unwanted. If you tell the truth about how long something will take, maybe your customer will go with someone else. Maybe the CTO will decide to outsource instead of giving the work to your team. You know those other guys are probably as good at estimation as you are, they're just deliberately going low so you have to too. (And thus, everybody now expects IT projects to overrun!) Other than that I agree with most of the linked article, I get very good results by breaking tasks down into smaller (<1 week) jobs and getting a best and worse case estimate for each. Sum both columns, multiply both by some number for unknown unknowns (usually 1.5, 2 when there are bad vibes) and you have a realistic minimum time and a worst case. If the max is >2X the min (it will be to begin with!), get more information to narrow the range, repeat until it's not outrageous.
- cratermoon 3y ago
  > accurate estimates are unwanted In my career I've learned that estimating usually boils down to a game involving guessing what the customer will accept, while the customer tries to figure out what they want. Dig deep enough, and there's usually a dependency like "it needs to be done before X", but when X will be done is also imperfectly known, so it's a tangle of dependencies. The cases of hard dates usually involve businesses dependent on seasonal, holiday revenue or scheduled events like sports or entertainment touring schedules. Estimating for those is much easier, because the game becomes one of figuring what can be delivered by the date(s) in question. Regulatory changes or fiscal timelines are usually not as fixed as they are believed, because there's lots of ways to paper a path over delays, although its costs money and business don't like that.
- dasil003 3y ago
  “Telling the truth” implies two things: that the actual effort needed is fixed, and that you know it. In practice I’ve found neither of those things to be anywhere close to true for large projects. Furthermore the pursuit of accuracy by doing finer and finer-grained estimates and then rolling them up can waste a lot of time because it provides fodder for bike shedding while missing the forest for the trees. I can’t count the number of times I’ve seen this process lead to a total train wreck. What works better is to be very clear on the big picture goals, do a speculative high level breakdown to inform an initial deadline and staffing commitment, but keep the precise scope flexible. Then get right into prototyping and building, attacking the areas of largest risk and refining the requirements as you go. You only should have fine-grained plans for the next 2-3 months with the rest of the roadmap intentionally being flexible, save for any key milestones that are needed to ensure progress to the high level goal. Of course this requires deep expertise with cross-functional influence and bi-directional trust with management, but if you don’t have those things big projects are fucked regardless. In that case your best bet is hunkering down with some agile methodology as a shield while looking for a better job.
kitanata 3y ago
Wow. A solid quality article. Great info. Well done!
sshb 3y ago
Great article on this topic: https://apenwarr.ca/log/20171213 https://apenwarr.ca/log/20171213
- thor_molecules 3y ago
  yeah this is really good - thank u
- flurdy 3y ago
  Very good. I would also balance the OP article with Allen Holub's talks https://www.youtube.com/watch?v=QVBlnCTu9Ms https://www.youtube.com/watch?v=QVBlnCTu9Ms
vp8989 3y ago
Anecdotally, I've observed across my ~12 year career so far that an emphasis on estimates and estimating is negatively correlated to productivity, lead time, velocity, impact, positive outcomes etc... I suspect the reason is because management is trying to use numbers to justify bin packing more work to an already oversubscribed team. What never shows up in those project management spreadsheets is the very real and predictable cost of context switching and the increase in mistakes from dealing with a larger amount of in-flight work.
- rqtwteye 3y ago
  Totally agree. My management mostly is interested in estimates and deadlines . They never engage in discussions about productivity or quality. This results in people continuing ineffective processes and other systems. It also encourages people to make tests pass at any cost or close tickets even when there are deeper problems that should be resolved. In short, the focus on estimates
- candiddevmike 3y ago
  Estimating and a lot of project management stuff is a form of procrastination for a lot of folks/businesses. It doesn't matter what fields the cards have or how they are arranged, the work still needs to get done, and all of this shuffling is just delaying the start of that work.
  - delusional 3y ago
    That's an interesting thought. I've never considered it before, but estimation as procrastination might be a good explanation.
  - AnimalMuppet 3y ago
    That's only true if there's a fixed set of work to get done. But that's rarely the case. Often, management has N different things they could have done, and enough people and time to do M of them, for M < N. Which ones should they do? Well, whatever maximizes profits. So they (management) estimate income from each thing that could be done, and ask engineering (hopefully) to estimate how much it will cost to implement (or how long it will take, which equates to cost). Then they make a (hopefully) more informed decision than they otherwise could have made. Look, there's lots of ways this gets done badly. I get that. But the idea itself is not nonsense.
jcon321 3y ago
All I know is government contracts, but I've been pretty successful at estimations. Much like the top posts in this thread, none of it is for free. Most of the time requirements come in a few sentences, maybe a paragraph, rare cases a document. There's no estimation until the requirements are flushed out in some type of use case which has some bearing on the technical constraints of the system. (The client pays for this, and provides feedback.) The next step is planning, which is where estimation comes from. (The client pays for this.) In terms of planning I do strive for maximum 16 hour chunks which helps accuracy. For estimations that I have low confidence in, I'll have conditional chunks that may be used. All of this is transparent to the client. If the client asks for an estimation before the above steps it's known by everyone that it has no official meaning, because everyone is aware of the above process.
jt2190 3y ago
The smaller the potential upside of the project is, the more the estimates matter. Put another way: A project that promises to return USD 10 for every USD 1 invested is still an attractive investment when the ratio is 8:1 or 7:1. This suggests that construction costs can double or triple without destroying and the investment still has a healthy return, and so an estimate really only needs to be very rough. (In fact, the act of estimating quickly adds to “construction” costs so should be kept to a bare minimum in these cases, to keep profits as high as possible.) It’s when the cost/profit ratio slips lower that estimates become more important, and ironically, waste more time and money. (Note that I’m talking specifically about estimation and not project planning… projects, especially large ones, still need plans.)
- commandlinefan 3y ago
  > A project that promises to return USD 10 for every USD 1 invested is still an attractive investment when the ratio is 8:1 or 7:1 What in the world experience have you or anybody else ever had where a software project can be estimated with that level of accuracy?
vrglvrglvrgl 3y ago
[dead]
throwawaaarrgh 3y ago
1. Have you done it before? If no, triple your estimate. 2. Do you know everything involved? If no, double your estimate. 3. Do you have someone that you will ask questions of or pair with to get through difficult sections quickly? If no, add half to the estimate. 4. Does your team like to nit pick pull requests? Add half to the estimate. 5. Factor average weekly toil into the estimate. If you don't measure this, triple the estimate. This may seem like a really big estimate. But measure the estimate against the outcome and tell me I'm wrong.
- falcolas 3y ago
  All this while remembering that you will probably only get 4 hours a day to dedicate to the project. Between responding to emails and chats, paperwork, training, meetings, and context switching - your day will be half gone before you can even touch a project.
- jgeada 3y ago
  Estimating a project that doesn't have detailed scope and specifications is a bit like asking "how long is a piece of string". Asking for an estimate before knowing what it is you are building is canonical MBA widget production management. Numbers and dates unrelated to reality so the idiots with the power can put data into their spreadsheets and claim they're doing something. Software has nearly 0 reproduction costs, so by definition you'll (mostly) only write software for something that has not been done before. How long will it take to quantify the unknown?
  - chasd00 3y ago
    what i do is break it into two projects. The first is a "discovery" project where you learn what you need to know to give an accurate estimate. You can usually get a discovery estimate in the ballpark by understanding how complex the ask is and how many people you're going to have to interview. Then you deliver the actual estimate along with all your evidence that will drive the scope definition.
  - throwawaaarrgh 3y ago
    Sorry, I guess I wasn't clear; my suggestions are for when you already have scope and specifications. When you think you know what to do and how to do it, you can then add all the extra time to come to the "actual estimate".
zeroonetwothree 3y ago
All you need to know about estimates is Hofstadter’s law.
AlbertCory 3y ago
I wrote about this: https://albertcory50.substack.com/p/how-to-finish-a-software-project https://albertcory50.substack.com/p/how-to-finish-a-software... The key thing to note is: It's not just big software projects; it's ALL big projects The California Bullet Train is a poster child for this, as is the Big Dig in Boston, and (probably) your kitchen remodeling. Once you accept this, you can learn from the way other disciplines handle it. Or fail to.
perk 3y ago
We created a small webapp that works pretty good for us: https://calestimate.com https://calestimate.com (No login needed to try it out)
brunoocasali 3y ago
Thank you for sharing your knowledge. Talking about estimations is always complex.
exabrial 3y ago
This is my rule: Start with the developer's estimate. Then for every management layer in the organization, multiple by 2.25 until you hit the c-suite. This turns out to be surprisingly accurate when measured against reality. I'm guessing someone has already thought of it and I don't really have a name for it.
- disgruntledphd2 3y ago
  This was my estimation process for construction add-ons (back when I did this between college courses). Take an estimate, double it and add 50%. I always regret when I don't take this approach.
- rqtwteye 3y ago
  My factor is 5. Works like a charm. It’s just not politically correct to say so.
- tryauuum 3y ago
  This is nice, although there's a self-fulfilling prophecy element in this. You will postpone your project to fill up the declared time
chasd00 3y ago
I work in consulting where if your estimates are not right then you don't eat. One thing many people forget is that writing code is only a small part of the overall effort in software development. I feel like i could write a book on the subject but I think the most common error is not understanding the scope of the overall effort from an idea to effective use by end users and only estimating the typing-on-the-keyboard part.
KronisLV 3y ago
> The "one size fits all" approach: Assuming that every task will take the same amount of time is a recipe for disaster. Different tasks have different complexities and require different skill sets. I sometimes wish we lived in a world where we'd have enough data to accurately predict at least the common tasks, thus freeing people up to reason about the more complex tasks. For example: adding a new path to a webapp, which will resolve to a Ruby on Rails template, that needs 2 dropdowns and 3 input fields, which need input validations and checks against already saved DB data on saving will take around X hours of work. Doing this with Python and Django will take around Y hours of work. Using a certain set of tools to assist with development will affect these estimates by Z hours. But I doubt anyone ever has data that granular.
- kqr 3y ago
  I think once we get to the point where the design is so fixed for a set of standardised tasks like that, that we know how long it will take, the design is also so fixed that we can run a script that generates the code for it, so the time to make it is close to zero.
  - KronisLV 3y ago
    > I think once we get to the point where the design is so fixed for a set of standardised tasks like that, that we know how long it will take, the design is also so fixed that we can run a script that generates the code for it, so the time to make it is close to zero. For all of the boilerplate stuff, I fully welcome and embrace it. For example, we already have codegen for various stacks, even Ruby on Rails: https://guides.rubyonrails.org/generators.html https://guides.rubyonrails.org/generators.html Sadly, model driven development never really took off, because seemingly everyone was interested in being able to iterate and ship new features in frameworks quickly, as opposed to slowing down and working on different ways to interact with what's already there. SOAP had WSDL, but REST needed a whole bunch of time until we got OpenAPI and levels of codegen that SoapUI had years ago: https://github.com/OpenAPITools/openapi-generator https://github.com/OpenAPITools/openapi-generator MySQL Workbench also has great bi-directional ER diagram support, with forward/backward engineering and schema sync: https://dev.mysql.com/doc/workbench/en/wb-design-engineering.html https://dev.mysql.com/doc/workbench/en/wb-design-engineering... But maybe I'm asking for too much, generating templates for views with model fields, or even model bindings for the ORM is already good, as is generating DB migrations as well.
boxed 3y ago
I like the Shape Up approach. I'm currently the solo dev at a company and I have taken Shape Up and cut it down to the bone. When I'm asked about how long something will take I say "give me x days and we'll see how far I can get". Often one or two weeks. This is honest and understandable.
jschveibinz 3y ago
Dr. Barry Boehm (USC) basically invented the rational approach to software estimation. Every software engineer responsible for estimating used his Cocomo techniques (or similar) through the early 2000’s. Most of his work still applies: https://www.gristprojectmanagement.us/software-2/software-effort-estimation-techniques.html https://www.gristprojectmanagement.us/software-2/software-ef... https://en.wikipedia.org/wiki/Barry_Boehm https://en.wikipedia.org/wiki/Barry_Boehm You can buy his books from online bookstores.
- kqr 3y ago
  COCOMO is an impressive project and I wish someone would update it with modern projects to see if/how things have changed. But the main rule of thumb I brought from COCOMO was that it takes 5 person-days to get 100 lines of code from idea to production. That has been remarkably accurate in my experience. I've witnessed high-performing teams do 125 lines in 5 days, and low-performing teams doing 60 lines in 5 days, but it's still well in the ballpark.
  - jschveibinz 3y ago
    Well, maybe the new “GPT” tools will help us to do that! Sounds like a good project for a masters thesis.
lifeisstillgood 3y ago
No! >>> someone at the top needs to see if the company can invest that much money and for how long, which requires some numeric value attached to a project. Software is an enabler. It's still blowing up the world. Almost all new projects are "build a new capability we don't currently have". Can you fire lasers from the backs of sharks? No then do this shark-laser project and you can. The actual cost is kinda irrelevant - either laser sharks are awesome for your business or not. Cost control in software development is not an issue of estimation and project management - that's for ditch digging. Cost control in software is about iterative development, early surfacing and engagement, it's about small teams well lead and left alone. Want integration - build an integration environment and a dashboard with tests on it - dont build a gannt chart. I think I need to get something off my chest
ultra_nick 3y ago
These are good basics, but will still underestimate large projects. For a padding rule, use: estimate^2 Because T tasks times N estimation errors/delays is an exponential growth in project time complexity. For statistics and data, see here: https://erikbern.com/2019/04/15/why-software-projects-take-longer-than-you-think-a-statistical-model.html https://erikbern.com/2019/04/15/why-software-projects-take-l...

180 points by bndr 3y ago

One thing I found crucial when dealing with estimation is to always, always explicitly add the level of confidence (in the estimate). This saves so many tough discussions later... either have it built in to your product development process (e.g. cone of uncertainty, or different stages with different predefined levels of confidence), or always communicate it right next to your estimates. Some stakeholders will push back initially, but you can reason and tell them that you need to invest more time into research, planning, PoCs etc. if they want a more accurate estimation. Eventually, it boils down to "are we agile yet", trust in the organization, maturity and culture. One of the questions I always ask when interviewing for a role is how this is done in the company, with actual examples when projects took longer than planned (and what happened). Tells a lot about the maturity of the org.

jon-wood 3y ago
I get really frustrated that despite the advice to give confidence levels being all over the place I’m aware of no project planning software which incorporates that advice. It all assumes a perfect world where you can say it will take 6.5 days to complete a task, and then cascades from there. I want project planning software which accepts lower and upper bounds, or an estimate + %age confidence, and then at the end gives me a project plan with error bars on each part of the timeline. Let me give stakeholders a diagram showing exactly what we mean by it being hard to say how long this will take, rather than what appears to be a rock solid plan timed to the day. Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several).
- ochronus 3y ago
  True that. It's inconvenient, but you can use custom fields (e.g. in JIRA) for this, but I agree, it's not a proper solution as it doesn't display e.g. ranges :/
  - alkonaut 3y ago
    Now I'm thinking I should build a better Jira where the basic unit of everything is a range instead of a number. I wonder how long that would take me.
    - kayodelycaon 3y ago
      If you’re looking to capture all of JIRA’s features, it would take you a very long time. Duplicating their extensive automation capabilities is a massive project. If you ever get around to that, let me know. JIRA really is best in class and I want to shove it into a very tiny, very lonely box before tossing it into the deepest part of the ocean I can find.
    - kqr 3y ago
      The area of statistically sane software for regular users is so underexploited that there are many instances of X you can plug into "X but with ranges instead of numbers" and carve out a niche for yourself. I myself am hoping someone will pick up X=spreadsheets at some point soon.
      - jon-wood 3y ago
        You’re looking for causal.app for that, and it’s as good as it sounds.
- Ensorceled 3y ago
  > Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several). My favourite is the ones who not only will push back on confidence levels but will then drill down into any estimates they think are "conservative".
- marcosdumay 3y ago
  If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. If you decide they are described by fat-tailed distributions¹, then the exact distribution and its parameters are much more important than your estimated intervals, so again collecting them adds nothing. So, yeah, collecting confidence intervals never adds anything. 1 - Like they should, because nobody does a project on anything they know well enough to describe with classical statistics.
  - kqr 3y ago
    > If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)
    - kelseyfrog 3y ago
      I believe what the parent is referring to is that the sum of two normally distributed variables can be modeled as normally distributed[1]. Ie: normal distributions are closed under addition. This fact leads to one's naive idea about adding variances working as expected. However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time. Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode. As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated. 1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra... 2. See the Fenton-Wilkinson approximation.
- thebiss 3y ago
  Microsoft Project does support PERT estimates, though it's difficult to find and use: https://support.microsoft.com/en-us/office/expected-duration-task-field-d2d5d25f-0fe9-4ab7-8724-7a5172a02e2d https://support.microsoft.com/en-us/office/expected-duration...
  - kqr 3y ago
    Doesn't PERT prescribe simply adding together the optimistic and pessimistic cases with each other? Percentiles are not additive.
- nicolas_t 3y ago
  Fogcreek's software fogbugz had that... It's a pity that it's mostly been abandoned.
Too 3y ago
Estimating in highly inaccurate resolution, eg fibonacci or powers of 2, is a way to embed the lack of confidence in single number estimates. You may think something takes 9 days, but if you are only allowed to answer either 8 or 16, choosing 16 is the only safe option and automatically includes some lack of confidence.
- digging 3y ago
  While that's true, it doesn't work if everyone doesn't have the exact same understanding of how it works. For example, most people I've worked with will choose 8 instead of 16 for an estimate of 9. This makes it risky to overestimate and appear less capable.
- ochronus 3y ago
  It's a suitable approach in the local context (tickets/tasks for the team), but won't work on the project level and with stakeholders. Stakeholders eventually will want a timeframe.

ohthehugemanate 3y ago

There is lots of good advice here, but I'm always amazed that these articles miss the #1 improvement to estimates recommended and validated by scientific studies: do not estimate in units of time. Everything else here applies, but don't estimate in time. Estimate in difficulty points, t shirt sizes, cups of coffee, gold stars, anything will do. Then as you track your outcomes (see TFA), measure the average relationship between your units and time. Then use that relationship to project a timeline based on your gold stars or whatever. Using the law of large numbers to average externalities and developer inaccuracy into your time estimate has been the most accurate method since Kahneman and Tversky's Nobel prize winning studies in the 1970s.

techdragon 3y ago
I’m always trying to gather extra ammunition for when I’m arguing for points based estimation methodology… do you (or anyone else) have any favourite scientific studies on this ?
switch007 3y ago
My previous manager and PM would scrutinise the time taken in Jira, then create a personal spreadsheet to roughly map points to hours/days (because senior stakeholders don't think in cups of coffee or t-shirt sizes), then refer to the spreadsheet privately during estimation. But he eventually let slip about the existence of it So we just bypassed the charade and starting giving hours/days. The whole team was really just doing hours/days -> points mapping in their heads during estimation anyway.
- ohthehugemanate 3y ago
  > The whole team was really just doing hours/days -> points mapping in their heads during estimation anyway. This is a common story. But because of the calculation step, your PM was still mapping estimated time units to actual time consumed, and using that mapping for predictions. So you were getting some of the benefits of an abstracted unit (accuracy, averaging in unexpected events) without the stress reduction or focus on quality over time. But if it worked for your team, I would advocate to keep it. Human groups are hard enough to coordinate as it is, there's no need to hew to a dogma.

ssss11 3y ago

I liked this article. Practical, not full of bs, not full of bias, some good tips. Might check out his other articles now.

okaleniuk 3y ago

I stepped in as a project manager for my R&D group when the war started. I wasn't going to but we had nobody else at the moment so here I am. One of the perks of this arrangement is that I basically can do whatever I want. Worse case scenario - I'm back to my engineering job I wasn't going to leave to begin with. So the first thing I did when I stepped in is I made estimations paid. As a business partner, you can request an estimation for a feature, but an estimation itself then becomes a project. Alternatively, you can just tell me your deadline and we'll manage things on our side to fit your date. Do you know how many times did we do the "estimation as a project"? Once. And their project got cancelled anyway so we didn't even have to do the actual work and nobody knows if our estimate was accurate. I've heard many times that business needs estimations. As it turns out, it doesn't. In times of inherent unpredictability, the real priorities show. All that business needs is to be better than competitors. It needs to create marketable value. It needs ideas, it needs analysis, it needs expertise and hard work. Estimations don't help create value. They only create more work. As soon as you make this work visible, poof - nobody wants estimations anymore!

regularfry 3y ago
Yep. I've been through this cycle too. The phrase that helped me sell the idea was "estimation is design". You can't come to an estimate that has any relationship to reality without knowing what you're intending to build, and that's a design function, even if it's very high-level, coarse, and approximate. Just expecting that design to materialise from thin air is extremely unrealistic, so you have to go through some sort of intentional process to get there. Making that process cause an appropriate amount of pain does make it clear that none of it is free.
AnimalMuppet 3y ago
I love this, but you can't always use it. Sometimes management needs an estimate (even if a coarse one) to decide whether the project is worth doing or not. Sometimes you're doing fixed-price bids for a customer, and you'd better know how much to ask for in the bid. And so on. But your main point is absolutely right. Estimates aren't free. They're a project, even if a small one.
marcosdumay 3y ago

Tade0 3y ago

I noticed one important point mentioned in the banking app example, but not elaborated on: Shirnk critical scope. By this I mean if you can produce a simpler version of whatever is that you're doing and have separate tasks for expanding it, do it. Your e-commerce solution can start off with just one payment provider. Your form validation can initially respond with just "nope!". You don't need that color picker shaped like a peacock in your MVP. Especially if these things prevent you from delivering something that someone, anyone can use. Also half of such things become irrelevant and ultimately get dropped.

spongeb00b 3y ago
I repeatedly see features being placed into a "phase 2" that appears approx 90% into a project as the deadline looms. Half of those features eventually get implemented, the rest it’s realised are not needed at all.
- alkonaut 3y ago
  That's the correct way of going about any software development. The problem though is the customer will likely be unwilling to agree to a contract that doesn't have at least 75% of the scope or the product is "unusable". Their existing/previous product is usually better all the way until every single feature of it is available in the new system (I have not yet in my career ever seen a system that isn't replacing a previous system - at least if you count an excel sheet or a piece of paper as "previous system"). What you end up with when shipping an MVP is you also convert a minimum number of users and you end up with two systems. The "MVP" is thus often "whatever allows all existing users to switch" which is why projects snowball to be too big and too late. You won't realize what people used the old system for, until you try to replace it. And it always turns out it has 2000 features when your discovery had found 20.

sanitycheck 3y ago

The elephant in the room here is, of course, that often accurate estimates are unwanted. If you tell the truth about how long something will take, maybe your customer will go with someone else. Maybe the CTO will decide to outsource instead of giving the work to your team. You know those other guys are probably as good at estimation as you are, they're just deliberately going low so you have to too. (And thus, everybody now expects IT projects to overrun!) Other than that I agree with most of the linked article, I get very good results by breaking tasks down into smaller (<1 week) jobs and getting a best and worse case estimate for each. Sum both columns, multiply both by some number for unknown unknowns (usually 1.5, 2 when there are bad vibes) and you have a realistic minimum time and a worst case. If the max is >2X the min (it will be to begin with!), get more information to narrow the range, repeat until it's not outrageous.

cratermoon 3y ago
> accurate estimates are unwanted In my career I've learned that estimating usually boils down to a game involving guessing what the customer will accept, while the customer tries to figure out what they want. Dig deep enough, and there's usually a dependency like "it needs to be done before X", but when X will be done is also imperfectly known, so it's a tangle of dependencies. The cases of hard dates usually involve businesses dependent on seasonal, holiday revenue or scheduled events like sports or entertainment touring schedules. Estimating for those is much easier, because the game becomes one of figuring what can be delivered by the date(s) in question. Regulatory changes or fiscal timelines are usually not as fixed as they are believed, because there's lots of ways to paper a path over delays, although its costs money and business don't like that.
dasil003 3y ago
“Telling the truth” implies two things: that the actual effort needed is fixed, and that you know it. In practice I’ve found neither of those things to be anywhere close to true for large projects. Furthermore the pursuit of accuracy by doing finer and finer-grained estimates and then rolling them up can waste a lot of time because it provides fodder for bike shedding while missing the forest for the trees. I can’t count the number of times I’ve seen this process lead to a total train wreck. What works better is to be very clear on the big picture goals, do a speculative high level breakdown to inform an initial deadline and staffing commitment, but keep the precise scope flexible. Then get right into prototyping and building, attacking the areas of largest risk and refining the requirements as you go. You only should have fine-grained plans for the next 2-3 months with the rest of the roadmap intentionally being flexible, save for any key milestones that are needed to ensure progress to the high level goal. Of course this requires deep expertise with cross-functional influence and bi-directional trust with management, but if you don’t have those things big projects are fucked regardless. In that case your best bet is hunkering down with some agile methodology as a shield while looking for a better job.

kitanata 3y ago

Wow. A solid quality article. Great info. Well done!

sshb 3y ago

Great article on this topic: https://apenwarr.ca/log/20171213 https://apenwarr.ca/log/20171213

thor_molecules 3y ago
yeah this is really good - thank u
flurdy 3y ago
Very good. I would also balance the OP article with Allen Holub's talks https://www.youtube.com/watch?v=QVBlnCTu9Ms https://www.youtube.com/watch?v=QVBlnCTu9Ms

vp8989 3y ago

Anecdotally, I've observed across my ~12 year career so far that an emphasis on estimates and estimating is negatively correlated to productivity, lead time, velocity, impact, positive outcomes etc... I suspect the reason is because management is trying to use numbers to justify bin packing more work to an already oversubscribed team. What never shows up in those project management spreadsheets is the very real and predictable cost of context switching and the increase in mistakes from dealing with a larger amount of in-flight work.

rqtwteye 3y ago
Totally agree. My management mostly is interested in estimates and deadlines . They never engage in discussions about productivity or quality. This results in people continuing ineffective processes and other systems. It also encourages people to make tests pass at any cost or close tickets even when there are deeper problems that should be resolved. In short, the focus on estimates
candiddevmike 3y ago
Estimating and a lot of project management stuff is a form of procrastination for a lot of folks/businesses. It doesn't matter what fields the cards have or how they are arranged, the work still needs to get done, and all of this shuffling is just delaying the start of that work.
- delusional 3y ago
  That's an interesting thought. I've never considered it before, but estimation as procrastination might be a good explanation.
- AnimalMuppet 3y ago
  That's only true if there's a fixed set of work to get done. But that's rarely the case. Often, management has N different things they could have done, and enough people and time to do M of them, for M < N. Which ones should they do? Well, whatever maximizes profits. So they (management) estimate income from each thing that could be done, and ask engineering (hopefully) to estimate how much it will cost to implement (or how long it will take, which equates to cost). Then they make a (hopefully) more informed decision than they otherwise could have made. Look, there's lots of ways this gets done badly. I get that. But the idea itself is not nonsense.

jcon321 3y ago

All I know is government contracts, but I've been pretty successful at estimations. Much like the top posts in this thread, none of it is for free. Most of the time requirements come in a few sentences, maybe a paragraph, rare cases a document. There's no estimation until the requirements are flushed out in some type of use case which has some bearing on the technical constraints of the system. (The client pays for this, and provides feedback.) The next step is planning, which is where estimation comes from. (The client pays for this.) In terms of planning I do strive for maximum 16 hour chunks which helps accuracy. For estimations that I have low confidence in, I'll have conditional chunks that may be used. All of this is transparent to the client. If the client asks for an estimation before the above steps it's known by everyone that it has no official meaning, because everyone is aware of the above process.

jt2190 3y ago

The smaller the potential upside of the project is, the more the estimates matter. Put another way: A project that promises to return USD 10 for every USD 1 invested is still an attractive investment when the ratio is 8:1 or 7:1. This suggests that construction costs can double or triple without destroying and the investment still has a healthy return, and so an estimate really only needs to be very rough. (In fact, the act of estimating quickly adds to “construction” costs so should be kept to a bare minimum in these cases, to keep profits as high as possible.) It’s when the cost/profit ratio slips lower that estimates become more important, and ironically, waste more time and money. (Note that I’m talking specifically about estimation and not project planning… projects, especially large ones, still need plans.)

commandlinefan 3y ago
> A project that promises to return USD 10 for every USD 1 invested is still an attractive investment when the ratio is 8:1 or 7:1 What in the world experience have you or anybody else ever had where a software project can be estimated with that level of accuracy?

vrglvrglvrgl 3y ago

[dead]

throwawaaarrgh 3y ago

1. Have you done it before? If no, triple your estimate. 2. Do you know everything involved? If no, double your estimate. 3. Do you have someone that you will ask questions of or pair with to get through difficult sections quickly? If no, add half to the estimate. 4. Does your team like to nit pick pull requests? Add half to the estimate. 5. Factor average weekly toil into the estimate. If you don't measure this, triple the estimate. This may seem like a really big estimate. But measure the estimate against the outcome and tell me I'm wrong.

falcolas 3y ago
All this while remembering that you will probably only get 4 hours a day to dedicate to the project. Between responding to emails and chats, paperwork, training, meetings, and context switching - your day will be half gone before you can even touch a project.
jgeada 3y ago
Estimating a project that doesn't have detailed scope and specifications is a bit like asking "how long is a piece of string". Asking for an estimate before knowing what it is you are building is canonical MBA widget production management. Numbers and dates unrelated to reality so the idiots with the power can put data into their spreadsheets and claim they're doing something. Software has nearly 0 reproduction costs, so by definition you'll (mostly) only write software for something that has not been done before. How long will it take to quantify the unknown?
- chasd00 3y ago
  what i do is break it into two projects. The first is a "discovery" project where you learn what you need to know to give an accurate estimate. You can usually get a discovery estimate in the ballpark by understanding how complex the ask is and how many people you're going to have to interview. Then you deliver the actual estimate along with all your evidence that will drive the scope definition.
- throwawaaarrgh 3y ago
  Sorry, I guess I wasn't clear; my suggestions are for when you already have scope and specifications. When you think you know what to do and how to do it, you can then add all the extra time to come to the "actual estimate".

zeroonetwothree 3y ago

All you need to know about estimates is Hofstadter’s law.

AlbertCory 3y ago

I wrote about this: https://albertcory50.substack.com/p/how-to-finish-a-software-project https://albertcory50.substack.com/p/how-to-finish-a-software... The key thing to note is: It's not just big software projects; it's ALL big projects The California Bullet Train is a poster child for this, as is the Big Dig in Boston, and (probably) your kitchen remodeling. Once you accept this, you can learn from the way other disciplines handle it. Or fail to.

perk 3y ago

We created a small webapp that works pretty good for us: https://calestimate.com https://calestimate.com (No login needed to try it out)

brunoocasali 3y ago

Thank you for sharing your knowledge. Talking about estimations is always complex.

exabrial 3y ago

This is my rule: Start with the developer's estimate. Then for every management layer in the organization, multiple by 2.25 until you hit the c-suite. This turns out to be surprisingly accurate when measured against reality. I'm guessing someone has already thought of it and I don't really have a name for it.

disgruntledphd2 3y ago
This was my estimation process for construction add-ons (back when I did this between college courses). Take an estimate, double it and add 50%. I always regret when I don't take this approach.
rqtwteye 3y ago
My factor is 5. Works like a charm. It’s just not politically correct to say so.
tryauuum 3y ago
This is nice, although there's a self-fulfilling prophecy element in this. You will postpone your project to fill up the declared time

chasd00 3y ago

I work in consulting where if your estimates are not right then you don't eat. One thing many people forget is that writing code is only a small part of the overall effort in software development. I feel like i could write a book on the subject but I think the most common error is not understanding the scope of the overall effort from an idea to effective use by end users and only estimating the typing-on-the-keyboard part.

KronisLV 3y ago

> The "one size fits all" approach: Assuming that every task will take the same amount of time is a recipe for disaster. Different tasks have different complexities and require different skill sets. I sometimes wish we lived in a world where we'd have enough data to accurately predict at least the common tasks, thus freeing people up to reason about the more complex tasks. For example: adding a new path to a webapp, which will resolve to a Ruby on Rails template, that needs 2 dropdowns and 3 input fields, which need input validations and checks against already saved DB data on saving will take around X hours of work. Doing this with Python and Django will take around Y hours of work. Using a certain set of tools to assist with development will affect these estimates by Z hours. But I doubt anyone ever has data that granular.

kqr 3y ago
I think once we get to the point where the design is so fixed for a set of standardised tasks like that, that we know how long it will take, the design is also so fixed that we can run a script that generates the code for it, so the time to make it is close to zero.
- KronisLV 3y ago
  > I think once we get to the point where the design is so fixed for a set of standardised tasks like that, that we know how long it will take, the design is also so fixed that we can run a script that generates the code for it, so the time to make it is close to zero. For all of the boilerplate stuff, I fully welcome and embrace it. For example, we already have codegen for various stacks, even Ruby on Rails: https://guides.rubyonrails.org/generators.html https://guides.rubyonrails.org/generators.html Sadly, model driven development never really took off, because seemingly everyone was interested in being able to iterate and ship new features in frameworks quickly, as opposed to slowing down and working on different ways to interact with what's already there. SOAP had WSDL, but REST needed a whole bunch of time until we got OpenAPI and levels of codegen that SoapUI had years ago: https://github.com/OpenAPITools/openapi-generator https://github.com/OpenAPITools/openapi-generator MySQL Workbench also has great bi-directional ER diagram support, with forward/backward engineering and schema sync: https://dev.mysql.com/doc/workbench/en/wb-design-engineering.html https://dev.mysql.com/doc/workbench/en/wb-design-engineering... But maybe I'm asking for too much, generating templates for views with model fields, or even model bindings for the ORM is already good, as is generating DB migrations as well.

boxed 3y ago

I like the Shape Up approach. I'm currently the solo dev at a company and I have taken Shape Up and cut it down to the bone. When I'm asked about how long something will take I say "give me x days and we'll see how far I can get". Often one or two weeks. This is honest and understandable.

jschveibinz 3y ago

Dr. Barry Boehm (USC) basically invented the rational approach to software estimation. Every software engineer responsible for estimating used his Cocomo techniques (or similar) through the early 2000’s. Most of his work still applies: https://www.gristprojectmanagement.us/software-2/software-effort-estimation-techniques.html https://www.gristprojectmanagement.us/software-2/software-ef... https://en.wikipedia.org/wiki/Barry_Boehm https://en.wikipedia.org/wiki/Barry_Boehm You can buy his books from online bookstores.

kqr 3y ago
COCOMO is an impressive project and I wish someone would update it with modern projects to see if/how things have changed. But the main rule of thumb I brought from COCOMO was that it takes 5 person-days to get 100 lines of code from idea to production. That has been remarkably accurate in my experience. I've witnessed high-performing teams do 125 lines in 5 days, and low-performing teams doing 60 lines in 5 days, but it's still well in the ballpark.
- jschveibinz 3y ago
  Well, maybe the new “GPT” tools will help us to do that! Sounds like a good project for a masters thesis.

lifeisstillgood 3y ago

No! >>> someone at the top needs to see if the company can invest that much money and for how long, which requires some numeric value attached to a project. Software is an enabler. It's still blowing up the world. Almost all new projects are "build a new capability we don't currently have". Can you fire lasers from the backs of sharks? No then do this shark-laser project and you can. The actual cost is kinda irrelevant - either laser sharks are awesome for your business or not. Cost control in software development is not an issue of estimation and project management - that's for ditch digging. Cost control in software is about iterative development, early surfacing and engagement, it's about small teams well lead and left alone. Want integration - build an integration environment and a dashboard with tests on it - dont build a gannt chart. I think I need to get something off my chest

ultra_nick 3y ago

These are good basics, but will still underestimate large projects. For a padding rule, use: estimate^2 Because T tasks times N estimation errors/delays is an exponential growth in project time complexity. For statistics and data, see here: https://erikbern.com/2019/04/15/why-software-projects-take-longer-than-you-think-a-statistical-model.html https://erikbern.com/2019/04/15/why-software-projects-take-l...