4 ms·

One thing I found crucial when dealing with estimation is to always, always explicitly add the level of confidence (in the estimate). This saves so many tough d

One thing I found crucial when dealing with estimation is to always, always explicitly add the level of confidence (in the estimate). This saves so many tough discussions later... either have it built in to your product development process (e.g. cone of uncertainty, or different stages with different predefined levels of confidence), or always communicate it right next to your estimates. Some stakeholders will push back initially, but you can reason and tell them that you need to invest more time into research, planning, PoCs etc. if they want a more accurate estimation. Eventually, it boils down to "are we agile yet", trust in the organization, maturity and culture. One of the questions I always ask when interviewing for a role is how this is done in the company, with actual examples when projects took longer than planned (and what happened). Tells a lot about the maturity of the org.

jon-wood 3y ago
I get really frustrated that despite the advice to give confidence levels being all over the place I’m aware of no project planning software which incorporates that advice. It all assumes a perfect world where you can say it will take 6.5 days to complete a task, and then cascades from there. I want project planning software which accepts lower and upper bounds, or an estimate + %age confidence, and then at the end gives me a project plan with error bars on each part of the timeline. Let me give stakeholders a diagram showing exactly what we mean by it being hard to say how long this will take, rather than what appears to be a rock solid plan timed to the day. Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several).
- ochronus 3y ago
  True that. It's inconvenient, but you can use custom fields (e.g. in JIRA) for this, but I agree, it's not a proper solution as it doesn't display e.g. ranges :/
  - alkonaut 3y ago
    Now I'm thinking I should build a better Jira where the basic unit of everything is a range instead of a number. I wonder how long that would take me.
    - kayodelycaon 3y ago
      If you’re looking to capture all of JIRA’s features, it would take you a very long time. Duplicating their extensive automation capabilities is a massive project. If you ever get around to that, let me know. JIRA really is best in class and I want to shove it into a very tiny, very lonely box before tossing it into the deepest part of the ocean I can find.
    - kqr 3y ago
      The area of statistically sane software for regular users is so underexploited that there are many instances of X you can plug into "X but with ranges instead of numbers" and carve out a niche for yourself. I myself am hoping someone will pick up X=spreadsheets at some point soon.
      - jon-wood 3y ago
        You’re looking for causal.app for that, and it’s as good as it sounds.
- Ensorceled 3y ago
  > Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several). My favourite is the ones who not only will push back on confidence levels but will then drill down into any estimates they think are "conservative".
- marcosdumay 3y ago
  If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. If you decide they are described by fat-tailed distributions¹, then the exact distribution and its parameters are much more important than your estimated intervals, so again collecting them adds nothing. So, yeah, collecting confidence intervals never adds anything. 1 - Like they should, because nobody does a project on anything they know well enough to describe with classical statistics.
  - kqr 3y ago
    > If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)
    - kelseyfrog 3y ago
      I believe what the parent is referring to is that the sum of two normally distributed variables can be modeled as normally distributed[1]. Ie: normal distributions are closed under addition. This fact leads to one's naive idea about adding variances working as expected. However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time. Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode. As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated. 1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra... 2. See the Fenton-Wilkinson approximation.
- thebiss 3y ago
  Microsoft Project does support PERT estimates, though it's difficult to find and use: https://support.microsoft.com/en-us/office/expected-duration-task-field-d2d5d25f-0fe9-4ab7-8724-7a5172a02e2d https://support.microsoft.com/en-us/office/expected-duration...
  - kqr 3y ago
    Doesn't PERT prescribe simply adding together the optimistic and pessimistic cases with each other? Percentiles are not additive.
- nicolas_t 3y ago
  Fogcreek's software fogbugz had that... It's a pity that it's mostly been abandoned.
Too 3y ago
Estimating in highly inaccurate resolution, eg fibonacci or powers of 2, is a way to embed the lack of confidence in single number estimates. You may think something takes 9 days, but if you are only allowed to answer either 8 or 16, choosing 16 is the only safe option and automatically includes some lack of confidence.
- digging 3y ago
  While that's true, it doesn't work if everyone doesn't have the exact same understanding of how it works. For example, most people I've worked with will choose 8 instead of 16 for an estimate of 9. This makes it risky to overestimate and appear less capable.
- ochronus 3y ago
  It's a suitable approach in the local context (tickets/tasks for the team), but won't work on the project level and with stakeholders. Stakeholders eventually will want a timeframe.

jon-wood 3y ago

I get really frustrated that despite the advice to give confidence levels being all over the place I’m aware of no project planning software which incorporates that advice. It all assumes a perfect world where you can say it will take 6.5 days to complete a task, and then cascades from there. I want project planning software which accepts lower and upper bounds, or an estimate + %age confidence, and then at the end gives me a project plan with error bars on each part of the timeline. Let me give stakeholders a diagram showing exactly what we mean by it being hard to say how long this will take, rather than what appears to be a rock solid plan timed to the day. Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several).

ochronus 3y ago
True that. It's inconvenient, but you can use custom fields (e.g. in JIRA) for this, but I agree, it's not a proper solution as it doesn't display e.g. ranges :/
- alkonaut 3y ago
  Now I'm thinking I should build a better Jira where the basic unit of everything is a range instead of a number. I wonder how long that would take me.
  - kayodelycaon 3y ago
    If you’re looking to capture all of JIRA’s features, it would take you a very long time. Duplicating their extensive automation capabilities is a massive project. If you ever get around to that, let me know. JIRA really is best in class and I want to shove it into a very tiny, very lonely box before tossing it into the deepest part of the ocean I can find.
  - kqr 3y ago
    The area of statistically sane software for regular users is so underexploited that there are many instances of X you can plug into "X but with ranges instead of numbers" and carve out a niche for yourself. I myself am hoping someone will pick up X=spreadsheets at some point soon.
    - jon-wood 3y ago
      You’re looking for causal.app for that, and it’s as good as it sounds.
Ensorceled 3y ago
> Also, let me fire any project manager who pushes back on giving confidence levels (yes, I’ve met several). My favourite is the ones who not only will push back on confidence levels but will then drill down into any estimates they think are "conservative".
marcosdumay 3y ago
If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. If you decide they are described by fat-tailed distributions¹, then the exact distribution and its parameters are much more important than your estimated intervals, so again collecting them adds nothing. So, yeah, collecting confidence intervals never adds anything. 1 - Like they should, because nobody does a project on anything they know well enough to describe with classical statistics.
- kqr 3y ago
  > If your decide your confidence intervals are described by classical statistics, then adding them won't change anything on the outcome. I'm trying to understand what you mean by this. Are you referring to the fact that for thin tails, the sum of expectations will grow significantly faster than the standard deviation, and thus the 90th percentile will, relatively speaking, tend toward the expectation with more tasks? (I.e. an appeal to the LLN.)
  - kelseyfrog 3y ago
    I believe what the parent is referring to is that the sum of two normally distributed variables can be modeled as normally distributed[1]. Ie: normal distributions are closed under addition. This fact leads to one's naive idea about adding variances working as expected. However, in task estimation[duration] aren't normally distributed - they're much more log-normally distributed, and you can't simply add log-normal distribution parameters in the same way[2]. Instead he log-normal distribution is (largely) closed under multiplication which is fairly useless when we want to add tasks to determine total time. Moreover, the reductive conceptualization of normally distributed estimations leads people to the erroneous communication of "X plus or minus Y". The assumption is that the bounds are symmetric(they aren't). The chance that the estimate is higher than the mode is much greater than it is lower than the mode. As a data scientist, I want to say that the model for task estimation is over-reduced. Even so, I feel like the push-back I'd get from suggesting that folks estimate in log-space would be that I'm being ridiculous complicated. 1. https://en.wikipedia.org/wiki/Sum_of_normally_distributed_random_variables https://en.wikipedia.org/wiki/Sum_of_normally_distributed_ra... 2. See the Fenton-Wilkinson approximation.
thebiss 3y ago
Microsoft Project does support PERT estimates, though it's difficult to find and use: https://support.microsoft.com/en-us/office/expected-duration-task-field-d2d5d25f-0fe9-4ab7-8724-7a5172a02e2d https://support.microsoft.com/en-us/office/expected-duration...
- kqr 3y ago
  Doesn't PERT prescribe simply adding together the optimistic and pessimistic cases with each other? Percentiles are not additive.
nicolas_t 3y ago
Fogcreek's software fogbugz had that... It's a pity that it's mostly been abandoned.

Too 3y ago

Estimating in highly inaccurate resolution, eg fibonacci or powers of 2, is a way to embed the lack of confidence in single number estimates. You may think something takes 9 days, but if you are only allowed to answer either 8 or 16, choosing 16 is the only safe option and automatically includes some lack of confidence.

digging 3y ago
While that's true, it doesn't work if everyone doesn't have the exact same understanding of how it works. For example, most people I've worked with will choose 8 instead of 16 for an estimate of 9. This makes it risky to overestimate and appear less capable.
ochronus 3y ago
It's a suitable approach in the local context (tickets/tasks for the team), but won't work on the project level and with stakeholders. Stakeholders eventually will want a timeframe.