3 ms·

I can estimate some tasks very accurately. I know that adding this new flag to my system will take a day. Other tasks I can accurately say I don't know how lo

by kzhahou 11y ago

I can estimate some tasks very accurately. I know that adding this new flag to my system will take a day. Other tasks I can accurately say I don't know how long they'll take. This new feature is... a couple of weeks? This new application is... a few months? I can roughly guess (estimate!) but there's a wide certainty margin. That uncertainty is never built into the mathematical model used by project managers, which is invariably a simple addition of scalar estimates. The only common way of dealing with uncertainty is to pad the estimates (multiply by two, etc), but that's still not a statistical boundary -- it's just a scalar. Furthermore, when adding uncertain values, they wind up not being independent, for a number of reasons -- the lessons from one feature are carried on to the next, or the next task is easier because of new existing code, or because as you approach the original deadline you invariably throw out feature requirements. So you can't necessarily just add all the uncertainty values. So a better approach to estimation would be to formulate a (simple) model which encodes each estimate as {guess, uncertainty, independence} and then provides a function for compounding the guesses and uncertainty and independence factors in a realistic way.

jacques_chester 11y ago
> That uncertainty is never built into the mathematical model used by project managers It depends on the project manager. The PERT estimation technique was based on giving a 3-part range estimate for broken-down tasks. These are then rolled up with a simple formula. In the PERT method independence is handled by creating explicit links between tasks, which leads to the concept of the critical path. Nobody does these. Because it's very tedious. There are a lot of tasks for which "just try it and see" is less effort than performing a full-blown estimation raindance.
- kzhahou 11y ago
  Thanks. Yeah, there's a fairly large number of formal management techniques, but like you said, they're very rarely used. PERT: https://en.wikipedia.org/wiki/Program_evaluation_and_review_technique https://en.wikipedia.org/wiki/Program_evaluation_and_review_... Barry Boehm's recent work: http://www.amazon.com/Incremental-Commitment-Spiral-Model-Principles/dp/0321808223 http://www.amazon.com/Incremental-Commitment-Spiral-Model-Pr...
  - jacques_chester 11y ago
    I worked on an estimation tool[1] inspired by PERT 3-point estimates for a while, simply because there weren't any when I went looking one time. It was a lot of fun reading up on the estimation literature in various disciplines. Personally I think that the real magic of PERT 3-point estimation is the "unpacking effect" -- making people think about the subtasks of a headline issue. When I'm in situations where I point a story, I like to ensure that we engineers say aloud what assumptions we're pointing on ("Bootstrap will make responsive mobile almost free") and then a quick low-detail breakdown of expected tasks ("new page, new RESTful endpoint, new model code"). I find that the latter, in particular, makes a big difference in how pointing goes. [1] http://confidest.com http://confidest.com
    - cpitman 11y ago
      The other part of 3 part estimates that I like is that it forces everyone to consider the worse case. What can go wrong? What risks are there to this task? And only then do we talk about how long we think it will probably take. The problem I have always had afterwards is communicating the results. Most people I've worked with cannot intuitively grasp confidence levels (90% vs 95% vs ...). They also invariably want a break down of how long each task or group of tasks will take so that they can negotiate down the estimate, not understanding that the total estimate is not a simple sum of each task. I'd love to hear better ways of explaining/using these sorts of estimates. The first time was my fault: I left the columns that had intermediate values visible. One of those was the variance in each estimate, which is just the Std Dev squared. The numbers were big. No one could stop talking about how big the numbers were. No one knew what a variance was, nor appreciated that it was just the number in the previous column, multiplied by itself. Learned the very important lesson that you hide all your work, no one gets it.
      - jacques_chester 11y ago
        Culture, really. Trust. Where I work, Engineering owns the estimates. No ifs, buts or maybes. We own estimates. Product managers have unlimited ability to reprioritise, but we and we alone get to say how hard it's going to be. That said, another secret is: use story points. They are dimensionless units that are only meaningful within a single project. As soon as you give dollars or days, everybody has an opinion or an agenda. But if you use points and project based on velocity, it's hard to negotiate. Because you're not negotiating. Those are the actual numbers. The actual historical data, staring you in the face. (Edited for clarity)
wai1234 11y ago
So, small tasks you CAN estimate, large ones not so much. Perhaps the technique to use is right in front of you: decomposition.
- kzhahou 11y ago
  I can try to decompose a ~2-month task into one-day tasks, and still not improve my overall confidence level.
- ddingus 11y ago
  On a small task, everything is small. The percent error can be totally gross (not precise at all) and not have a meaningful impact overall. A large error on small is just a multiplier of small. It's all small. When you decompose a larger task into smaller ones, that's all still true for each of the smaller ones, right? But you must remember to add the errors! And that is precisely why overall confidence may not likely improve with decomposition vs. an aggregate, or coarse, high level estimate of the whole thing. And this is true for more than software. Why? New tasks can be broken down into two coarse buckets: done before and novel. Novel things really are novel! It's important to recognize this case and qualify it. Progress might look like: week 1 - 10 percent done, week 2 - 10 percent done, week 3 - 15 percent done, week 4 - 50 percent done! Yes!, week 5 - 5 percent done... everybody take a drink, week 6 - 40 percent done, etc... Because it's novel, there are new dynamics in play, and even the best planning may not account for, and actually usually does not account for, things that just come up and that must be dealt with. For the done before case, it's better as a lot is known, but there are still many factors out of the team control. Updates, changes, regressions, unfound bugs, all contribute to variance. The more the organization completely owns, the less this may be true, but it's always somewhat true. The pragmatic person might classify everything as having some percentage of novel, and break it down into known novel and unknown, "it just hit us!" type novel and go from there and likely be better off when having to manage expectations. Another way to think of this, depending on it being software or something else, is production vs "R&D" and the research and development part just isn't cog / production like. Time and materials estimates, if they are even given, must be considered accordingly. If they are not, expectations do not match realities, and now there is a management and potentially a funding problem in addition to the primary problem needing a solution.

by kzhahou 11y ago

jacques_chester 11y ago

> That uncertainty is never built into the mathematical model used by project managers It depends on the project manager. The PERT estimation technique was based on giving a 3-part range estimate for broken-down tasks. These are then rolled up with a simple formula. In the PERT method independence is handled by creating explicit links between tasks, which leads to the concept of the critical path. Nobody does these. Because it's very tedious. There are a lot of tasks for which "just try it and see" is less effort than performing a full-blown estimation raindance.

kzhahou 11y ago
Thanks. Yeah, there's a fairly large number of formal management techniques, but like you said, they're very rarely used. PERT: https://en.wikipedia.org/wiki/Program_evaluation_and_review_technique https://en.wikipedia.org/wiki/Program_evaluation_and_review_... Barry Boehm's recent work: http://www.amazon.com/Incremental-Commitment-Spiral-Model-Principles/dp/0321808223 http://www.amazon.com/Incremental-Commitment-Spiral-Model-Pr...
- jacques_chester 11y ago
  I worked on an estimation tool[1] inspired by PERT 3-point estimates for a while, simply because there weren't any when I went looking one time. It was a lot of fun reading up on the estimation literature in various disciplines. Personally I think that the real magic of PERT 3-point estimation is the "unpacking effect" -- making people think about the subtasks of a headline issue. When I'm in situations where I point a story, I like to ensure that we engineers say aloud what assumptions we're pointing on ("Bootstrap will make responsive mobile almost free") and then a quick low-detail breakdown of expected tasks ("new page, new RESTful endpoint, new model code"). I find that the latter, in particular, makes a big difference in how pointing goes. [1] http://confidest.com http://confidest.com
  - cpitman 11y ago
    The other part of 3 part estimates that I like is that it forces everyone to consider the worse case. What can go wrong? What risks are there to this task? And only then do we talk about how long we think it will probably take. The problem I have always had afterwards is communicating the results. Most people I've worked with cannot intuitively grasp confidence levels (90% vs 95% vs ...). They also invariably want a break down of how long each task or group of tasks will take so that they can negotiate down the estimate, not understanding that the total estimate is not a simple sum of each task. I'd love to hear better ways of explaining/using these sorts of estimates. The first time was my fault: I left the columns that had intermediate values visible. One of those was the variance in each estimate, which is just the Std Dev squared. The numbers were big. No one could stop talking about how big the numbers were. No one knew what a variance was, nor appreciated that it was just the number in the previous column, multiplied by itself. Learned the very important lesson that you hide all your work, no one gets it.
    - jacques_chester 11y ago
      Culture, really. Trust. Where I work, Engineering owns the estimates. No ifs, buts or maybes. We own estimates. Product managers have unlimited ability to reprioritise, but we and we alone get to say how hard it's going to be. That said, another secret is: use story points. They are dimensionless units that are only meaningful within a single project. As soon as you give dollars or days, everybody has an opinion or an agenda. But if you use points and project based on velocity, it's hard to negotiate. Because you're not negotiating. Those are the actual numbers. The actual historical data, staring you in the face. (Edited for clarity)

wai1234 11y ago

So, small tasks you CAN estimate, large ones not so much. Perhaps the technique to use is right in front of you: decomposition.

kzhahou 11y ago
I can try to decompose a ~2-month task into one-day tasks, and still not improve my overall confidence level.
ddingus 11y ago
On a small task, everything is small. The percent error can be totally gross (not precise at all) and not have a meaningful impact overall. A large error on small is just a multiplier of small. It's all small. When you decompose a larger task into smaller ones, that's all still true for each of the smaller ones, right? But you must remember to add the errors! And that is precisely why overall confidence may not likely improve with decomposition vs. an aggregate, or coarse, high level estimate of the whole thing. And this is true for more than software. Why? New tasks can be broken down into two coarse buckets: done before and novel. Novel things really are novel! It's important to recognize this case and qualify it. Progress might look like: week 1 - 10 percent done, week 2 - 10 percent done, week 3 - 15 percent done, week 4 - 50 percent done! Yes!, week 5 - 5 percent done... everybody take a drink, week 6 - 40 percent done, etc... Because it's novel, there are new dynamics in play, and even the best planning may not account for, and actually usually does not account for, things that just come up and that must be dealt with. For the done before case, it's better as a lot is known, but there are still many factors out of the team control. Updates, changes, regressions, unfound bugs, all contribute to variance. The more the organization completely owns, the less this may be true, but it's always somewhat true. The pragmatic person might classify everything as having some percentage of novel, and break it down into known novel and unknown, "it just hit us!" type novel and go from there and likely be better off when having to manage expectations. Another way to think of this, depending on it being software or something else, is production vs "R&D" and the research and development part just isn't cog / production like. Time and materials estimates, if they are even given, must be considered accordingly. If they are not, expectations do not match realities, and now there is a management and potentially a funding problem in addition to the primary problem needing a solution.