7 ms·

Years ago, when I was working at a company, I did a detailed estimate of a large project. I broke it down into components and individual tasks, then put estima

Years ago, when I was working at a company, I did a detailed estimate of a large project. I broke it down into components and individual tasks, then put estimated hours against each one, then put it all into a giant spreadsheet, to track planned vs actual times. If I remember rightly, it totalled about three months of elapsed work time. And it took me over a week to compile the estimate. I also remember that I hit the predicted end date to within two days. But the interesting bit was each individual task was inaccurate - lots were wildly underestimated, balanced by those which were wildly overestimated. Stuff like things I thought would take a day taking 15 minutes, other things I thought would take an hour taking a week.

ThePhysicist 5y ago
A relative error factor of 40 sounds quite high. I also often overestimate task complexity but not to that degree, except maybe in research-like projects. I think often these over-estimates are caused by tasks where you just know the requirement but not yet how to realize it. For example, estimating the requirement "setting up TLS certificates" might yield an estimate of one week if you have never done that before. After researching how to do it you might end up with one hour of work, e.g. because you learned that you can use ACME to auto-generate certificates. Does that sound right?
- rahoulb 5y ago
  This was dev work - and generally when something goes over by that amount it's because my understanding of the requirements and the client's understanding of the requirements were very very different.
jpomykala 5y ago
Years ago, I also needed to make an estimation, so I simply described which functionalities will be available in each week. I also added a 2 weeks margin to polish everything at the end of the planned deadline, and it worked great. The client was happy because he could see the progress every week, and I was happy because I was on time with everything. During the polishing phase, we were meeting every day to make sure everything is adjusted or fixed properly.
RegW 5y ago
Interesting - and no one tried to beat you down on the end date when a task went quickly, or berated you when one overan? I truely believe that developing software is a creative process that you are always doing for the first time (unless you did it wrong the first time). Why does everyone expect us to know how long its going to take. The basic rule of thumb I use is to take my best guess and multiply by PI. (answer: because 3 isn't usually enough).
- hhggffdd 5y ago
  I think it’s because of perpetual instability of the tools we use. There are always new libraries and apis to learn. I think a lot of it is self-inflicted. If we built stuff the way we built it 10 years ago, we would still get the same product finished but having done the same thing in the same way, estimates would be more reliable. But boy did we succeed at allocating a heap of VC cash into devs pockets.
  - rahoulb 5y ago
    For me and dev work it's generally about misunderstanding the requirements. They ask for X, I think X involves A, B and C, when actually it involves A, D, E, F, G and H.
    - 5e92cb50239222b 5y ago
      Not to mention that often they ask for X while actually meaning Y.
    - rubidium 5y ago
      Yep. And that’s on you and the product owner to get aligned on. Fix that alignment and you fix the problem.
      - lostcolony 5y ago
        Time needed to fix that alignment is impossible to estimate, and also impossible to prove you have done sufficiently.
ochrist 5y ago
Did you use three point estimation: https://en.wikipedia.org/wiki/Three-point_estimation https://en.wikipedia.org/wiki/Three-point_estimation
- rahoulb 5y ago
  I didn't know about that - but I suspect (it was a while back) that I did two points - best and worst case.
xondono 5y ago
> But the interesting bit was each individual task was inaccurate - lots were wildly underestimated, balanced by those which were wildly overestimated. This is called random luck. Something tells me that if we repeated the same experiment with different projects, that distribution of errors wouldn’t cancel out most of the time.
- jmchuster 5y ago
  Not all the time, but most of the time, yes it cancels out. If you work in a business like consulting, you'll quickly learn that you make more money by being able to give estimates in this manner. And when it's part of your job, it becomes a skill that you need to train and become better at, and your estimates get more consistent and more accurate.
  - xondono 5y ago
    Yeah, I’ve done it myself. I’ve given price estimates for complex PCBAs before designing them within 0.5%, but I don’t attribute that precision to talent, I probably get the estimates in the +/-10% and got lucky in that case.
    - jmchuster 5y ago
      Ah, sorry, I misread your original statement then. Were you trying to say that getting an estimate right within 2 days was lucky, but something like 1 week would've been totally normal? It's hard to tell the percentage, since OP just said "large project".
codingdave 5y ago
Back in the first dotcom boom, we started tracking our rough swags that a dev would throw out on gut instinct vs. the detailed estimates we put together for actual customer projects. We founds the swag were more accurate, exactly for the reasons you stated -- we would over and under estimate details, but they balanced out and the scope of the overall project was correct. We then got into talks with other dev shops, with more experience than our young group... and they had the same result. Young or old, experienced or not, a rough swipe at the scale of a project is usually accurate enough to know what it will take. I've seen the same thing play out in Agile processes - we are asked for story points, and while many are accurate, some are way bigger than we thought, some smaller. But the overall time to complete a project tends to fall right about where the devs thought it would before the pointing exercises. All the estimation in the world will still put the same answers into the hands of your project managers - "Pick a date, or pick your scope. Not both."
- deleted 5y ago
  [deleted]
dboreham 5y ago
This all depends on being able to make that detailed task breakdown at the _beginning_ of the project. In my experience software development is an endeavor where that level of understanding is seldom present at the outset. Typically, at least in the fields I've worked, at the beginning you may have answered the question "is this even possible"? (but not always), but your task breakdown consists of a small set of "figure out how to approach X" tasks, and a scattering of "well-understood thing we know we need to do sometime". In this context, it's impossible to estimate the project. The best you can do is to plan roughly what work to do in the next few days/weeks, then re-group to see what comes next. I believe Agile incorporates some of these insights.
denton-scratch 5y ago
Years ago, I was given training in an estimating technique called Function Point Analysis (yes, yet another estimating technique). You broke the task down into (I think) five elementary components: inputs, outputs, processes, inter-process exchanges, something. You rated each component out of 5 for difficulty. Each type of component has a weight, so you can get a sum of the weighted scores of the components, apply some kind of fiddle-function to the sum, and that's your function-point count. You can then use intrusive management to measure the function-point output of each developer per day, so that after some months, you can make accurate estimates. Because the task estimate is denominated in function-points, not days, you can use the FPs-per-day rates of individuals or teams to estimate the elapsed time for a task. I tried to use it a few times. What I learned is that it's not possible to make accurate estimates. Actually, that was one of the observations of the FPA training: that you are supposed to keep on doing it all through a project. Well, you really needed a whole corporation to commit to supporting something like that. I'm not surprised it was no use to me.

ThePhysicist 5y ago

A relative error factor of 40 sounds quite high. I also often overestimate task complexity but not to that degree, except maybe in research-like projects. I think often these over-estimates are caused by tasks where you just know the requirement but not yet how to realize it. For example, estimating the requirement "setting up TLS certificates" might yield an estimate of one week if you have never done that before. After researching how to do it you might end up with one hour of work, e.g. because you learned that you can use ACME to auto-generate certificates. Does that sound right?

rahoulb 5y ago
This was dev work - and generally when something goes over by that amount it's because my understanding of the requirements and the client's understanding of the requirements were very very different.

jpomykala 5y ago

Years ago, I also needed to make an estimation, so I simply described which functionalities will be available in each week. I also added a 2 weeks margin to polish everything at the end of the planned deadline, and it worked great. The client was happy because he could see the progress every week, and I was happy because I was on time with everything. During the polishing phase, we were meeting every day to make sure everything is adjusted or fixed properly.

RegW 5y ago

Interesting - and no one tried to beat you down on the end date when a task went quickly, or berated you when one overan? I truely believe that developing software is a creative process that you are always doing for the first time (unless you did it wrong the first time). Why does everyone expect us to know how long its going to take. The basic rule of thumb I use is to take my best guess and multiply by PI. (answer: because 3 isn't usually enough).

hhggffdd 5y ago
I think it’s because of perpetual instability of the tools we use. There are always new libraries and apis to learn. I think a lot of it is self-inflicted. If we built stuff the way we built it 10 years ago, we would still get the same product finished but having done the same thing in the same way, estimates would be more reliable. But boy did we succeed at allocating a heap of VC cash into devs pockets.
- rahoulb 5y ago
  For me and dev work it's generally about misunderstanding the requirements. They ask for X, I think X involves A, B and C, when actually it involves A, D, E, F, G and H.
  - 5e92cb50239222b 5y ago
    Not to mention that often they ask for X while actually meaning Y.
  - rubidium 5y ago
    Yep. And that’s on you and the product owner to get aligned on. Fix that alignment and you fix the problem.
    - lostcolony 5y ago
      Time needed to fix that alignment is impossible to estimate, and also impossible to prove you have done sufficiently.

ochrist 5y ago

Did you use three point estimation: https://en.wikipedia.org/wiki/Three-point_estimation https://en.wikipedia.org/wiki/Three-point_estimation

rahoulb 5y ago
I didn't know about that - but I suspect (it was a while back) that I did two points - best and worst case.

xondono 5y ago

> But the interesting bit was each individual task was inaccurate - lots were wildly underestimated, balanced by those which were wildly overestimated. This is called random luck. Something tells me that if we repeated the same experiment with different projects, that distribution of errors wouldn’t cancel out most of the time.

jmchuster 5y ago
Not all the time, but most of the time, yes it cancels out. If you work in a business like consulting, you'll quickly learn that you make more money by being able to give estimates in this manner. And when it's part of your job, it becomes a skill that you need to train and become better at, and your estimates get more consistent and more accurate.
- xondono 5y ago
  Yeah, I’ve done it myself. I’ve given price estimates for complex PCBAs before designing them within 0.5%, but I don’t attribute that precision to talent, I probably get the estimates in the +/-10% and got lucky in that case.
  - jmchuster 5y ago
    Ah, sorry, I misread your original statement then. Were you trying to say that getting an estimate right within 2 days was lucky, but something like 1 week would've been totally normal? It's hard to tell the percentage, since OP just said "large project".

codingdave 5y ago

Back in the first dotcom boom, we started tracking our rough swags that a dev would throw out on gut instinct vs. the detailed estimates we put together for actual customer projects. We founds the swag were more accurate, exactly for the reasons you stated -- we would over and under estimate details, but they balanced out and the scope of the overall project was correct. We then got into talks with other dev shops, with more experience than our young group... and they had the same result. Young or old, experienced or not, a rough swipe at the scale of a project is usually accurate enough to know what it will take. I've seen the same thing play out in Agile processes - we are asked for story points, and while many are accurate, some are way bigger than we thought, some smaller. But the overall time to complete a project tends to fall right about where the devs thought it would before the pointing exercises. All the estimation in the world will still put the same answers into the hands of your project managers - "Pick a date, or pick your scope. Not both."

deleted 5y ago
[deleted]

dboreham 5y ago

This all depends on being able to make that detailed task breakdown at the _beginning_ of the project. In my experience software development is an endeavor where that level of understanding is seldom present at the outset. Typically, at least in the fields I've worked, at the beginning you may have answered the question "is this even possible"? (but not always), but your task breakdown consists of a small set of "figure out how to approach X" tasks, and a scattering of "well-understood thing we know we need to do sometime". In this context, it's impossible to estimate the project. The best you can do is to plan roughly what work to do in the next few days/weeks, then re-group to see what comes next. I believe Agile incorporates some of these insights.

denton-scratch 5y ago

Years ago, I was given training in an estimating technique called Function Point Analysis (yes, yet another estimating technique). You broke the task down into (I think) five elementary components: inputs, outputs, processes, inter-process exchanges, something. You rated each component out of 5 for difficulty. Each type of component has a weight, so you can get a sum of the weighted scores of the components, apply some kind of fiddle-function to the sum, and that's your function-point count. You can then use intrusive management to measure the function-point output of each developer per day, so that after some months, you can make accurate estimates. Because the task estimate is denominated in function-points, not days, you can use the FPs-per-day rates of individuals or teams to estimate the elapsed time for a task. I tried to use it a few times. What I learned is that it's not possible to make accurate estimates. Actually, that was one of the observations of the FPA training: that you are supposed to keep on doing it all through a project. Well, you really needed a whole corporation to commit to supporting something like that. I'm not surprised it was no use to me.