6 ms·

There is actually a nugget of thought in the article beyond the explanations you always hear (and which make up all the comments in this thread as I'm writing t

There is actually a nugget of thought in the article beyond the explanations you always hear (and which make up all the comments in this thread as I'm writing this): Even if you base your project schedules on accurate estimates of the median time required for each step, you will still be behind schedule for most projects. The reason is actually quite simple: when things go well, they can only go so well, but when they go bad, they can go really bad. Or, in a more quantitative way: while each step of the project will be equally likely to take longer or shorter than median, the steps that take longer can take much longer, while the steps that go faster only have a limited potential for balancing out the delays.

mason55 11y ago
Projects timelines are like naked shorts in the stock market. You have a capped upside but unlimited downside - things can never be done faster than instantly but they can take literally forever (i.e. just never get finished).
- mianos 11y ago
  You are being a bit loose with the term 'unlimited' but I guess that's marketing for you. You stand to lose the difference between your strike and zero. Last time I looked that's not 'unlimited'.
  - tbrownaw 11y ago
    I thought 'strike' was an options thing rather than a short thing?
  - mason55 11y ago
    I'm not really sure what you mean. If you naked short without covering with options then the price can pretty much go up forever and you cover the difference. I don't understand where zero comes into play.
    - mianos 11y ago
      Sorry, my mistake. I was thinking the option on the back of it. Yes, you are correct, for simple stocks, if the stock price rises forever you theoretically have the possibility of infinite losses. Practically I'd not use infinite, but that has busted many people.
mikekchar 11y ago
This is very true. The best way I have found to combat it is by splitting up the tasks into many smaller pieces. If you have a task that is scheduled to take a month, then it can easily go out to 10 months in the worst case. If you have a task that is scheduled to take a day, it can easily go out to 10 days in the worst case. This allows you to make corrections to your schedule. There are 2 other important issues. First, you need to update your schedule based on the amount of time tasks are actually taking. If you only have 10 tasks (each taking a month), the error bars can be quite significant. But if you have 200 tasks (each taking a day), the mean completion time will have quite small error bars. So if you keep a rolling average (say over the last 30 tasks), you can have a fairly good estimate for completion time (agile developers will notice that this is "velocity"). The second very important issue is to be completely anal about your definition of "done" and about making sure that the completed tasks actually meet that definition. Your mean completion time will only be useful if you are measuring the time to completion accurately (obviously). This effect is so powerful that I recommend "same sizing" tasks and planning everything to have a completion time of somewhere between 1 and 2 days. There is actually one last thing you need to do. Requirements discovery never happens completely before you start development. As you write the code, you discover new things to do. It is obviously important to modify your plan to accommodate that new information. If you don't, you will end up building something that nobody needs. However, the amount of new work seems to be predictable. I made graphs of new work added to projects over a couple of years and it appears that the growth of new requirements is very similar to some of the defect discovery models (for example Littlewood). Just making a graph of new stories/tasks added over time will give you a decent idea, but I have found that a rule of thumb of adding 30% or so (over the whole project) for new requirements seems to work well.
- caseysoftware 11y ago
  There's another hidden aspect: Be aware of dependencies. When you have a series of dependencies (A -> B -> C) stacked against each other, once A is late, B and C are almost guaranteed to be late. If B is late too, C suffers even more and has little chance of being on time. If you can lay out the tasks so that dependencies have slack between them so that lateness can be absorbed without shifting later tasks, you are more likely chance to hit deadlines and potentially complete the project on time. Or in more formal terms, track the Critical Path. In practical terms, I would see this all the time commuting home in DC. I walked to the train to the bus to home. If I timed it perfectly, my commute was ~25 minutes. But if I couldn't cross the street in time, I'd catch a later train which made for a later bus. Or a slightly later train would be a much later bus. When those variances stacked up, it could take an hour.
Dwolb 11y ago
Ah I like this explanation a lot. It's almost as if project task timing distributions are negatively skewed.
adolgert 11y ago
The article suggests that failures in an n-step process are distributed as a Binomial distribution, which doesn't have the kind of long tail you describe. I hazard that you're right and they're imprecise. It would be great if they measured the shape of the distribution of completion times, scaling for project length, and then suggested a theory behind it.
dilap 11y ago
Yup. If you look at the factor your estimates are off, they'll follow a lognormal distribution. * And it's natural, if you think about it, that any schedule which doesn't include slack between tasks will almost certainly run long. (Even assuming you've got a mean of 0 -- as many tasks go short as go long -- you're not going to get lucky enough that the tasks that go short do it 'just right' to make up for the tasks that go long.) So maybe include some slack. Still, maybe you want to know how often will your schedule run long. So do some measurements of historical data to find out how your task estimates work out in practice. Plug those in and do a few simulations, and viola, you can get a precise number of how you're likely to do. Plus handy tool to evaluate schedule quality.
ape4 11y ago
I'll take me 2 weeks to use this API to make this application. If I work diligently it'll be done on time. But....oh no, the API can't do everything I need. Now its going to take 6 months.
astazangasta 11y ago
According to this reasoning you should set targets based on expected time, not "median" time. You're saying the tail of "completion time" is long, and so the median is a poor estimator for expected time. But if you actually knew the distribution of times for task completion (which is implied if you know the 'median time'), you could simply use the expected value of this distribution and your projects would then tend to complete on time. I don't think this is the correct explanation. I think it's far more likely that (1) people don't know the true distribution of task times, so estimates are just crap guesses based on hubris, what managers want to hear, etc., and (2) scope creep.
- 21echoes 11y ago
  Exactly. All this article is really saying is "most of the time, adding together median expected times underestimates the sum of mean expected times". Well then, easy fix: add together the mean expected times off the bat.
  - calinet6 11y ago
    Bingo. When we think about the steps intuitively, we tend to gravitate toward the median. "How long does this usually take?" is asking about a median in surprisingly precise terms. Instead we have to understand the distribution, and the mean as well, and add up how long it takes on average, rather than just in the most numerous case. Further, and perhaps more importantly, by understanding each distribution, you have far greater knowledge about what the expected time will be, and where the delays are most probable; and you can begin to start analyzing where those delays might be coming from in the interactions of all the parts of the system. This is straight-up quality talk right out of W. Edwards Deming, and it's how you start to improve how you produce in general. Good stuff.
  - gkop 11y ago
    There's more to it than your easy fix, the article alludes to it but does a poor job of covering it: if you know the distributions of the intermediate steps, but you only sum the means of the steps, then you are throwing away data that could help you calculate a better estimate.
kirushik 11y ago
In addition to that, project's stages ahead-of-schedule deliveries are typically wasted, but behind-of-schedule deliveries are accumulated. As an example: Arriving to the airport one hour in advance doesn't allow you to actually board a flight an hour earlier. But get to the airport one hour late — and you're flying tomorrow (or whenever the next flight is scheduled).
noiv 11y ago
> they can only go so well, but when they go bad, they can go really bad. Well, they can, but usually the acceptable explanation for a project going really well is it actually was less work than estimated and the consequence is fewer resources next time. So, there is virtually no motivation left to drive a project into that area.
zzalpha 11y ago
So it's like shorting a stock... Limited upside, unlimited downside. Interesting observation, I'd never thought of project planning that way.

mason55 11y ago

Projects timelines are like naked shorts in the stock market. You have a capped upside but unlimited downside - things can never be done faster than instantly but they can take literally forever (i.e. just never get finished).

mianos 11y ago
You are being a bit loose with the term 'unlimited' but I guess that's marketing for you. You stand to lose the difference between your strike and zero. Last time I looked that's not 'unlimited'.
- tbrownaw 11y ago
  I thought 'strike' was an options thing rather than a short thing?
- mason55 11y ago
  I'm not really sure what you mean. If you naked short without covering with options then the price can pretty much go up forever and you cover the difference. I don't understand where zero comes into play.
  - mianos 11y ago
    Sorry, my mistake. I was thinking the option on the back of it. Yes, you are correct, for simple stocks, if the stock price rises forever you theoretically have the possibility of infinite losses. Practically I'd not use infinite, but that has busted many people.

mikekchar 11y ago

This is very true. The best way I have found to combat it is by splitting up the tasks into many smaller pieces. If you have a task that is scheduled to take a month, then it can easily go out to 10 months in the worst case. If you have a task that is scheduled to take a day, it can easily go out to 10 days in the worst case. This allows you to make corrections to your schedule. There are 2 other important issues. First, you need to update your schedule based on the amount of time tasks are actually taking. If you only have 10 tasks (each taking a month), the error bars can be quite significant. But if you have 200 tasks (each taking a day), the mean completion time will have quite small error bars. So if you keep a rolling average (say over the last 30 tasks), you can have a fairly good estimate for completion time (agile developers will notice that this is "velocity"). The second very important issue is to be completely anal about your definition of "done" and about making sure that the completed tasks actually meet that definition. Your mean completion time will only be useful if you are measuring the time to completion accurately (obviously). This effect is so powerful that I recommend "same sizing" tasks and planning everything to have a completion time of somewhere between 1 and 2 days. There is actually one last thing you need to do. Requirements discovery never happens completely before you start development. As you write the code, you discover new things to do. It is obviously important to modify your plan to accommodate that new information. If you don't, you will end up building something that nobody needs. However, the amount of new work seems to be predictable. I made graphs of new work added to projects over a couple of years and it appears that the growth of new requirements is very similar to some of the defect discovery models (for example Littlewood). Just making a graph of new stories/tasks added over time will give you a decent idea, but I have found that a rule of thumb of adding 30% or so (over the whole project) for new requirements seems to work well.

caseysoftware 11y ago
There's another hidden aspect: Be aware of dependencies. When you have a series of dependencies (A -> B -> C) stacked against each other, once A is late, B and C are almost guaranteed to be late. If B is late too, C suffers even more and has little chance of being on time. If you can lay out the tasks so that dependencies have slack between them so that lateness can be absorbed without shifting later tasks, you are more likely chance to hit deadlines and potentially complete the project on time. Or in more formal terms, track the Critical Path. In practical terms, I would see this all the time commuting home in DC. I walked to the train to the bus to home. If I timed it perfectly, my commute was ~25 minutes. But if I couldn't cross the street in time, I'd catch a later train which made for a later bus. Or a slightly later train would be a much later bus. When those variances stacked up, it could take an hour.

Dwolb 11y ago

Ah I like this explanation a lot. It's almost as if project task timing distributions are negatively skewed.

adolgert 11y ago

The article suggests that failures in an n-step process are distributed as a Binomial distribution, which doesn't have the kind of long tail you describe. I hazard that you're right and they're imprecise. It would be great if they measured the shape of the distribution of completion times, scaling for project length, and then suggested a theory behind it.

dilap 11y ago

Yup. If you look at the factor your estimates are off, they'll follow a lognormal distribution. * And it's natural, if you think about it, that any schedule which doesn't include slack between tasks will almost certainly run long. (Even assuming you've got a mean of 0 -- as many tasks go short as go long -- you're not going to get lucky enough that the tasks that go short do it 'just right' to make up for the tasks that go long.) So maybe include some slack. Still, maybe you want to know how often will your schedule run long. So do some measurements of historical data to find out how your task estimates work out in practice. Plug those in and do a few simulations, and viola, you can get a precise number of how you're likely to do. Plus handy tool to evaluate schedule quality.

ape4 11y ago

I'll take me 2 weeks to use this API to make this application. If I work diligently it'll be done on time. But....oh no, the API can't do everything I need. Now its going to take 6 months.

astazangasta 11y ago

According to this reasoning you should set targets based on expected time, not "median" time. You're saying the tail of "completion time" is long, and so the median is a poor estimator for expected time. But if you actually knew the distribution of times for task completion (which is implied if you know the 'median time'), you could simply use the expected value of this distribution and your projects would then tend to complete on time. I don't think this is the correct explanation. I think it's far more likely that (1) people don't know the true distribution of task times, so estimates are just crap guesses based on hubris, what managers want to hear, etc., and (2) scope creep.

21echoes 11y ago
Exactly. All this article is really saying is "most of the time, adding together median expected times underestimates the sum of mean expected times". Well then, easy fix: add together the mean expected times off the bat.
- calinet6 11y ago
  Bingo. When we think about the steps intuitively, we tend to gravitate toward the median. "How long does this usually take?" is asking about a median in surprisingly precise terms. Instead we have to understand the distribution, and the mean as well, and add up how long it takes on average, rather than just in the most numerous case. Further, and perhaps more importantly, by understanding each distribution, you have far greater knowledge about what the expected time will be, and where the delays are most probable; and you can begin to start analyzing where those delays might be coming from in the interactions of all the parts of the system. This is straight-up quality talk right out of W. Edwards Deming, and it's how you start to improve how you produce in general. Good stuff.
- gkop 11y ago
  There's more to it than your easy fix, the article alludes to it but does a poor job of covering it: if you know the distributions of the intermediate steps, but you only sum the means of the steps, then you are throwing away data that could help you calculate a better estimate.

kirushik 11y ago

In addition to that, project's stages ahead-of-schedule deliveries are typically wasted, but behind-of-schedule deliveries are accumulated. As an example: Arriving to the airport one hour in advance doesn't allow you to actually board a flight an hour earlier. But get to the airport one hour late — and you're flying tomorrow (or whenever the next flight is scheduled).

noiv 11y ago

> they can only go so well, but when they go bad, they can go really bad. Well, they can, but usually the acceptable explanation for a project going really well is it actually was less work than estimated and the consequence is fewer resources next time. So, there is virtually no motivation left to drive a project into that area.

zzalpha 11y ago

So it's like shorting a stock... Limited upside, unlimited downside. Interesting observation, I'd never thought of project planning that way.