4 ms·

And one of the classic problems with estimates is that common processes tend to bias estimates. For example, what (I think) McConnell calls "picking the first

And one of the classic problems with estimates is that common processes tend to bias estimates. For example, what (I think) McConnell calls "picking the first non-impossible date". An exec asks how long project Magic Pony will take. A manager asks the team and the team comes up with a number. If the number is higher than what the exec imagined, or even higher than what the manager imagines the exec imagines, pressure is applied on the number. "Are you sure? That seems like a lot." A negotiation ensues, and often the number lands on the first date that engineers can't absolutely prove is impossible. Or the first date that engineers won't quit their jobs over. For an estimate, we in theory want one where the team has a good chance of hitting it. Say, a 75% chance of success. But iterative pressure from the powerful will shift the distribution to 25%, 5%, 1%.

quacked 5y ago
The classic cognitive trick--people, unless specifically trained not to, will always estimate the best possible scenario, even if past experience has told them that the best possible scenario never happens. I'm dealing with this at my company right now. Our clients, and even our internal teams, will always refuse to estimate according to our past behavior. Us: "We estimate your build will be functional in prod in five months." Client: "Unacceptable. That's way too long. We need it in three." Us: "Okay, we technically can do that, but that relies on a big internal release that we think is risky to base estimates on." Client: "All I'm hearing is that you can do it in three." Then we get to three months and hit delays over and over again until finally, five months later, the build is functional in prod. The thing that really frustrates me is that when we deliver the initial estimate, the clients always refuse to accept our medium-case estimate, with the implication that they'll drop us as a vendor if we follow that. So we give them our best-case estimate, fail to hit it, and then they buy more from us anyway, complaining the entire time. Why don't we all just look at the past 10 builds we delivered for them, admit that the best-case never happens, and stop pretending like it's unacceptable to take longer? The point here is that they keep accepting it. Everyone complains, but we all keep selling and buying to and from each other. Let's just admit it's harder than we're saying to get software built and accept longer estimates, given that we're all going to keep selling and buying anyway.
- wpietri 5y ago
  Totally. In most places what people call "estimates" are just wishful thinking. And that client behavior you describe is what I think of as "Kirk-style management". An executive thinks that the way to get technical things done is to be demanding and shouty toward the technical person, insisting that the number given isn't good enough. Scotty's pathological response to the pathological situation was to lie his ass off about estimates: https://wiki.c2.com/?ScottyFactor https://wiki.c2.com/?ScottyFactor
  - quacked 5y ago
    Thanks for that article--bookmarked.
- mgkimsal 5y ago
  > Why don't we all just look at the past 10 builds we delivered for them, admit that the best-case never happens, and stop pretending like it's unacceptable to take longer? The point here is that they keep accepting it. It's theater, mostly. I think some people think they have to do this. How did they get you to keep delivering when you did? It was their threats/complaining! If they stopped doing that... there's no telling how long it might take you to deliver. Given that every project you've called at 4 months has delivered in ... 4 months, for the past 7 years... your estimating/delivery ability is still tangential to their managing of the delivery by regular barrages of threats, complaints and haranguing.
  - wpietri 5y ago
    Wow, great insight. It reminds me of the dynamics with walking down the sidewalk past a dog. If I walk too close to his yard, he decides I'm a threat. He barks! I walk away. It's easy for him to do post hoc ergo propter hoc and conclude that his barking chased away a dangerous threat. In reality I didn't do a thing different, but his lesson is that barking works!
  - quacked 5y ago
    It's a great point, and one that could be rectified by their teams actually having standards. Threats can't be empty. "Deliver early, we'll give you a bonus. Deliver on-time, we'll pay. Deliver late, we'll hold you accountable and pay less according to this attachment. Deliver really late, and you get a strike. Three strikes and we drop you." Of course, that would require backbone, fortitude, accurate forecasting, integrity, and management support.
RegW 5y ago
One technique is to ask for 3 point estimates. Get everyone involved to estimate each item in three different ways: 1. How long it will take if everything goes well and according to plan. 2. How long it will take if everything goes wrong and we uncover further problems along the way. Probably getting those risks listed. 3. How long it is likely to take. I imagine we tend towards doing just one of the first two, but having to do both should lead to more agreement on the third. That said - its been about 30 years since I was taught this as part of critical path analysis, and I've never seen it used in the wild.
- wpietri 5y ago
  For sure. I know plenty of ways to deliver solid estimates and hit dates. I think the problem is execs mostly prefer having estimates they like over accurate estimates.
- regularfry 5y ago
  The problem even with that method is that if everyone agrees on "most likely", and that is perfectly correct, and you use it, you'll still be wrong more often than right. The reason is the underlying distribution: because the probability distribution is usually heavily skewed to the left, with a long tail to the right and lots more room for things to extend than to contract, the day which is "most likely" when looked at in isolation is actually well to the left of the median date, the date with a 50% probability of being before or after completion. If you want an 80% chance of being right, say, you need to do some more maths with the three numbers to figure out what the right date to use is. I would very much like to see this done. The problem is that estimation isn't seen as work. It's seen (or, more accurately, framed) as something devs should be able to just magic from thin air with perfect accuracy, so why would we bother adding complexity? Very few organisations bother tracking estimation precision, I think because the incentives are aligned such that it's actually to the advantage of the people writing the cheques that developers under-estimate. I think the power over the developers that a missed estimate gives to an adjacent organisation is a real factor in ensuring this never gets improved.
  - drdec 5y ago
denton-scratch 5y ago
> A negotiation ensues ... and then the estimate is no longer an estimate, it's the outcome of a negotiation. > Are you sure? That seems like a lot. "No, I'm not sure, boss; it's an estimate."

quacked 5y ago

The classic cognitive trick--people, unless specifically trained not to, will always estimate the best possible scenario, even if past experience has told them that the best possible scenario never happens. I'm dealing with this at my company right now. Our clients, and even our internal teams, will always refuse to estimate according to our past behavior. Us: "We estimate your build will be functional in prod in five months." Client: "Unacceptable. That's way too long. We need it in three." Us: "Okay, we technically can do that, but that relies on a big internal release that we think is risky to base estimates on." Client: "All I'm hearing is that you can do it in three." Then we get to three months and hit delays over and over again until finally, five months later, the build is functional in prod. The thing that really frustrates me is that when we deliver the initial estimate, the clients always refuse to accept our medium-case estimate, with the implication that they'll drop us as a vendor if we follow that. So we give them our best-case estimate, fail to hit it, and then they buy more from us anyway, complaining the entire time. Why don't we all just look at the past 10 builds we delivered for them, admit that the best-case never happens, and stop pretending like it's unacceptable to take longer? The point here is that they keep accepting it. Everyone complains, but we all keep selling and buying to and from each other. Let's just admit it's harder than we're saying to get software built and accept longer estimates, given that we're all going to keep selling and buying anyway.

wpietri 5y ago
Totally. In most places what people call "estimates" are just wishful thinking. And that client behavior you describe is what I think of as "Kirk-style management". An executive thinks that the way to get technical things done is to be demanding and shouty toward the technical person, insisting that the number given isn't good enough. Scotty's pathological response to the pathological situation was to lie his ass off about estimates: https://wiki.c2.com/?ScottyFactor https://wiki.c2.com/?ScottyFactor
- quacked 5y ago
  Thanks for that article--bookmarked.
mgkimsal 5y ago
> Why don't we all just look at the past 10 builds we delivered for them, admit that the best-case never happens, and stop pretending like it's unacceptable to take longer? The point here is that they keep accepting it. It's theater, mostly. I think some people think they have to do this. How did they get you to keep delivering when you did? It was their threats/complaining! If they stopped doing that... there's no telling how long it might take you to deliver. Given that every project you've called at 4 months has delivered in ... 4 months, for the past 7 years... your estimating/delivery ability is still tangential to their managing of the delivery by regular barrages of threats, complaints and haranguing.
- wpietri 5y ago
  Wow, great insight. It reminds me of the dynamics with walking down the sidewalk past a dog. If I walk too close to his yard, he decides I'm a threat. He barks! I walk away. It's easy for him to do post hoc ergo propter hoc and conclude that his barking chased away a dangerous threat. In reality I didn't do a thing different, but his lesson is that barking works!
- quacked 5y ago
  It's a great point, and one that could be rectified by their teams actually having standards. Threats can't be empty. "Deliver early, we'll give you a bonus. Deliver on-time, we'll pay. Deliver late, we'll hold you accountable and pay less according to this attachment. Deliver really late, and you get a strike. Three strikes and we drop you." Of course, that would require backbone, fortitude, accurate forecasting, integrity, and management support.

RegW 5y ago

One technique is to ask for 3 point estimates. Get everyone involved to estimate each item in three different ways: 1. How long it will take if everything goes well and according to plan. 2. How long it will take if everything goes wrong and we uncover further problems along the way. Probably getting those risks listed. 3. How long it is likely to take. I imagine we tend towards doing just one of the first two, but having to do both should lead to more agreement on the third. That said - its been about 30 years since I was taught this as part of critical path analysis, and I've never seen it used in the wild.

wpietri 5y ago
For sure. I know plenty of ways to deliver solid estimates and hit dates. I think the problem is execs mostly prefer having estimates they like over accurate estimates.
regularfry 5y ago
The problem even with that method is that if everyone agrees on "most likely", and that is perfectly correct, and you use it, you'll still be wrong more often than right. The reason is the underlying distribution: because the probability distribution is usually heavily skewed to the left, with a long tail to the right and lots more room for things to extend than to contract, the day which is "most likely" when looked at in isolation is actually well to the left of the median date, the date with a 50% probability of being before or after completion. If you want an 80% chance of being right, say, you need to do some more maths with the three numbers to figure out what the right date to use is. I would very much like to see this done. The problem is that estimation isn't seen as work. It's seen (or, more accurately, framed) as something devs should be able to just magic from thin air with perfect accuracy, so why would we bother adding complexity? Very few organisations bother tracking estimation precision, I think because the incentives are aligned such that it's actually to the advantage of the people writing the cheques that developers under-estimate. I think the power over the developers that a missed estimate gives to an adjacent organisation is a real factor in ensuring this never gets improved.
- drdec 5y ago

denton-scratch 5y ago

> A negotiation ensues ... and then the estimate is no longer an estimate, it's the outcome of a negotiation. > Are you sure? That seems like a lot. "No, I'm not sure, boss; it's an estimate."