7 ms·

Even bad estimates are valuable if you use them right

71 points by ntietz 8y ago

awinter-py 8y ago
> If you never use the estimates to adjust what you are working on, then putting in the time to do estimates is a pure waste yes! using plans to create context for the larger project is the critical missing piece. Business stakeholders just want delivery dates but the team doing the work benefits when the estimates fit into an organic, flexible master plan that is changed when things are late or hard.
sureaboutthis 8y ago
Just a few days ago, one of the restaurants I own was getting a new air conditioner installed as a condition of lease renewal. Of course, the landlord did not tell me the AC people were coming to do the work and just showed up at 7AM with the expected high temperature to be in the mid-90sF. It was already 78F when they started. I asked my manager to ask the AC guy how long it would take. "He doesn't know", she said. Such an answer drives me insane. You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take? But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong. At the three hour mark, I was told it would be about an hour. After 5 1/2 hours, I was told half an hour. It took 6 1/2 hours to complete. Here are the issues with all that: 1) If I had an estimate, I wouldn't have needed to drive to the restaurant and talk to the installers myself. 2) If I didn't get an estimate, I would have had to raise hell with the landlord who I have a good relationship with. 3) Once I had an estimate, I didn't feel the anxiety or need to constantly check on the progress which would have irritated the installers. And believe me, I would have irritated them!
- arethuza 8y ago
  Unfortunately, a lot of people treat "estimate" as "a binding commitment written in blood" - so I sometimes understand when people are reluctant to give such a thing.
  - jonawesomegreen 8y ago
    So much this. Whenever I'm trying to pull an estimate out of someone that is reluctant I always try to reassure them that I know its an estimate, I know it might be wrong, I know conditions can change but its still helpful to have a guess.
- frgtpsswrdlame 8y ago
  >You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take? >But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong. If he had told you it would take between 1 hour and 10 hours would you have been happy with that answer?
- da02 8y ago
  What food do you serve at the restaurant? Does owning a restaurant ever lead to suicidal thoughts? Did you inherit it?
- binmanthrowaway 8y ago
  > the landlord did not tell me > the landlord who I have a good relationship with No, you don't have a good relationship.
billmalarky 8y ago
> you must explain to your stakeholders beforehand that these estimates are only for course correction during the development process, and they’re separate from estimates you will give of when features will be done overall. ... > if you do put in the time to do estimates, you will spend less time coding - but because your team will be able to respond to things immediately, you will still reach your objective more quickly. This sums up the best argument I've read about the advantages of best practices agile software development. It's a shame some companies cargo cult agile as a way to bully engineers into being "more productive" in a brute force manner. That is, unrealistically cramming tickets into a sprint to gamify higher "velocity." Really agile is just about a more efficient development process. Agile provides a framework for discovery and course correction via planning/imagination/whiteboarding, and that is significantly faster than discovery via writing code.
- ntietz 8y ago
  Yes, exactly! It's such a shame that people think "more efficient process" means "we'll ship more lines of code" when really it is all about getting to the _correct solution_ more quickly.
  - AstralStorm 8y ago
    Presuming you know what the correct solution is. This is the key point of agile processes, that you get to quickly discover what is required, optional and completely unnecessary. Generally by having a real customer (not a silly proxy) in the room or by sending out probe releases often and collecting feedback.
  - _0ffh 8y ago
    Of course, "more lines of code" is about the worst possible metric for sw development success imaginable, anyway.
- AstralStorm 8y ago
  Why stakeholders decide about anything related to your development practices? Is it because... A) your hiring people like well known keywords B) someone is beholden to a bureaucratic process C) someone lied and broke trust? If you cannot deliver, just say so. If there are insurmountable barriers, as well. Why provide a bad guess to which you will be held, and this cause people to make wrong decisions based on the guess? Estimates are only good when you have strong basis for them. Software developer cannot in general have such a basis unless they're copying or adapting in trivial ways already made software. A civil engineer, on the other hand, can use a reference group strategy easily to compute cost and materials required, but not necessarily timescale. They can provide at best a reasonable, very wide range.
matheweis 8y ago
This was one of the very few valuable things I learned working towards a bachelors degree. One of my professors popped a quiz on the class with the question “how many piano tuners live in the state of New York”. After collecting the answers, he spent the rest of the session lecturing (especially those like me who left it blank) on how to make estimates without having any firsthand knowledge in the problem space, and how valuable that skill could be.
- AstralStorm 8y ago
  And none of the methods work in software development.
tsunamifury 8y ago
One caveat: the relative level of definition of work has a huge effect on the work. For example, well trodden engineering implementations can have rational top and bottom end estimates. A UX design or Eng implementation of a "never been done before" or strategically-shifting problem is far far harder to estimate.
onewhonknocks 8y ago
Akin's 10th law of spacecraft design: "When in doubt, estimate. In an emergency, guess. But be sure to go back and clean up the mess when the real numbers come along."
AstralStorm 8y ago
A team cannot reduce scope without involving the customer. And if you have the misfortune that you're dealing with someone who thinks difficulty is negotiable, you're in a real pickle. You're missing the critical point - measure the estimation error and you'll see how hopeless almost all estimates are. In practice, they do not improve.
chiefalchemist 8y ago
The book The Influential Mind does a chapter on (essentially) tips & tricks for estimates (of all kinds). Long to short, be careful of group think. It's best to collect the estimates independently, then aggregate. Else the group members influence each other and the advantage of "crowd sourcing" goes out the window. The book is worth a go. It reads quick. A bit light- weight but still useful. https://upliftconnect.com/tali-sharot-influential-mind/ https://upliftconnect.com/tali-sharot-influential-mind/
jacquesm 8y ago
The biggest value of bad estimates imo are that they teach you to get better at estimating. You should record your estimates and the actual time it took. That way the next time you need to make an estimate you have some more data to plug into your formula for making a guess: the previous slippage. So you improve over time and your post-mortem of why something slipped will help you to get your initial guesses closer to where they should have been in the first place. Between those two anybody can learn how to estimate with some reasonable degree of fidelity.
quantumhobbit 8y ago
There are just too many unknowns to do estimates well in software. And getting things wrong in estimates isn't a matter of being off by a couple percentage points but being off in by orders of magnitude. It isn't unusual for what appears to be a simple unit of work that seems like it should take a few hours to explode into days or weeks of work. And it does so in a way that is completely opaque to stakeholders. Partly this is because of how we build software as layers of abstraction on top of other layers of abstraction. When you are at the top layer things are relatively quick and easy to estimate, but once you have to work at a lower level than normal all bets are off. And it is really hard to know beforehand if you will need to dig into the lower levels. For example at my job I am updating an app that queries an api to filter its requests by some parameter. When the api supports the desired filter that is great. When the api doesn't, I have to dig into unfamiliar code for the api to support it. But turns out the api is just forwarding its request to another api, so I have to dig into yet another api to build the filter. And so on. The rabbit hole can get deep. And of course there isn't any documentation to help me know about this beforehand. The article is right that the key is frequent updates and communication as soon as "blockers" like this are discovered. But it can be very difficult to communicate this to stakeholders who are uninterested in the technical details and are inflexible in scope. Trust is key, so that they believe you when you say that this task which normally takes an hour or so will now take several days.

71 points by ntietz 8y ago

> If you never use the estimates to adjust what you are working on, then putting in the time to do estimates is a pure waste yes! using plans to create context for the larger project is the critical missing piece. Business stakeholders just want delivery dates but the team doing the work benefits when the estimates fit into an organic, flexible master plan that is changed when things are late or hard.

sureaboutthis 8y ago

Just a few days ago, one of the restaurants I own was getting a new air conditioner installed as a condition of lease renewal. Of course, the landlord did not tell me the AC people were coming to do the work and just showed up at 7AM with the expected high temperature to be in the mid-90sF. It was already 78F when they started. I asked my manager to ask the AC guy how long it would take. "He doesn't know", she said. Such an answer drives me insane. You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take? But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong. At the three hour mark, I was told it would be about an hour. After 5 1/2 hours, I was told half an hour. It took 6 1/2 hours to complete. Here are the issues with all that: 1) If I had an estimate, I wouldn't have needed to drive to the restaurant and talk to the installers myself. 2) If I didn't get an estimate, I would have had to raise hell with the landlord who I have a good relationship with. 3) Once I had an estimate, I didn't feel the anxiety or need to constantly check on the progress which would have irritated the installers. And believe me, I would have irritated them!

arethuza 8y ago
Unfortunately, a lot of people treat "estimate" as "a binding commitment written in blood" - so I sometimes understand when people are reluctant to give such a thing.
- jonawesomegreen 8y ago
  So much this. Whenever I'm trying to pull an estimate out of someone that is reluctant I always try to reassure them that I know its an estimate, I know it might be wrong, I know conditions can change but its still helpful to have a guess.
frgtpsswrdlame 8y ago
>You mean he has no clue? Will it take three months?! 30 minutes?! How can we plan to work around no AC in a restaurant when the installers have no idea how long it will take? >But of course they had a rough estimate. Why didn't they give me one? It would relieve a lot of anxiety if nothing else even if it was wrong. If he had told you it would take between 1 hour and 10 hours would you have been happy with that answer?
da02 8y ago
What food do you serve at the restaurant? Does owning a restaurant ever lead to suicidal thoughts? Did you inherit it?
binmanthrowaway 8y ago
> the landlord did not tell me > the landlord who I have a good relationship with No, you don't have a good relationship.

billmalarky 8y ago

> you must explain to your stakeholders beforehand that these estimates are only for course correction during the development process, and they’re separate from estimates you will give of when features will be done overall. ... > if you do put in the time to do estimates, you will spend less time coding - but because your team will be able to respond to things immediately, you will still reach your objective more quickly. This sums up the best argument I've read about the advantages of best practices agile software development. It's a shame some companies cargo cult agile as a way to bully engineers into being "more productive" in a brute force manner. That is, unrealistically cramming tickets into a sprint to gamify higher "velocity." Really agile is just about a more efficient development process. Agile provides a framework for discovery and course correction via planning/imagination/whiteboarding, and that is significantly faster than discovery via writing code.

ntietz 8y ago
Yes, exactly! It's such a shame that people think "more efficient process" means "we'll ship more lines of code" when really it is all about getting to the _correct solution_ more quickly.
- AstralStorm 8y ago
  Presuming you know what the correct solution is. This is the key point of agile processes, that you get to quickly discover what is required, optional and completely unnecessary. Generally by having a real customer (not a silly proxy) in the room or by sending out probe releases often and collecting feedback.
- _0ffh 8y ago
  Of course, "more lines of code" is about the worst possible metric for sw development success imaginable, anyway.
AstralStorm 8y ago
Why stakeholders decide about anything related to your development practices? Is it because... A) your hiring people like well known keywords B) someone is beholden to a bureaucratic process C) someone lied and broke trust? If you cannot deliver, just say so. If there are insurmountable barriers, as well. Why provide a bad guess to which you will be held, and this cause people to make wrong decisions based on the guess? Estimates are only good when you have strong basis for them. Software developer cannot in general have such a basis unless they're copying or adapting in trivial ways already made software. A civil engineer, on the other hand, can use a reference group strategy easily to compute cost and materials required, but not necessarily timescale. They can provide at best a reasonable, very wide range.

matheweis 8y ago

This was one of the very few valuable things I learned working towards a bachelors degree. One of my professors popped a quiz on the class with the question “how many piano tuners live in the state of New York”. After collecting the answers, he spent the rest of the session lecturing (especially those like me who left it blank) on how to make estimates without having any firsthand knowledge in the problem space, and how valuable that skill could be.

AstralStorm 8y ago
And none of the methods work in software development.

tsunamifury 8y ago

One caveat: the relative level of definition of work has a huge effect on the work. For example, well trodden engineering implementations can have rational top and bottom end estimates. A UX design or Eng implementation of a "never been done before" or strategically-shifting problem is far far harder to estimate.

onewhonknocks 8y ago

Akin's 10th law of spacecraft design: "When in doubt, estimate. In an emergency, guess. But be sure to go back and clean up the mess when the real numbers come along."

AstralStorm 8y ago

A team cannot reduce scope without involving the customer. And if you have the misfortune that you're dealing with someone who thinks difficulty is negotiable, you're in a real pickle. You're missing the critical point - measure the estimation error and you'll see how hopeless almost all estimates are. In practice, they do not improve.

chiefalchemist 8y ago

The book The Influential Mind does a chapter on (essentially) tips & tricks for estimates (of all kinds). Long to short, be careful of group think. It's best to collect the estimates independently, then aggregate. Else the group members influence each other and the advantage of "crowd sourcing" goes out the window. The book is worth a go. It reads quick. A bit light- weight but still useful. https://upliftconnect.com/tali-sharot-influential-mind/ https://upliftconnect.com/tali-sharot-influential-mind/

jacquesm 8y ago

The biggest value of bad estimates imo are that they teach you to get better at estimating. You should record your estimates and the actual time it took. That way the next time you need to make an estimate you have some more data to plug into your formula for making a guess: the previous slippage. So you improve over time and your post-mortem of why something slipped will help you to get your initial guesses closer to where they should have been in the first place. Between those two anybody can learn how to estimate with some reasonable degree of fidelity.

quantumhobbit 8y ago

There are just too many unknowns to do estimates well in software. And getting things wrong in estimates isn't a matter of being off by a couple percentage points but being off in by orders of magnitude. It isn't unusual for what appears to be a simple unit of work that seems like it should take a few hours to explode into days or weeks of work. And it does so in a way that is completely opaque to stakeholders. Partly this is because of how we build software as layers of abstraction on top of other layers of abstraction. When you are at the top layer things are relatively quick and easy to estimate, but once you have to work at a lower level than normal all bets are off. And it is really hard to know beforehand if you will need to dig into the lower levels. For example at my job I am updating an app that queries an api to filter its requests by some parameter. When the api supports the desired filter that is great. When the api doesn't, I have to dig into unfamiliar code for the api to support it. But turns out the api is just forwarding its request to another api, so I have to dig into yet another api to build the filter. And so on. The rabbit hole can get deep. And of course there isn't any documentation to help me know about this beforehand. The article is right that the key is frequent updates and communication as soon as "blockers" like this are discovered. But it can be very difficult to communicate this to stakeholders who are uninterested in the technical details and are inflexible in scope. Trust is key, so that they believe you when you say that this task which normally takes an hour or so will now take several days.