6 ms·

Communicating Software Estimates

38 points by wkirby 2y ago

atoav 2y ago
Generally good advice, however I think the actual example phrases could still be clarified, e.g. bg explaining why clear estimates in software engineering are not as common as in other fields — note the person you are talking to might have a completely different experience in their field of work. And then you give your buest guess and a clear explaination of your level of confidence, which factors could lead to shorter or longer project durations.
nlawalker 2y ago
> When you're communicating an estimate the most likely mistake is that the other party considers it to be a commitment. The root cause is typically that you're communicating it to someone who doesn't want an estimate. In that case, communicating the estimate with the commitment is often a mistake unless you're willing to negotiate the commitment, because that's effectively what it invites.
teeray 2y ago
The best estimates I’ve found are “This will take: days, weeks, months, years”. No numeric values allowed. Yes, you can’t do (inherently faulty) math on these estimates to arrive at aggregate metrics: this is a feature. However, it still allows you to meaningfully schedule work.
- Dyac 2y ago
  If pushed, I get my developers to give estimates in jumps of ~5x. Their options are 2 days 2 weeks 2 months 10 months Then I triple the estimates before sharing with the business. We don't estimate individual tickets/bugs at all, just overarching projects. I also ask the business to estimate the commercial/user impact of the projects too, and we track and report the reality against their estimate, to hold a bit of a mirror up to them and as a way of pushing back on doing pointless work. Those estimates we use similar orders of magnitude for - £1k, £10k, £100k, £1m, £10m. Fermi estimations like these really help avoid protracted negotiations and the lack of precision is a feature that makes clear they are estimated.
nine_zeros 2y ago
The best way to estimate: "Wild guess - X weeks but could be X+Y weeks for unknown unknowns" (where X is 200% estimate in your mind and Y is 700% of estimate in your mind)
cynicalsecurity 2y ago
Bad companies demand estimates. Good companies just live by a Steam clock.
tyleo 2y ago
Having run a software team with good estimates, I’m disappointed in some of the commentary of, “just make up a long or obscure estimate.” I don’t expect a considerable amount of unknowns to come up in a project unless you have changing requirements or bad planning. You can’t do anything about changing requirements so that can be forgiven. Poor planning cannot though.
- necovek 2y ago
  It usually centers on whether changing requirements are due to poor planning or not? I don't necessarily think it is, and I believe it's easily fixed by keeping the reason (expected value) for anything close to any requirements being passed down to engineering teams.
- datadrivenangel 2y ago
  Unreasonable stakeholders will absolutely hold you accountable to not meeting a schedule even when they're constantly changing requirements. And they can and will throw you under the bus for it. You are right though, that a decent team that does decent planning should be fairly accurate much of the time. And there are project management techniques to handle the remaining uncertainty.
  - tuna74 2y ago
    If requirements change estimates change as well.
    - perrygeo 2y ago
      In theory, yes. In practice, if you try to change the estimate, you'll be shot down and asked to work harder. The people changing the requirements assume that software is infinitely malleable and any change is doable with a small amount of effort. After all the devs are smart so they'll figure it out, like they always do. To these people, asking for a change in estimate is indistinguishable from complaining or being lazy.
      - 2y ago
srveale 2y ago
Chiming in with helpful rules of thumb I go by: 1. Never give an estimate in the same conversation in which it was requested, unless you're quite sure the answer is "less than an hour". Especially if they are going to turn around and quote your timeframe to a customer. The reason for this rule is: 2. The more you think about an estimate, the higher it will go. "Oh yeah and we'll also have to..." is 100x more common than "I found a shortcut that doesn't compromise on..." 3. Even if they know it's an estimate and not a deadline, they might not know the difference between "40 hours of dedicated effort" and "One week's worth of business hours". Making that distinction is just as important as the estimate/deadline distinction. I'm lucky to work where we don't spend much effort on estimates and time tracking, both of which just make things take longer and cause productivity-reducing stress.
- datadrivenangel 2y ago
  The distinction between labor time and calendar time is an important one! 40 hours of dedicated effort is great, but most people parse that as "1 week" when in reality the available people may only be able to get 10 or 20 hours in a week due to other demands, or it's 38 hours of dedicated time that gets done in 1 week but the 2 hours of security and legal reviews take a month of waiting to finally get completed.
  - ozim 2y ago
    I call it lead time and labor time. 1FTE labor will be x hours but lead time can be couple weeks depending on where are we in development cycle.
    - ryoshu 2y ago
      This is great framing.
- wkirby 2y ago
  > The more you think about an estimate, the higher it will go This is excellent advice. At Apsis, we have a collection of "red flags" for any project that we call out specifically as likely sources of increased complexity and therefore a higher estimate. Some of these are: * integrations external APIs * existing tech from the client with no provided documentation * any new infrastructure being spun up
msteffen 2y ago
When I switched from engineering to management, I spent a long time thinking about software estimates, read about many different approaches, and argued with many other managers (I’ll try to write up my own blog post for the next time the topic comes up on here). The framing in this blog post is better than some (it at least discusses uncertainty) but I believe it’s still basically the wrong framework. People (managers) like to assume that estimates are a random variable, and software projects are samples from project space. Estimates can be produced for a project by analogizing it to similar projects. This is not true at all. You cannot reason about software projects by analogy. Some assertions (that I will explore in said promised blog post): - Software development is chaotic: arbitrarily small changes in the input (project requirements, who’s implementing it, stakeholders, team, etc) may lead to arbitrarily large changes in time-to-completion (including “this project is now impossible”). In short, any project may explode at any moment prior to completion. - The risk of a project is particularly sensitive to the engineer implementing it, and depends on that person’s specific knowledge. Have they used these APIs or this database before? Any part of a project that the engineer hasn’t used before represents potentially unbounded technical risk. - the way to manage this risk is to include buffer in the project timeline, and, critically, to use that buffer not for development tasks that were harder than expected, but for re-planning and re-negotiating the deliverables with stakeholders, to un-explode the project. Agile builds this re-evaluation into the project lifecycle, which is the main (maybe only?) thing is does really right.
- necovek 2y ago
  That's too much of a carrot IMHO: I would have appreciated it more if you dove deeper into one of those points instead of promising a blog post on a dozen. Edit: thanks for editing the comment to make it clearer what are your points. This makes my comment somewhat useless :)
- necovek 2y ago
  I agree that the points you raise are important to realise and affect delivery duration significantly. I, however, disagree with that approach to managing delivery: if you tie in expected value with deliverables, you empower and enable engineering teams to smartly cut scope and deliver in shorter amount of time the biggest chunk of the value. It's usually only described as "cutting scope", but it requires real creativity and smarts on the engineering team (they can best tell what's feasible in remaining time, when they have tackled all the unknown unknowns you bring up) to do that in a way that still brings the most of the original value. If you move this to your "renegotiation" step, it becomes too slow and hurts delivery as well.
akomtu 2y ago
An estimate is really a probability distribution, that often resembles a Gaussian distribution and thus can be captured with two numbers: the average and the variance. Combining multiple estimates together would involve some math around these two numbers. But the surprising thing is that managers in the software industry don't really understand probabilities, and to make it understandable those managers want to reduce the model to just one number, and when this model fails to capture the probabilistic nature of the underlying process, they attempt to overrule the math with authority.
- mikrl 2y ago
  >they attempt to overrule the math with authority It’s entirely possible to turn that to your advantage, but takes some luck/skill, not dissimilar to profiting off of stock market crashes. The asymmetry between mathematics and authority tends to favour the math.
- jbay808 2y ago
  This is correct, although in my experience the distribution is often more like a lognormal than a Gaussian: https://thesearesystems.substack.com/p/task-estimation-conquering-hofstadters https://thesearesystems.substack.com/p/task-estimation-conqu...
lifeisstillgood 2y ago
What is an estimate for? It is really an estimate of cost where big-O is mostly developer cost and that gives a handle on the next part priority. Both of these then add up to an illusion of control. Software is quickly becoming the largest moving part of many many organisations - and trying to control it is likely the wrong approach Software is a form of capital just like a robot on the Tesla assembly line. That robot and the software being estimated in the article are to do a specific job in a specific environment- an investment that will enable more production than without. But all capital rots - or rather depreciates. Taking a nice view you can have 10% on maintenance and repairs (but realistically more than that). Don’t try to estimate these - don’t ask your handyman for a Gantt chart for a hundred tiny fixes - just get them done.
- wkirby 2y ago
  I think you're identifying something adjacent but materially different: an attempt to estimate and wrangle ongoing maintenance, rather than an estimate for an initial development effort. It's worth calling out that we (Apsis) are a software development agency; our estimates are for our clients, usually for greenfield work, and usually are part of the process of bidding for projects. A very different situation from an engineering team in year 5 of production deployment. That said, even as a member of an internal engineering team, you'll need to coordinate work with the other areas of the business: you can't maneuver a large organization with no calendar for when new features, new products, etc., will become available. Estimates are an inherent part of engineering interacting with the rest of an organization, and I think this framework is still applicable.
  - lifeisstillgood 2y ago
    So I think that the idea of “you cannot maneuver a large org without a calendar” is going back to tommdemarco and risk management as the soul of project mgmt If you run a company and ask me for an estimate on when software will be ready because the rest of the company depends on it, then what you are doing is making a huge bet on the software team delivering to time and quality (and cost!). Now that’s just hoping. There is plenty to do to fix it (reduce scope, improve team dynamics and psychological safety, dual delivery teams etc). But in the end it’s a bet, a guess. Now perhaps there are better ways. I would suggest we look twice at using calendars and deadlines as ways of acgieveing co-ordination - software is waaay better at co-ordination than people and calendars are. Adjust the business model, early releases for special Clients - these can be non-software approaches to risk mitigation, but we find the tech solutions more fun.
datadrivenangel 2y ago
Combining Software time/cost estimates with value estimates make it even more fun, because most software is low value (with some chance of being extremely valuable). Combining these into an ROI probability space makes everybody sad, but the expected value is still positive even if the most likely outcome is software that wasn't really worth paying for!
tuna74 2y ago
A more interesting question is how you follow up on your estimates? How can you improve if you don't track and reflect on the actual result on what was estimated?
- wkirby 2y ago
  Fortunately for us, we have no choice! We (Apsis Labs) are a dev agency for hire: if our estimates are wrong, we lose our clients, and then we don't have money and then we all die of starvation. Being good at estimating our work, and then communicating those estimates and expectations to our clients, is how we've stayed fed for 12 years.
dasil003 2y ago
This article describes some challenges and pitfalls of communicating estimates but doesn’t offer much actual advice. In my experience if you’re talking about risk distributions or hand wringing over the difference between an estimate and a commitment, you’re already on the defensive and not doing your reputation any favors. The better approach is to zoom out and understand the business or product goals, including constraints felt by your stakeholders, and communicate in their terms. Speak confidently to what you can do, with appropriate footnotes for limitations and risks. Even in a low trust environment you don’t succeed by focusing on the negative (though you might want to keep a paper trail), you succeed by showing what you can achieve.

38 points by wkirby 2y ago

Generally good advice, however I think the actual example phrases could still be clarified, e.g. bg explaining why clear estimates in software engineering are not as common as in other fields — note the person you are talking to might have a completely different experience in their field of work. And then you give your buest guess and a clear explaination of your level of confidence, which factors could lead to shorter or longer project durations.

nlawalker 2y ago

> When you're communicating an estimate the most likely mistake is that the other party considers it to be a commitment. The root cause is typically that you're communicating it to someone who doesn't want an estimate. In that case, communicating the estimate with the commitment is often a mistake unless you're willing to negotiate the commitment, because that's effectively what it invites.

teeray 2y ago

The best estimates I’ve found are “This will take: days, weeks, months, years”. No numeric values allowed. Yes, you can’t do (inherently faulty) math on these estimates to arrive at aggregate metrics: this is a feature. However, it still allows you to meaningfully schedule work.

Dyac 2y ago
If pushed, I get my developers to give estimates in jumps of ~5x. Their options are 2 days 2 weeks 2 months 10 months Then I triple the estimates before sharing with the business. We don't estimate individual tickets/bugs at all, just overarching projects. I also ask the business to estimate the commercial/user impact of the projects too, and we track and report the reality against their estimate, to hold a bit of a mirror up to them and as a way of pushing back on doing pointless work. Those estimates we use similar orders of magnitude for - £1k, £10k, £100k, £1m, £10m. Fermi estimations like these really help avoid protracted negotiations and the lack of precision is a feature that makes clear they are estimated.

nine_zeros 2y ago

The best way to estimate: "Wild guess - X weeks but could be X+Y weeks for unknown unknowns" (where X is 200% estimate in your mind and Y is 700% of estimate in your mind)

cynicalsecurity 2y ago

Bad companies demand estimates. Good companies just live by a Steam clock.

tyleo 2y ago

Having run a software team with good estimates, I’m disappointed in some of the commentary of, “just make up a long or obscure estimate.” I don’t expect a considerable amount of unknowns to come up in a project unless you have changing requirements or bad planning. You can’t do anything about changing requirements so that can be forgiven. Poor planning cannot though.

necovek 2y ago
It usually centers on whether changing requirements are due to poor planning or not? I don't necessarily think it is, and I believe it's easily fixed by keeping the reason (expected value) for anything close to any requirements being passed down to engineering teams.
datadrivenangel 2y ago
Unreasonable stakeholders will absolutely hold you accountable to not meeting a schedule even when they're constantly changing requirements. And they can and will throw you under the bus for it. You are right though, that a decent team that does decent planning should be fairly accurate much of the time. And there are project management techniques to handle the remaining uncertainty.
- tuna74 2y ago
  If requirements change estimates change as well.
  - perrygeo 2y ago
    In theory, yes. In practice, if you try to change the estimate, you'll be shot down and asked to work harder. The people changing the requirements assume that software is infinitely malleable and any change is doable with a small amount of effort. After all the devs are smart so they'll figure it out, like they always do. To these people, asking for a change in estimate is indistinguishable from complaining or being lazy.
    - 2y ago

srveale 2y ago

Chiming in with helpful rules of thumb I go by: 1. Never give an estimate in the same conversation in which it was requested, unless you're quite sure the answer is "less than an hour". Especially if they are going to turn around and quote your timeframe to a customer. The reason for this rule is: 2. The more you think about an estimate, the higher it will go. "Oh yeah and we'll also have to..." is 100x more common than "I found a shortcut that doesn't compromise on..." 3. Even if they know it's an estimate and not a deadline, they might not know the difference between "40 hours of dedicated effort" and "One week's worth of business hours". Making that distinction is just as important as the estimate/deadline distinction. I'm lucky to work where we don't spend much effort on estimates and time tracking, both of which just make things take longer and cause productivity-reducing stress.

datadrivenangel 2y ago
The distinction between labor time and calendar time is an important one! 40 hours of dedicated effort is great, but most people parse that as "1 week" when in reality the available people may only be able to get 10 or 20 hours in a week due to other demands, or it's 38 hours of dedicated time that gets done in 1 week but the 2 hours of security and legal reviews take a month of waiting to finally get completed.
- ozim 2y ago
  I call it lead time and labor time. 1FTE labor will be x hours but lead time can be couple weeks depending on where are we in development cycle.
  - ryoshu 2y ago
    This is great framing.
wkirby 2y ago
> The more you think about an estimate, the higher it will go This is excellent advice. At Apsis, we have a collection of "red flags" for any project that we call out specifically as likely sources of increased complexity and therefore a higher estimate. Some of these are: * integrations external APIs * existing tech from the client with no provided documentation * any new infrastructure being spun up

msteffen 2y ago

When I switched from engineering to management, I spent a long time thinking about software estimates, read about many different approaches, and argued with many other managers (I’ll try to write up my own blog post for the next time the topic comes up on here). The framing in this blog post is better than some (it at least discusses uncertainty) but I believe it’s still basically the wrong framework. People (managers) like to assume that estimates are a random variable, and software projects are samples from project space. Estimates can be produced for a project by analogizing it to similar projects. This is not true at all. You cannot reason about software projects by analogy. Some assertions (that I will explore in said promised blog post): - Software development is chaotic: arbitrarily small changes in the input (project requirements, who’s implementing it, stakeholders, team, etc) may lead to arbitrarily large changes in time-to-completion (including “this project is now impossible”). In short, any project may explode at any moment prior to completion. - The risk of a project is particularly sensitive to the engineer implementing it, and depends on that person’s specific knowledge. Have they used these APIs or this database before? Any part of a project that the engineer hasn’t used before represents potentially unbounded technical risk. - the way to manage this risk is to include buffer in the project timeline, and, critically, to use that buffer not for development tasks that were harder than expected, but for re-planning and re-negotiating the deliverables with stakeholders, to un-explode the project. Agile builds this re-evaluation into the project lifecycle, which is the main (maybe only?) thing is does really right.

necovek 2y ago
That's too much of a carrot IMHO: I would have appreciated it more if you dove deeper into one of those points instead of promising a blog post on a dozen. Edit: thanks for editing the comment to make it clearer what are your points. This makes my comment somewhat useless :)
necovek 2y ago
I agree that the points you raise are important to realise and affect delivery duration significantly. I, however, disagree with that approach to managing delivery: if you tie in expected value with deliverables, you empower and enable engineering teams to smartly cut scope and deliver in shorter amount of time the biggest chunk of the value. It's usually only described as "cutting scope", but it requires real creativity and smarts on the engineering team (they can best tell what's feasible in remaining time, when they have tackled all the unknown unknowns you bring up) to do that in a way that still brings the most of the original value. If you move this to your "renegotiation" step, it becomes too slow and hurts delivery as well.

akomtu 2y ago

An estimate is really a probability distribution, that often resembles a Gaussian distribution and thus can be captured with two numbers: the average and the variance. Combining multiple estimates together would involve some math around these two numbers. But the surprising thing is that managers in the software industry don't really understand probabilities, and to make it understandable those managers want to reduce the model to just one number, and when this model fails to capture the probabilistic nature of the underlying process, they attempt to overrule the math with authority.

mikrl 2y ago
>they attempt to overrule the math with authority It’s entirely possible to turn that to your advantage, but takes some luck/skill, not dissimilar to profiting off of stock market crashes. The asymmetry between mathematics and authority tends to favour the math.
jbay808 2y ago
This is correct, although in my experience the distribution is often more like a lognormal than a Gaussian: https://thesearesystems.substack.com/p/task-estimation-conquering-hofstadters https://thesearesystems.substack.com/p/task-estimation-conqu...

lifeisstillgood 2y ago

What is an estimate for? It is really an estimate of cost where big-O is mostly developer cost and that gives a handle on the next part priority. Both of these then add up to an illusion of control. Software is quickly becoming the largest moving part of many many organisations - and trying to control it is likely the wrong approach Software is a form of capital just like a robot on the Tesla assembly line. That robot and the software being estimated in the article are to do a specific job in a specific environment- an investment that will enable more production than without. But all capital rots - or rather depreciates. Taking a nice view you can have 10% on maintenance and repairs (but realistically more than that). Don’t try to estimate these - don’t ask your handyman for a Gantt chart for a hundred tiny fixes - just get them done.

wkirby 2y ago
I think you're identifying something adjacent but materially different: an attempt to estimate and wrangle ongoing maintenance, rather than an estimate for an initial development effort. It's worth calling out that we (Apsis) are a software development agency; our estimates are for our clients, usually for greenfield work, and usually are part of the process of bidding for projects. A very different situation from an engineering team in year 5 of production deployment. That said, even as a member of an internal engineering team, you'll need to coordinate work with the other areas of the business: you can't maneuver a large organization with no calendar for when new features, new products, etc., will become available. Estimates are an inherent part of engineering interacting with the rest of an organization, and I think this framework is still applicable.
- lifeisstillgood 2y ago
  So I think that the idea of “you cannot maneuver a large org without a calendar” is going back to tommdemarco and risk management as the soul of project mgmt If you run a company and ask me for an estimate on when software will be ready because the rest of the company depends on it, then what you are doing is making a huge bet on the software team delivering to time and quality (and cost!). Now that’s just hoping. There is plenty to do to fix it (reduce scope, improve team dynamics and psychological safety, dual delivery teams etc). But in the end it’s a bet, a guess. Now perhaps there are better ways. I would suggest we look twice at using calendars and deadlines as ways of acgieveing co-ordination - software is waaay better at co-ordination than people and calendars are. Adjust the business model, early releases for special Clients - these can be non-software approaches to risk mitigation, but we find the tech solutions more fun.

datadrivenangel 2y ago

Combining Software time/cost estimates with value estimates make it even more fun, because most software is low value (with some chance of being extremely valuable). Combining these into an ROI probability space makes everybody sad, but the expected value is still positive even if the most likely outcome is software that wasn't really worth paying for!

tuna74 2y ago

A more interesting question is how you follow up on your estimates? How can you improve if you don't track and reflect on the actual result on what was estimated?

wkirby 2y ago
Fortunately for us, we have no choice! We (Apsis Labs) are a dev agency for hire: if our estimates are wrong, we lose our clients, and then we don't have money and then we all die of starvation. Being good at estimating our work, and then communicating those estimates and expectations to our clients, is how we've stayed fed for 12 years.

dasil003 2y ago

This article describes some challenges and pitfalls of communicating estimates but doesn’t offer much actual advice. In my experience if you’re talking about risk distributions or hand wringing over the difference between an estimate and a commitment, you’re already on the defensive and not doing your reputation any favors. The better approach is to zoom out and understand the business or product goals, including constraints felt by your stakeholders, and communicate in their terms. Speak confidently to what you can do, with appropriate footnotes for limitations and risks. Even in a low trust environment you don’t succeed by focusing on the negative (though you might want to keep a paper trail), you succeed by showing what you can achieve.