9 ms·

> And here’s the brutal question: what good are estimates if they hardly ever align with reality? You could have spent that time on building software. No one h

> And here’s the brutal question: what good are estimates if they hardly ever align with reality? You could have spent that time on building software. No one has ever been able to answer this question for me. In my experience the accuracy of estimates is highly variable. Experienced developers who know each other and the system they're working on well tend to offer more realistic estimates, but even on the most smoothly run teams it's still fundamentally guesswork. From my perspective it seems like the only real effect of estimating units of work is making developers resent people outside their team. Sure, it gives product people a number they can say when they get harassed about when something will be done, but it's no more accurate than one they could have just made up on their own. I don't think I see any fundamental difference between estimating and what this author calls forecasting in this respect. It does seem like generating these metrics could consume less time than meeting to make up estimates, but it's not obvious to me that it always would be. What real value does this add to any business? Any product or business people here? I'm genuinely curious what the purpose of estimating is. It feels like nobody is winning. Product and business people get annoyed by missed "estimates" because what they really want is to know the future, which is impossible. Developers resent being asked to predict the unpredictable. Not to mention I don't know any programmers who prefer talking about doing stuff over actually doing stuff -- all the meetings that come with management styles that include things like estimates feel like an enormous waste of money. Who is winning here?

TedDoesntTalk 6y ago
No one can tell the future. Some people try to do it by looking at the past. Investing has a mantra for that, "Past performance is no guarantee of future results." The sad part is this never changes and never will.
- dirklectisch 6y ago
  Author of the article here. There are situations that are way more predictable than the stock market. We as humans use the past to successfully predict the future all the time. For example we predict that we won't be able to walk through walls not because we understand how atoms bounce off each other but because we experienced not being able to do that. Software Development is certainly more predictable than the stock market but less predictable than the walking through walls example. In a stable team, often past performance is a strong indicator of what will happen next. It's not perfect, black swans can occur, but no one is asked to predict if your project will get cancelled for example.
choxi 6y ago
The estimate isn’t as useful but the process you go through to arrive at it can be. It forces our team to ask questions about high risk aspects of our code - where do we need to get data from? Does this touch <scary legacy service>? Or at least that’s what I’ve told myself, but admittedly it doesn’t address the central issue of we still don’t know how long this will take. We can usually tell if something is “days” vs “weeks” of work but we still get those terribly wrong sometimes. I’ve found people don’t like it when I say “it’ll get done when it‘s done” though so I’m forced to keep trying. In practice we just try to write code in a way where most things can be done in “days” of time because the variance in your outcome tends to grow very quickly when the estimate is larger.
- pmarino90 6y ago
  It is indeed true that arguing about complexity, while estimating, is useful to spark conversations about the item at hand. I feel that _estimations_ per se are not a good way to achieve those interesting conversations. You can, for example, analyze those items using the cynefin framework. The tricky thing about estimations is that, for some people, they end up being a promise that then is used to plan the future leading, in some cases, to an unpleasant reality check.
blt 6y ago
Imo, time spent on estimating would usually be better spent on design review and careful thinking about what the product should actually do. In a project with clear goals and vision, it's usually not hard to see what needs to be done.
- m463 6y ago
  I look back and see that there were some managers I've had who were masterful, but shamefully I can only appreciate their talents from here in the deep future.
- kabdib 6y ago
  Usually the hard choices are what you have to cut. It'll be clear what needs to be done, you just won't have the resources to get there. Then there are the "researchy" type projects, where you have a demo system and maybe some expert help from corporate research, and your job is to take some flighty technology held together with an unholy mashup of F#, MatLab and a depth camera and turn it into a consumer product, something people will want to buy for a hundred bucks. The first half of the project is figuring out if it's possible, the second half of the project is getting it working, the third half is polishing and shipping. ("Every project can be divided into three equal efforts, each consisting of ninety percent of the work.")
  - Aeolun 6y ago
    I’d argue that more like 75% of the project is polishing and shipping.
seer 6y ago
Well usually a dev team is not working in isolation. And a lot of the time input from one team is output to another. There are ways to mitigate the “you are blocked by this team until they finish their stuff” organizational problem, for example api contract based dev, but fundamentally you’re still blocked finishing your stuff until someone finishes theirs. And imagine if hundreds of those teams need to coordinate for a product lunch, you can’t really throw your hands in the air and just let it go to chance. Or more precisely the companies that are trying “something” would usually outperform the companies that don’t. And we’ve evolved a lot of ways to mitigate uncertainties. Scrum is all about that btw, make projects and estimations really (really) small so that any wrong estimations do not affect other teams for more than 2 sprints or something like it. All the software companies that are more than “one team of devs” really do struggle with the fundamental uncertainty of it all, its just the ones that cope with it better are outperforming the ones that don’t. Its kinda fascinating to watch this whole thing evolve over the decades like a natural darwinian selection.
m463 6y ago
> You could have spent that time on building software. When people ask "how long will it take" and "I need status" a lot of things bump around in my mind: - I read a pretty common mistake for new military officers is to spend too much time trying to get perfect strategic information - I think if I hadn't had to spend so much time, agonizing time, trying to get accurate status, I would have had it for you by now. - status is not fun, it's like doing your taxes - there's also a great annoyance: if I am actually good at breaking things down and listing all the pieces to get from A to B, it makes micromanagement of the worst kind possible. You know, when someone looks at all your line items and CANNOT HELP but cleverly rearrange everything like chess pieces on a chessboard, without being overly concerned for the consequences.
deleted 6y ago
[deleted]
hn_throwaway_99 6y ago
There are two very important reasons for estimation: 1. Coordination with other teams or real-world events. In most decent-sized companies there are many people involved in a product or feature launch: product marketing, brand marketing, support, not to mention other dev teams. Many times these teams will need to broadcast external, hard dates (e.g. "our PR firmed lined up exclusives with journalists on date X") Being substantially late can affect lots of other people. However, importantly, this is not always the case. If you're estimating for the benefit of coordinating teams, be very explicit about which other teams depend on your estimates, and why. 2. To prioritize and decide which features to build. Prioritization is solely about balancing expected benefit with expected cost, so if you're wildly off about the cost, your time may have been better spent building something else. Again, though, if you underestimate by a bit, but in the end come up and say "I wouldn't have really worked on things in a different order anyway" then nothing was really lost. I can definitely think of cases, though, where if I really knew how long something would take at the start I would have chosen to do something else, or at least do it in a different way.
- deleted 6y ago
  [deleted]
- Aeolun 6y ago
  Given what we’ve seen in the article, that estimates are often off by a factor of 4, what does this really help other teams? It might make them feel good about themselves when the dev team inevitably tells them they’re not going to meet the target date, but that’s not really helpful overall.
  - mdorazio 6y ago
    Because off by a factor of 4 is still far superior to having no idea whatsoever about what's getting built, when it might be done, and what's potentially slipping. I can't plan anything else very well if I have to wait for enough data to forecast, which itself might not even be accurate.
- pif 6y ago
  > our PR firmed lined up exclusives with journalists on date X That is the problem. Wait for the product to get to an acceptable form, and then call the press in for tomorrow!
xgb84j 6y ago
Asking how long a feature will take is like asking how much it will cost to renovate an old house. Nobody can know for sure but you as an expert can give a better estimate than most other people. Nobody just asks a structural engineer to just build a house and that costs don't matter. Why is that "wrong" estimate so important? Because business people do not see the real cost associated with features. They can only guess how many buttons you need to add to the UI. In order to build good products they need to weigh feature benefits against engineering costs. Even if the cost is wrong, it's about the magnitude compared to other options. If business people get annoyed by missed estimates they don't understand engineering estimates and how to use them properly.
PopeDotNinja 6y ago
Busy people gotta look busy!
tsimionescu 6y ago
There is some level of estimation that we would do even if we were told not to. We always choose what to work on or what to push back on based on some idea of the time it would take. With very rare exceptions, you will usually have some idea of the size of a feature, at least order of magnitude. Taking it to the absurd, if I asked you to add feature A (say, showing the current date and time in the UI) to your product, I doubt you would say 'it could take me 10 minutes, it could take me 10 years'. You will have some better idea of the size. You will sometimes be wrong about the work needed (oh, it turns out that our hardware clocks are bad and we can't rely on them, we need to change the clocks in the whole line, add another year to my estimate), but that is not going to be the most common occurrence, and so you will usually be roughly right, but only up to something like 4-10x (one day estimate - > 1-10 days; 1 month estimate - > 1-10 months). The problem is when someone takes all these rough estimates and wants to add them up to divine a release date. Of course, they don't want to go with the honest estimate of '5 months - 4 years', so they tend to average it, and treat the 10x as a 'worse case scenario ', when it is just uncertainty in the estimation. So, they end up with something like '8 months to 1 year', which rarely works out in practice. In the happy case though, the original estimates still come in handy later when it is time to drop features. 'we' re months out from the deadline, this feature was estimated at 1.5 months, and those 7 other features at 2 weeks each; let's cut the 1.5 month feature, it will not fit anyway', which is a pretty good use of that estimate.
chrisweekly 6y ago
"Plans are worthless, but planning is indispensable." (can't recall where this is from but it resonates)
sdiupIGPWEfh 6y ago
I've seen that these estimates are inevitably used to judge the performance of teams and individual contributors. If you ask why developers put up with this, my answer is that it's not done openly or transparently. Yet I see it happen, and it is at times acknowledged in hushed tones among insiders. No one's explicitly being promoted or awarded bonuses based on their ability to meet ticket estimates or anything like that. However, a team's failure to achieve predictable burn-downs will be used to justify intervention, and an employee's failure to sustain a certain velocity will be used as part of firing decisions. It's a pulse for management to watch and leverage when needed.
draebek 6y ago
I'm surprised to be the first to write: You estimate because you do contract work for US state/local government, and the paperwork and process is horrible, and so you have to write a contract where you lay out the whole final product up front, and you have to decide what to charge for the whole thing before even starting. That number is going in the contract. It isn't going up. Fun fact: ISTR at least one locality where, if they found out you started work before the contract was signed, there was some law (or maybe it was a policy, but in any case, no one could break it) forbidding the department from paying for said work. Once I asked about doing time and materials instead of big design up front. The government department I was working with started saying how that would require them to certify hours, and there was a whole process, and I got the feeling that if I pressed the issue then they'd rather just find someone who could give them a dollar amount up front.

TedDoesntTalk 6y ago

No one can tell the future. Some people try to do it by looking at the past. Investing has a mantra for that, "Past performance is no guarantee of future results." The sad part is this never changes and never will.

dirklectisch 6y ago
Author of the article here. There are situations that are way more predictable than the stock market. We as humans use the past to successfully predict the future all the time. For example we predict that we won't be able to walk through walls not because we understand how atoms bounce off each other but because we experienced not being able to do that. Software Development is certainly more predictable than the stock market but less predictable than the walking through walls example. In a stable team, often past performance is a strong indicator of what will happen next. It's not perfect, black swans can occur, but no one is asked to predict if your project will get cancelled for example.

choxi 6y ago

The estimate isn’t as useful but the process you go through to arrive at it can be. It forces our team to ask questions about high risk aspects of our code - where do we need to get data from? Does this touch <scary legacy service>? Or at least that’s what I’ve told myself, but admittedly it doesn’t address the central issue of we still don’t know how long this will take. We can usually tell if something is “days” vs “weeks” of work but we still get those terribly wrong sometimes. I’ve found people don’t like it when I say “it’ll get done when it‘s done” though so I’m forced to keep trying. In practice we just try to write code in a way where most things can be done in “days” of time because the variance in your outcome tends to grow very quickly when the estimate is larger.

pmarino90 6y ago
It is indeed true that arguing about complexity, while estimating, is useful to spark conversations about the item at hand. I feel that _estimations_ per se are not a good way to achieve those interesting conversations. You can, for example, analyze those items using the cynefin framework. The tricky thing about estimations is that, for some people, they end up being a promise that then is used to plan the future leading, in some cases, to an unpleasant reality check.

blt 6y ago

Imo, time spent on estimating would usually be better spent on design review and careful thinking about what the product should actually do. In a project with clear goals and vision, it's usually not hard to see what needs to be done.

m463 6y ago
I look back and see that there were some managers I've had who were masterful, but shamefully I can only appreciate their talents from here in the deep future.
kabdib 6y ago
Usually the hard choices are what you have to cut. It'll be clear what needs to be done, you just won't have the resources to get there. Then there are the "researchy" type projects, where you have a demo system and maybe some expert help from corporate research, and your job is to take some flighty technology held together with an unholy mashup of F#, MatLab and a depth camera and turn it into a consumer product, something people will want to buy for a hundred bucks. The first half of the project is figuring out if it's possible, the second half of the project is getting it working, the third half is polishing and shipping. ("Every project can be divided into three equal efforts, each consisting of ninety percent of the work.")
- Aeolun 6y ago
  I’d argue that more like 75% of the project is polishing and shipping.

seer 6y ago

Well usually a dev team is not working in isolation. And a lot of the time input from one team is output to another. There are ways to mitigate the “you are blocked by this team until they finish their stuff” organizational problem, for example api contract based dev, but fundamentally you’re still blocked finishing your stuff until someone finishes theirs. And imagine if hundreds of those teams need to coordinate for a product lunch, you can’t really throw your hands in the air and just let it go to chance. Or more precisely the companies that are trying “something” would usually outperform the companies that don’t. And we’ve evolved a lot of ways to mitigate uncertainties. Scrum is all about that btw, make projects and estimations really (really) small so that any wrong estimations do not affect other teams for more than 2 sprints or something like it. All the software companies that are more than “one team of devs” really do struggle with the fundamental uncertainty of it all, its just the ones that cope with it better are outperforming the ones that don’t. Its kinda fascinating to watch this whole thing evolve over the decades like a natural darwinian selection.

m463 6y ago

> You could have spent that time on building software. When people ask "how long will it take" and "I need status" a lot of things bump around in my mind: - I read a pretty common mistake for new military officers is to spend too much time trying to get perfect strategic information - I think if I hadn't had to spend so much time, agonizing time, trying to get accurate status, I would have had it for you by now. - status is not fun, it's like doing your taxes - there's also a great annoyance: if I am actually good at breaking things down and listing all the pieces to get from A to B, it makes micromanagement of the worst kind possible. You know, when someone looks at all your line items and CANNOT HELP but cleverly rearrange everything like chess pieces on a chessboard, without being overly concerned for the consequences.

deleted 6y ago

[deleted]

hn_throwaway_99 6y ago

There are two very important reasons for estimation: 1. Coordination with other teams or real-world events. In most decent-sized companies there are many people involved in a product or feature launch: product marketing, brand marketing, support, not to mention other dev teams. Many times these teams will need to broadcast external, hard dates (e.g. "our PR firmed lined up exclusives with journalists on date X") Being substantially late can affect lots of other people. However, importantly, this is not always the case. If you're estimating for the benefit of coordinating teams, be very explicit about which other teams depend on your estimates, and why. 2. To prioritize and decide which features to build. Prioritization is solely about balancing expected benefit with expected cost, so if you're wildly off about the cost, your time may have been better spent building something else. Again, though, if you underestimate by a bit, but in the end come up and say "I wouldn't have really worked on things in a different order anyway" then nothing was really lost. I can definitely think of cases, though, where if I really knew how long something would take at the start I would have chosen to do something else, or at least do it in a different way.

deleted 6y ago
[deleted]
Aeolun 6y ago
Given what we’ve seen in the article, that estimates are often off by a factor of 4, what does this really help other teams? It might make them feel good about themselves when the dev team inevitably tells them they’re not going to meet the target date, but that’s not really helpful overall.
- mdorazio 6y ago
  Because off by a factor of 4 is still far superior to having no idea whatsoever about what's getting built, when it might be done, and what's potentially slipping. I can't plan anything else very well if I have to wait for enough data to forecast, which itself might not even be accurate.
pif 6y ago
> our PR firmed lined up exclusives with journalists on date X That is the problem. Wait for the product to get to an acceptable form, and then call the press in for tomorrow!

xgb84j 6y ago

Asking how long a feature will take is like asking how much it will cost to renovate an old house. Nobody can know for sure but you as an expert can give a better estimate than most other people. Nobody just asks a structural engineer to just build a house and that costs don't matter. Why is that "wrong" estimate so important? Because business people do not see the real cost associated with features. They can only guess how many buttons you need to add to the UI. In order to build good products they need to weigh feature benefits against engineering costs. Even if the cost is wrong, it's about the magnitude compared to other options. If business people get annoyed by missed estimates they don't understand engineering estimates and how to use them properly.

PopeDotNinja 6y ago

Busy people gotta look busy!

tsimionescu 6y ago

There is some level of estimation that we would do even if we were told not to. We always choose what to work on or what to push back on based on some idea of the time it would take. With very rare exceptions, you will usually have some idea of the size of a feature, at least order of magnitude. Taking it to the absurd, if I asked you to add feature A (say, showing the current date and time in the UI) to your product, I doubt you would say 'it could take me 10 minutes, it could take me 10 years'. You will have some better idea of the size. You will sometimes be wrong about the work needed (oh, it turns out that our hardware clocks are bad and we can't rely on them, we need to change the clocks in the whole line, add another year to my estimate), but that is not going to be the most common occurrence, and so you will usually be roughly right, but only up to something like 4-10x (one day estimate - > 1-10 days; 1 month estimate - > 1-10 months). The problem is when someone takes all these rough estimates and wants to add them up to divine a release date. Of course, they don't want to go with the honest estimate of '5 months - 4 years', so they tend to average it, and treat the 10x as a 'worse case scenario ', when it is just uncertainty in the estimation. So, they end up with something like '8 months to 1 year', which rarely works out in practice. In the happy case though, the original estimates still come in handy later when it is time to drop features. 'we' re months out from the deadline, this feature was estimated at 1.5 months, and those 7 other features at 2 weeks each; let's cut the 1.5 month feature, it will not fit anyway', which is a pretty good use of that estimate.

chrisweekly 6y ago

"Plans are worthless, but planning is indispensable." (can't recall where this is from but it resonates)

sdiupIGPWEfh 6y ago

I've seen that these estimates are inevitably used to judge the performance of teams and individual contributors. If you ask why developers put up with this, my answer is that it's not done openly or transparently. Yet I see it happen, and it is at times acknowledged in hushed tones among insiders. No one's explicitly being promoted or awarded bonuses based on their ability to meet ticket estimates or anything like that. However, a team's failure to achieve predictable burn-downs will be used to justify intervention, and an employee's failure to sustain a certain velocity will be used as part of firing decisions. It's a pulse for management to watch and leverage when needed.

draebek 6y ago

I'm surprised to be the first to write: You estimate because you do contract work for US state/local government, and the paperwork and process is horrible, and so you have to write a contract where you lay out the whole final product up front, and you have to decide what to charge for the whole thing before even starting. That number is going in the contract. It isn't going up. Fun fact: ISTR at least one locality where, if they found out you started work before the contract was signed, there was some law (or maybe it was a policy, but in any case, no one could break it) forbidding the department from paying for said work. Once I asked about doing time and materials instead of big design up front. The government department I was working with started saying how that would require them to certify hours, and there was a whole process, and I got the feeling that if I pressed the issue then they'd rather just find someone who could give them a dollar amount up front.