3 ms·

The reason software estimation is hard is that estimation is hard. We all imagine that our problems are unique and other disciplines don't have them. Software e

The reason software estimation is hard is that estimation is hard. We all imagine that our problems are unique and other disciplines don't have them. Software estimation is so special that no one outside of CS can possibly contribute to it. In looking for software reasons why our estimates are bad, we ignore the glaring fact that estimates of any kind are often bad. But they are. The reasons are psychological biases, and they're brilliantly explained by Daniel Kahneman in Thinking Fast and Slow. In the chapter The Outside View he tells the difference between the Inside view and the Outside view. In the outside view, you know nothing about the case except the class that it belongs to. In the inside view you know, or think you know, a lot about this particular project. A short quote that summarizes it (his problem domain is a group writing a new textbook): ======================== The spectacular accuracy of the outside-view forecast in our problem was surely a fluke and should not count as evidence for the validity of the outside view. The argument for the outside-view should be made on general grounds: if the reference class is properly chosen, the outside-view will give an indication of where the ballpark is, and it may suggest, as it did in our case, that the inside-view forecasts are not even close to it. =========================== Somehow, I'm reminded of the blindness that amateur pilots have about fatal crashes. One actually said to me, "I've looked at all those crashes, and I'm confident I wouldn't have made those mistakes." So for software disasters, we tend to shudder and say, "Hopefully nothing that terrible will happen to us."

kqr 4y ago
This is a good point. I've had much better luck estimating using the outside view, as Kahneman calls it (or willful ignorance, as Weisberg calls it). I think the problem is people in IT (including developers) don't have much statistical training. Taking the outside view and believing it requires faith in statistical principles. The people I meet in our field want to hear about nice narratives and specific examples, not reference-class generalisations and probability distributions.
- AlbertCory 4y ago
  Well said. It's not that they can't provide estimates, it's that they don't want to. If you read RunSet's answer above: he really believes every project is unique, and you can't estimate anything about it. That's the attitude we have here: "I'm doing something that's never been done before!"
RunSet 4y ago
> We all imagine that our problems are unique and other disciplines don't have them. Software estimation is so special that no one outside of CS can possibly contribute to it. Software development is special in that, by virtue of it involving automation, each estimation is largely unprecedented. If you are estimating how long it might take to build a house, you have a wealth of precedent on which to base your estimate to ensure you are at least in the ballpark. Perhaps the house will be founded on a sinkhole and you will have to start over or there may be a lumber shortage when obtaining the supplies for the house, but you at least have a reference for a similar process that went well in the past to know what your own endeavor ought to resemble. If you are estimating how long it might take to automate a task (presumably a task that has never before been automated, hence the need to automate it in the first place) you are, relatively speaking, blazing a new trail. It seems possible to me that "accurately estimate the time required to complete this software" is the halting problem promoted to a management position.
- TeMPOraL 4y ago
  The corollary here is also that, if you do enough trailblazing that you start recognizing patterns that makes your estimates much more accurate, it means you likely know enough to automate away the predictable portions of the process - making future deliveries faster, but again not possible to estimate accurately. And, if those predictable parts aren't very specific to your business niche, others will discover them too, and someone will eventually automate them away in form of a library/framework, service, methodology, or something else you'll end up adopting. Ultimately, having such close feedback loop with automation, by virtue of being done in a virtual medium, makes the software process anti-inductive wrt. estimates. Like with stock market, exploiting some regularity effectively makes it go away.
- AlbertCory 4y ago
  That's the "everything we do is a one-off" fallacy. It isn't. I believe there is a small set of questions you could ask before starting, and that would give you the ballpark estimate. Or, "the outside-view" if you like.
commandlinefan 4y ago
> estimation is hard ... and pointless. Imagine a sort of strawman (but realistic) candid conversation: "Why do you need to know how long this is going to take to deliver?" "Because we only have a limited budget" "Do you know how much money you have?" "Yes" "So if I estimate that it's going to take longer than you have budgeted...?"

kqr 4y ago

This is a good point. I've had much better luck estimating using the outside view, as Kahneman calls it (or willful ignorance, as Weisberg calls it). I think the problem is people in IT (including developers) don't have much statistical training. Taking the outside view and believing it requires faith in statistical principles. The people I meet in our field want to hear about nice narratives and specific examples, not reference-class generalisations and probability distributions.

AlbertCory 4y ago
Well said. It's not that they can't provide estimates, it's that they don't want to. If you read RunSet's answer above: he really believes every project is unique, and you can't estimate anything about it. That's the attitude we have here: "I'm doing something that's never been done before!"

RunSet 4y ago

> We all imagine that our problems are unique and other disciplines don't have them. Software estimation is so special that no one outside of CS can possibly contribute to it. Software development is special in that, by virtue of it involving automation, each estimation is largely unprecedented. If you are estimating how long it might take to build a house, you have a wealth of precedent on which to base your estimate to ensure you are at least in the ballpark. Perhaps the house will be founded on a sinkhole and you will have to start over or there may be a lumber shortage when obtaining the supplies for the house, but you at least have a reference for a similar process that went well in the past to know what your own endeavor ought to resemble. If you are estimating how long it might take to automate a task (presumably a task that has never before been automated, hence the need to automate it in the first place) you are, relatively speaking, blazing a new trail. It seems possible to me that "accurately estimate the time required to complete this software" is the halting problem promoted to a management position.

TeMPOraL 4y ago
The corollary here is also that, if you do enough trailblazing that you start recognizing patterns that makes your estimates much more accurate, it means you likely know enough to automate away the predictable portions of the process - making future deliveries faster, but again not possible to estimate accurately. And, if those predictable parts aren't very specific to your business niche, others will discover them too, and someone will eventually automate them away in form of a library/framework, service, methodology, or something else you'll end up adopting. Ultimately, having such close feedback loop with automation, by virtue of being done in a virtual medium, makes the software process anti-inductive wrt. estimates. Like with stock market, exploiting some regularity effectively makes it go away.
AlbertCory 4y ago
That's the "everything we do is a one-off" fallacy. It isn't. I believe there is a small set of questions you could ask before starting, and that would give you the ballpark estimate. Or, "the outside-view" if you like.

commandlinefan 4y ago

> estimation is hard ... and pointless. Imagine a sort of strawman (but realistic) candid conversation: "Why do you need to know how long this is going to take to deliver?" "Because we only have a limited budget" "Do you know how much money you have?" "Yes" "So if I estimate that it's going to take longer than you have budgeted...?"