4 ms·

> Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over aga

> Estimations are based on experience, yet developers are constantly building brand new types of products. If all you built was "blog engines" over and over again, you would get damned good at estimating them. The trick in software is as soon as something needs to be reused over and over again, it is abstracted, put in a library, built into a framework, so you can make your product different (read: build something entirely new) -- and EVERYONE sucks at estimating the cost of building things entirely new. That's it. Most, actually all, literature on software estimation I've seem ignores this simple fact. It's feasible to estimate cost and time for building houses, even skyscrapers, if you do that all the time. But how long does it take to climb a never-climbed mountain?

phillmv 13y ago
And even then, large capital projects are routinely wildly off estimates. I think the delta in "how far off we were" just tends to be larger more frequently.
- jacques_chester 13y ago
  In the Industrial Megaprojects book I quoted elsewhere, the single best predictor of every measure of project performance was "Front End Loading" -- how completely the project has been studied and specified before it is funded. There are at least three FEL stages. Companies that muck up FEL-1 are doomed to failure, companies that muck up FEL-2 or FEL-3 are doomed to poor returns.
jacques_chester 13y ago
> But how long does it take to climb a never-climbed mountain? This estimate could be derived in a number of ways. By analogy to other mountains, by creating a statistical model of climbing time based on parameters like gradient and climber skill, or by aggregating expert judgements of the mountain's likely climbing time. A better example of a difficult-to-estimate task was the Manhattan Project. How long would it take to build the first atomic weapon? That was a truly novel problem, insofar as it would require fundamental physics research to solve. But almost none of our work is of that nature. To be sure, there are unexpected requirements and complexities, but we are almost never performing fundamental research to achieve our ends.
- jared314 13y ago
  > but we are almost never performing fundamental research to achieve our ends "Fundamental research" is a matter of perspective. If a developer does not know Rails, he must perform fundamental research into building a new blog engine. While the unknowns, and variability, are significantly smaller, they are still unknown until he does the research. If she were to trod off the beaten path, even slightly, he could run into a community known, but developer unknown, roadblock/design flaw that didn't show up in basic research. (ex. A Rails blog engine vs a multi-threaded Rails blog engine)
  - jacques_chester 13y ago
    While all estimates are relative to the context, I disagree that fundamental research is. Fundamental research is about discovering or inventing something nobody has ever done before. What will be discovered or invented is an unknown; it may be unknown that it is unknown. Research is basically an open-ended search process. There's no foreseable end-state to base an estimate on -- though there are terminating conditions we can use to end research (ran out of money, made amazing discovery that will be passed to engineers to commercialise, gathered sufficient data to write publishable paper etc). Take the linked post's example of AIDS research. How long will it take to cure AIDS? There is simply no way to know. We can produce estimates, but because it is a research problem, any such estimate would have to have wildly broad ranges to be accurate ("somewhere between 10 and 100 years"). In cases of pure research, you can't estimate the end-goal. Until it's over you may not even know that there were new end-goals you were unaware of (eg. 3M's Post It notes).

phillmv 13y ago

And even then, large capital projects are routinely wildly off estimates. I think the delta in "how far off we were" just tends to be larger more frequently.

jacques_chester 13y ago
In the Industrial Megaprojects book I quoted elsewhere, the single best predictor of every measure of project performance was "Front End Loading" -- how completely the project has been studied and specified before it is funded. There are at least three FEL stages. Companies that muck up FEL-1 are doomed to failure, companies that muck up FEL-2 or FEL-3 are doomed to poor returns.

jacques_chester 13y ago

> But how long does it take to climb a never-climbed mountain? This estimate could be derived in a number of ways. By analogy to other mountains, by creating a statistical model of climbing time based on parameters like gradient and climber skill, or by aggregating expert judgements of the mountain's likely climbing time. A better example of a difficult-to-estimate task was the Manhattan Project. How long would it take to build the first atomic weapon? That was a truly novel problem, insofar as it would require fundamental physics research to solve. But almost none of our work is of that nature. To be sure, there are unexpected requirements and complexities, but we are almost never performing fundamental research to achieve our ends.

jared314 13y ago
> but we are almost never performing fundamental research to achieve our ends "Fundamental research" is a matter of perspective. If a developer does not know Rails, he must perform fundamental research into building a new blog engine. While the unknowns, and variability, are significantly smaller, they are still unknown until he does the research. If she were to trod off the beaten path, even slightly, he could run into a community known, but developer unknown, roadblock/design flaw that didn't show up in basic research. (ex. A Rails blog engine vs a multi-threaded Rails blog engine)
- jacques_chester 13y ago
  While all estimates are relative to the context, I disagree that fundamental research is. Fundamental research is about discovering or inventing something nobody has ever done before. What will be discovered or invented is an unknown; it may be unknown that it is unknown. Research is basically an open-ended search process. There's no foreseable end-state to base an estimate on -- though there are terminating conditions we can use to end research (ran out of money, made amazing discovery that will be passed to engineers to commercialise, gathered sufficient data to write publishable paper etc). Take the linked post's example of AIDS research. How long will it take to cure AIDS? There is simply no way to know. We can produce estimates, but because it is a research problem, any such estimate would have to have wildly broad ranges to be accurate ("somewhere between 10 and 100 years"). In cases of pure research, you can't estimate the end-goal. Until it's over you may not even know that there were new end-goals you were unaware of (eg. 3M's Post It notes).