4 ms·

It also does not help at all that software "estimates" are typically just sticking a finger in the air and seeing how hard the wind is blowing at the moment. Ho

It also does not help at all that software "estimates" are typically just sticking a finger in the air and seeing how hard the wind is blowing at the moment. How many engineers are doing things like using real numbers from previous work to build models before estimating? Do we really think that every project is such a unique snowflake that we can't build models? You don't even need that much data, as long as you're understanding that less data means less certainty. Which is also why every estimate should be either a range, or a single number with a confidence. "Previous work shows we can handle a project of this size in 6 months 85% of the time." Of course, this is not all on engineers either. I'm sure everyone has seen the project manager who takes estimates as commitments. Cultures that allow that to happen will see repressing of generally good estimates. (If you use modeling, you just boost your confidence interval appropriately. 6 months at 85% or 9 months at 95%? Pick which number you give based on your experience with the PM!)

xb1010 8y ago
Most projects I work on indeed consider themselves unique disruptive snowflakes. It is impossible to give estimates for such projects, since established prior art is frowned upon.
humanrebar 8y ago
> Previous work shows we can handle a project of this size in 6 months 85% of the time More like: Previous work shoes that this project should take between one week and eighteen months at a 95% confidence interval. ...there aren't many projects one can do 300 times to narrow that window. And asking for a date instead of a confidence interval is asking for an ambiguous estimate in the first place.
- jdmichal 8y ago
  Another tenet is that work is work. You attempt to break the work into chunks that are close enough to previous work efforts to use your model. If you can't do that, then you don't actually have any idea of what you're doing yet. Which is fine, but you certainly can't estimate in that state.
Cederfjard 8y ago
Is this how you/your team/your company work? Do you know of any resources you would recommend for those of us who are interested in learning more?
- jdmichal 8y ago
  This is generally part of the "No Estimates" movement. I learned about it from an internal talk, but I will ask the speaker if he has any resources I can point you to. We are not there yet, but are moving in that direction. We weren't previously even collecting the metrics required to create models in the first place, so we didn't have any data.
repolfx 8y ago
The conversion of estimates into commitments isn't some exception to the rule, it is the rule which is why only fools give the business estimates for things. Also ... Do we really think that every project is such a unique snowflake that we can't build models? Well, yes. That's inherent to what we do. Every project IS a special snowflake. If it wasn't we'd just download the software and install it. Oddly enough, installing and configuring software that is pre-made is very predictable and such projects rarely overrun. The sort of projects we're talking about are where pre-existing software won't do and thus, by definition, what you need is something special.
- ianwalter 8y ago
  Exactly. It's hilarious to imagine management coming to a dev with a problem and a set of objectives and the dev replying mockingly "What are we some kind of special snowflake that we have to satisfy a different set of requirements than an off-the-shelf solution can satisfy?".
- jdmichal 8y ago
  Every structure and bridge and road and tunnel project is obviously unique also. Unique terrain, unique architecture, unique engineering. If the idea is to look to other engineering to improve this, again, it's not like software is some kind of special.
  - repolfx 8y ago
    They're only unique in trivial ways. Roads are made of the same material everywhere - they're different only in some strict geometric sense. Meanwhile even very ambitious tunnelling projects like the Gotthard Base Tunnel, if you watch documentaries about them, they only hit a very small number of really unexpected issues even over long periods of time. Tunnelling is an incredibly predictable business. You buy some TBMs, set them going and worst case, you hit unexpected rock types or have an unexpected malfunction. The technology is advanced but rock is the same as rock has always been. Once the project planning is over requirements don't change. The course you're on doesn't change. Workers are largely trainable and replaceable - you never find you're unexpectedly relying on some random worker somewhere deep in the org chart who is 50x more skilled than their coworkers and who just got poached by a company that pays 3x as much as you can. The users/customers have no input: they get what you deliver and a tunnel is such a simple thing talking about whether they like it or not is a nonsensical question - it's a tunnel.
LargeWu 8y ago
In my experience, PM's or whoever usually want estimates on the spot. They want estimates fast, but then they treat them as though they were well researched. If you tell them "there's a 75% probability we can get it done in 2 months" they hear "we can get it done in 2 months". There's also a tendency to leave out important information. I recently worked on a project where 1) more than half of the features were not in the requirements because the product manager assumed we could re-use existing functionality from a completely separate application (we couldn't); 2) half of the requirements from the new work were left out of requirements because they didn't bother to specify them; 3) a critical requirement regarding how data was to be aggregated in search results (which is the primary feature) was not included because the architect didn't bother to tell anybody about it except for a passing "hey, what if we did this instead" before going on vacation. This caused 2 weeks of re-work after we were done, that would have taken 2 days if we did it from the start. Our engineering team reviews all requirements as a team before giving estimates and this STILL happened, because we couldn't know what was missing. Good estimates begin with good requirements, and usually the people responsible for those do not end up being accountable, it ALWAYS falls on engineering.
- JoeAltmaier 8y ago
  That serial approach (get good requirements; design and code; test) always works that way. An iterative approach can work better. Why isn't the architect in standup at least during the beginning of the project? All this could have been shortcut.
  - shanghaiaway 8y ago
    Because developers want to be left alone and also because they think meetings are unnecessary
zerkten 8y ago
> How many engineers are doing things like using real numbers from previous work to build models before estimating? Fogbugz created a feature called Evidence-based Scheduling (http://help.fogcreek.com/7676/evidence-based-scheduling-ebs http://help.fogcreek.com/7676/evidence-based-scheduling-ebs). It's probably the only real quantitative approach that I have seen while working on web development projects. I'm probably wrong to assume that more "technical" fields like aerospace do much, much more than this, but it was really interesting to use this with what was mostly CRUD-work. EDIT: added link.

xb1010 8y ago

Most projects I work on indeed consider themselves unique disruptive snowflakes. It is impossible to give estimates for such projects, since established prior art is frowned upon.

humanrebar 8y ago

> Previous work shows we can handle a project of this size in 6 months 85% of the time More like: Previous work shoes that this project should take between one week and eighteen months at a 95% confidence interval. ...there aren't many projects one can do 300 times to narrow that window. And asking for a date instead of a confidence interval is asking for an ambiguous estimate in the first place.

jdmichal 8y ago
Another tenet is that work is work. You attempt to break the work into chunks that are close enough to previous work efforts to use your model. If you can't do that, then you don't actually have any idea of what you're doing yet. Which is fine, but you certainly can't estimate in that state.

Cederfjard 8y ago

Is this how you/your team/your company work? Do you know of any resources you would recommend for those of us who are interested in learning more?

jdmichal 8y ago
This is generally part of the "No Estimates" movement. I learned about it from an internal talk, but I will ask the speaker if he has any resources I can point you to. We are not there yet, but are moving in that direction. We weren't previously even collecting the metrics required to create models in the first place, so we didn't have any data.

repolfx 8y ago

The conversion of estimates into commitments isn't some exception to the rule, it is the rule which is why only fools give the business estimates for things. Also ... Do we really think that every project is such a unique snowflake that we can't build models? Well, yes. That's inherent to what we do. Every project IS a special snowflake. If it wasn't we'd just download the software and install it. Oddly enough, installing and configuring software that is pre-made is very predictable and such projects rarely overrun. The sort of projects we're talking about are where pre-existing software won't do and thus, by definition, what you need is something special.

ianwalter 8y ago
Exactly. It's hilarious to imagine management coming to a dev with a problem and a set of objectives and the dev replying mockingly "What are we some kind of special snowflake that we have to satisfy a different set of requirements than an off-the-shelf solution can satisfy?".
jdmichal 8y ago
Every structure and bridge and road and tunnel project is obviously unique also. Unique terrain, unique architecture, unique engineering. If the idea is to look to other engineering to improve this, again, it's not like software is some kind of special.
- repolfx 8y ago
  They're only unique in trivial ways. Roads are made of the same material everywhere - they're different only in some strict geometric sense. Meanwhile even very ambitious tunnelling projects like the Gotthard Base Tunnel, if you watch documentaries about them, they only hit a very small number of really unexpected issues even over long periods of time. Tunnelling is an incredibly predictable business. You buy some TBMs, set them going and worst case, you hit unexpected rock types or have an unexpected malfunction. The technology is advanced but rock is the same as rock has always been. Once the project planning is over requirements don't change. The course you're on doesn't change. Workers are largely trainable and replaceable - you never find you're unexpectedly relying on some random worker somewhere deep in the org chart who is 50x more skilled than their coworkers and who just got poached by a company that pays 3x as much as you can. The users/customers have no input: they get what you deliver and a tunnel is such a simple thing talking about whether they like it or not is a nonsensical question - it's a tunnel.

LargeWu 8y ago

In my experience, PM's or whoever usually want estimates on the spot. They want estimates fast, but then they treat them as though they were well researched. If you tell them "there's a 75% probability we can get it done in 2 months" they hear "we can get it done in 2 months". There's also a tendency to leave out important information. I recently worked on a project where 1) more than half of the features were not in the requirements because the product manager assumed we could re-use existing functionality from a completely separate application (we couldn't); 2) half of the requirements from the new work were left out of requirements because they didn't bother to specify them; 3) a critical requirement regarding how data was to be aggregated in search results (which is the primary feature) was not included because the architect didn't bother to tell anybody about it except for a passing "hey, what if we did this instead" before going on vacation. This caused 2 weeks of re-work after we were done, that would have taken 2 days if we did it from the start. Our engineering team reviews all requirements as a team before giving estimates and this STILL happened, because we couldn't know what was missing. Good estimates begin with good requirements, and usually the people responsible for those do not end up being accountable, it ALWAYS falls on engineering.

JoeAltmaier 8y ago
That serial approach (get good requirements; design and code; test) always works that way. An iterative approach can work better. Why isn't the architect in standup at least during the beginning of the project? All this could have been shortcut.
- shanghaiaway 8y ago
  Because developers want to be left alone and also because they think meetings are unnecessary

zerkten 8y ago

> How many engineers are doing things like using real numbers from previous work to build models before estimating? Fogbugz created a feature called Evidence-based Scheduling (http://help.fogcreek.com/7676/evidence-based-scheduling-ebs http://help.fogcreek.com/7676/evidence-based-scheduling-ebs). It's probably the only real quantitative approach that I have seen while working on web development projects. I'm probably wrong to assume that more "technical" fields like aerospace do much, much more than this, but it was really interesting to use this with what was mostly CRUD-work. EDIT: added link.