4 ms·

"...The gist of why estimates are hard: every new piece of software is a machine that has never been built before..." Yes, and that's why a lightweight, repeat

"...The gist of why estimates are hard: every new piece of software is a machine that has never been built before..." Yes, and that's why a lightweight, repeatable estimation process beats every other way of doing it. As you continue to estimate, you create and refine a mental model of the project's complexity. For some projects, you're able to create a mental model that has high fidelity quite easily. For others, it takes a bit of work. The entire article here was an exposition on this fact. Some folks figure that out and want to just give up. Estimation is impossible! Other folks, however, are going to figure it out if it kills them. What happens in these cases is they start to list every possible variable that could be involved in such a model in every scenario, and then create one uber, ultimate, super model that works in all situations. Over time and through lots of trial and error, both approaches have been found to be bullshit. Instead of giving up, or creating more and more complex models that take more and more time to work, with each project you're better off starting with the most ludicrously simple model you can and then adding complexity as needed. The trick is incremental complexity, repetition, and convergence. If you have that nailed, the details of your actual model, oddly enough, do not matter that much. Obligatory link to previous comment: https://news.ycombinator.com/item?id=6389227 https://news.ycombinator.com/item?id=6389227

jacques_chester 13y ago
I agree that the estimation process needs to be tuned to fit the local context. There's no point building a 50-parameter statistical model for building a 3-page website. But likewise, SWAGs aren't appropriate for megaprojects. A more involved process is called for.
- wpietri 13y ago
  Or one can just not do megaprojects. An instructive example comes from one of Mary Poppendieck's books. The Empire State Building was never planned and estimated in the traditional sense. They picked a deadline and used a flow-based approach to make it happen. They were already building the lower floors before the upper floors were designed. One of the problems they faced was that nobody had designed an electrical system nearly that size. So they split the building down the middle vertically and in three slices horizontally. That gave them 6 30-story buildings to wire, an understood problem. Another good example is the Internet: a globe-spanning network that connects a large fraction of humanity. There is no central control, no central design, no plan. Much larger than any megaproject, but done without a megaproject's methods. I agree that megaprojects use more involved methods, but I think "called for" is, as yet, unproven. I think it's more about the people and the social structures making the decisions than it is about what's being created. When all you have is primates, everything looks like a primate dominance hierarchy.
  - jacques_chester 13y ago
    > The Empire State Building was never planned and estimated in the traditional sense. They picked a deadline and used a flow-based approach to make it happen. Industrial projects tend to have operability thresholds; that is, they're not composed of relatively homogenous outputs. Each floor of a skyscraper is similar to the other floors. But a petrochemical process plant can't be built in slices; you have to a minimum amount of design, planning and construction simply to get to the point of turning it on. > There is no central control, no central design, no plan. You may have heard of the Internet Engineering Taskforce. I agree that many systems have useful emergent properties born of simple rules; and where possible this should be used. Or rather, where such systems are found to already exist, they should be left largely alone. However, sometimes solving a problem with interacting agents is more costly than simply picking the straight solution. If you can get an answer with a simple differential equation, then it's a waste of time to set up a particle swarm optimiser.
    - wpietri 13y ago
      > But a petrochemical process plant can't be built in slices; you have to a minimum amount of design, planning and construction simply to get to the point of turning it on. Poppendieck also gives examples of using similar processes for 3M's manufacturing plants. Regardless, I think drawing physical analogies for software is risky; software is infinitely soft. > You may have heard of the Internet Engineering Taskforce. Yes, I have. And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does.
      - jacques_chester 13y ago
        > Regardless, I think drawing physical analogies for software is risky; software is infinitely soft. Sure, but I also think that this doesn't imply infinite intractability for actual problems. That a problem is NP-hard, for example, doesn't mean we can't find quite-good solutions to it that have business value. > And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does. You said that the internet was not designed. The protocols didn't evolve without supervision. Every part of them was designed for a purpose. The question here is whether you think I'm saying "complex systems with emergent properties can be estimated or planned". That's not what I'm saying. I'm saying that not all problems are complex and not all problem systems have emergent properties. Many problems are eminently suitable for estimation. It does not follow that since in some cases estimation is going to provide very little net business value that we ought to do away with it in all cases. Could you elaborate on the 3M example, or provide a link? I'd like to read more.
dreamfactory 13y ago
Interestingly, Thoughtworks have observed that over a project they get the same consistency simply counting number of tasks as counting effort estimates for those tasks. The key for project management is therefore to focus on maintaining a backlog of manageable tasks and throughput, not in estimating them individually.
- jacques_chester 13y ago
  That's an interesting finding. Do you have a link to the research?
  - dreamfactory 13y ago
    Think it's in here - http://info.thoughtworks.com/how-do-you-estimate-on-an-Agile-project.html http://info.thoughtworks.com/how-do-you-estimate-on-an-Agile...
- DanielBMarkham 13y ago
  We're finding this out in a lot of places. The key issue is variability. Larger sample sizes usually lead to less variability, which means flow-based systems will hold up better. If you've broken your work out into tasks, instead of stories, presumably you have a lot of them, and they're mostly the same size over a large set. So sure, should work fine. If, however, you have a small number of highly-variable chunks of work, then flow-based systems fail. It all depends on the nature of the item pool.
  - jacques_chester 13y ago
    Summation can apparently reduce error without, actually, reducing error. I wrote an article about different ways of measuring estimate accuracy[1] that explains the difference. It's a question for each business as to whether that matters. [1] http://confidest.com/articles/how-accurate-was-that-estimate/ http://confidest.com/articles/how-accurate-was-that-estimate...

jacques_chester 13y ago

I agree that the estimation process needs to be tuned to fit the local context. There's no point building a 50-parameter statistical model for building a 3-page website. But likewise, SWAGs aren't appropriate for megaprojects. A more involved process is called for.

wpietri 13y ago
Or one can just not do megaprojects. An instructive example comes from one of Mary Poppendieck's books. The Empire State Building was never planned and estimated in the traditional sense. They picked a deadline and used a flow-based approach to make it happen. They were already building the lower floors before the upper floors were designed. One of the problems they faced was that nobody had designed an electrical system nearly that size. So they split the building down the middle vertically and in three slices horizontally. That gave them 6 30-story buildings to wire, an understood problem. Another good example is the Internet: a globe-spanning network that connects a large fraction of humanity. There is no central control, no central design, no plan. Much larger than any megaproject, but done without a megaproject's methods. I agree that megaprojects use more involved methods, but I think "called for" is, as yet, unproven. I think it's more about the people and the social structures making the decisions than it is about what's being created. When all you have is primates, everything looks like a primate dominance hierarchy.
- jacques_chester 13y ago
  > The Empire State Building was never planned and estimated in the traditional sense. They picked a deadline and used a flow-based approach to make it happen. Industrial projects tend to have operability thresholds; that is, they're not composed of relatively homogenous outputs. Each floor of a skyscraper is similar to the other floors. But a petrochemical process plant can't be built in slices; you have to a minimum amount of design, planning and construction simply to get to the point of turning it on. > There is no central control, no central design, no plan. You may have heard of the Internet Engineering Taskforce. I agree that many systems have useful emergent properties born of simple rules; and where possible this should be used. Or rather, where such systems are found to already exist, they should be left largely alone. However, sometimes solving a problem with interacting agents is more costly than simply picking the straight solution. If you can get an answer with a simple differential equation, then it's a waste of time to set up a particle swarm optimiser.
  - wpietri 13y ago
    > But a petrochemical process plant can't be built in slices; you have to a minimum amount of design, planning and construction simply to get to the point of turning it on. Poppendieck also gives examples of using similar processes for 3M's manufacturing plants. Regardless, I think drawing physical analogies for software is risky; software is infinitely soft. > You may have heard of the Internet Engineering Taskforce. Yes, I have. And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does.
    - jacques_chester 13y ago
      > Regardless, I think drawing physical analogies for software is risky; software is infinitely soft. Sure, but I also think that this doesn't imply infinite intractability for actual problems. That a problem is NP-hard, for example, doesn't mean we can't find quite-good solutions to it that have business value. > And either you don't know what they do or you're drawing a false analogy between traditional planning processes and what the IETF does. You said that the internet was not designed. The protocols didn't evolve without supervision. Every part of them was designed for a purpose. The question here is whether you think I'm saying "complex systems with emergent properties can be estimated or planned". That's not what I'm saying. I'm saying that not all problems are complex and not all problem systems have emergent properties. Many problems are eminently suitable for estimation. It does not follow that since in some cases estimation is going to provide very little net business value that we ought to do away with it in all cases. Could you elaborate on the 3M example, or provide a link? I'd like to read more.

dreamfactory 13y ago

Interestingly, Thoughtworks have observed that over a project they get the same consistency simply counting number of tasks as counting effort estimates for those tasks. The key for project management is therefore to focus on maintaining a backlog of manageable tasks and throughput, not in estimating them individually.

jacques_chester 13y ago
That's an interesting finding. Do you have a link to the research?
- dreamfactory 13y ago
  Think it's in here - http://info.thoughtworks.com/how-do-you-estimate-on-an-Agile-project.html http://info.thoughtworks.com/how-do-you-estimate-on-an-Agile...
DanielBMarkham 13y ago
We're finding this out in a lot of places. The key issue is variability. Larger sample sizes usually lead to less variability, which means flow-based systems will hold up better. If you've broken your work out into tasks, instead of stories, presumably you have a lot of them, and they're mostly the same size over a large set. So sure, should work fine. If, however, you have a small number of highly-variable chunks of work, then flow-based systems fail. It all depends on the nature of the item pool.
- jacques_chester 13y ago
  Summation can apparently reduce error without, actually, reducing error. I wrote an article about different ways of measuring estimate accuracy[1] that explains the difference. It's a question for each business as to whether that matters. [1] http://confidest.com/articles/how-accurate-was-that-estimate/ http://confidest.com/articles/how-accurate-was-that-estimate...