4 ms·

I'd love to hear a post mortem blog post on a project that was ran in this way in real life & what the pros/cons at different stages were. The author first tal

I'd love to hear a post mortem blog post on a project that was ran in this way in real life & what the pros/cons at different stages were. The author first talks about the waste of time that is upfront analysis/planning or adding a buffer for contingency and how Monte Carlo simulation fixes for that, but then the caveats seem to be based on an established team, of a fixed size doing similar style work. For me, this stage of a project has never faced big hiccups in estimation & our teams are normally quite good at knowing "what's a 2 vs a 5 point story" — if there _is_ a big discussion in a planning meeting at that stage, it's normally because one person has overlooked a key requirement (or the other is assuming a requirement that doesn't exist) so it's a good opportunity to solve for that. To use the analogy, we're probably quite good in established teams at estimating how long it'll take to write 10 blog posts. The trouble comes when someone says "The latest blog posts aren't generating enough interest. Can you estimate how long it'd take for you to write 10 poems instead? We want a mix of iambic pentameter and maybe some trochaic tetrameter (TBC). We've gotten budget approval to increase the size of the team, so we'll have 5 new people (off-shore) joining next week so we can go faster. Also, some of the old blog posts seem to be getting lots of visitors in South America, so we may need to translate these to Portuguese/Spanish too. When can you give me an estimate for all that?"

srparish 5y ago
The book "How to Measure Anything" by Douglas W. Hubbard has a chapter on using Monte Carlo simulations to do project planning and gather estimates. It's successfully been used for project planning large complex projects like building nuclear power plants, or projects that NASA or the Navy or similar have done. The approach is slightly different then the article above describes. Instead it has each engineer go through calibration exercises until they can fairly accurately produce 5th and 95th confidence interval estimates. Then each engineer provides 5th and 95th confidence interval estimates for each item that needs to be worked on. Those confidence intervals are kept separate. You can then run a Monte Carlo simulation where each piece of work is weighted randomly assigned to each engineer that provided estimates for that particular item, and randomly picks a number based off their provided confidence interval estimate on how long it ends up taking them to complete the item for this particular simulation. I was on a small team that used the above technique. We were in a large company trying to launch a new product, and given manufacturing lead time, and seasonality of the market demand, it was very important that we could provide a good estimate to the business when we could have the software portion of the MVP completed. The business provided us with what they thought the MVP features were, we further added in engineering tasks that weren't business facing, but needed to be completed. We confidence interval estimated those, and then also confidence interval estimated our personal vacation days, sick days, as well as a bucket of "unidentified work". The 80th percentile Monte Carlo simulation put us out a little more then a year. Our actual delivery was off by only a week from the Monte Carlo tp80; I don't remember in which direction, but it wasn't consequential to the business.

srparish 5y ago

The book "How to Measure Anything" by Douglas W. Hubbard has a chapter on using Monte Carlo simulations to do project planning and gather estimates. It's successfully been used for project planning large complex projects like building nuclear power plants, or projects that NASA or the Navy or similar have done. The approach is slightly different then the article above describes. Instead it has each engineer go through calibration exercises until they can fairly accurately produce 5th and 95th confidence interval estimates. Then each engineer provides 5th and 95th confidence interval estimates for each item that needs to be worked on. Those confidence intervals are kept separate. You can then run a Monte Carlo simulation where each piece of work is weighted randomly assigned to each engineer that provided estimates for that particular item, and randomly picks a number based off their provided confidence interval estimate on how long it ends up taking them to complete the item for this particular simulation. I was on a small team that used the above technique. We were in a large company trying to launch a new product, and given manufacturing lead time, and seasonality of the market demand, it was very important that we could provide a good estimate to the business when we could have the software portion of the MVP completed. The business provided us with what they thought the MVP features were, we further added in engineering tasks that weren't business facing, but needed to be completed. We confidence interval estimated those, and then also confidence interval estimated our personal vacation days, sick days, as well as a bucket of "unidentified work". The 80th percentile Monte Carlo simulation put us out a little more then a year. Our actual delivery was off by only a week from the Monte Carlo tp80; I don't remember in which direction, but it wasn't consequential to the business.