4 ms·

Estimating in the "second type of environment" is also bogus. The existence of some legal or regulatory liability doesn't really make estimations in that enviro

Estimating in the "second type of environment" is also bogus. The existence of some legal or regulatory liability doesn't really make estimations in that environment that much better. The shining example of this is Big Dig in Boston, MA: years late and 7-fold over budget. On the software side, the initial launch of Obamacare web site was a disaster. Technically not late, but severely badly designed and deployed. I know first hand that many government research (NSF, SBIR, ...) grants are chronically late or under-delivered. So I don't buy the assertion that some environments somehow make estimating any better. Humans are routinely bad at estimating, but mostly because of social and cultural reasons rather than technical or engineering reasons.

Pannoniae 3y ago
"but mostly because of social and cultural reasons rather than technical or engineering reasons" Can you please expand on that why that is the case? Also, are there ways to improve estimation skills?
- beryilma 3y ago
  Inability to say no and push back. I am guilty of it myself. Most experienced software developers know in their gut a lower bound for a project's timeline, which always exceeds the timeline in managers' or PM's mind. When the VP asks how long a project will take, it is hard to not say something optimistic. Or when your release timeline is every six months, it is hard to say a project will take more than a release cycle. These are social issues more than technical. I know a mostly accurate and simple estimation approach, which is roughly the Pareto Principle: make a good effort to build a decent prototype and multiply the time it took by 4 to get to an estimate. But of course good luck selling this to your management! My advice to improve estimations would be around: - Building a prototype before making any estimations. Software is in a lucky position where you can build prototypes before the actual thing. - Applying the Pareto Principle, if you can. The reason for multiplying by 3-4 is because you have to account for research, design reviews, implementation, testing, bug fixes, documentation, and internationalization, etc.
treflop 3y ago
I feel like you are cherry picking extremely high-profile projects that you were personally not involved in, so (1) sometimes estimations do end up wrong and mistakes can happen, so being able to pick any example and choosing the ones that failed is disingenuous (2) picking someone else’s projects means you have absolutely no insight into the process. sometimes I will very intentionally under- or over-estimate a deadline for political - or really practical - reasons and someone on the outside will think I messed up. the projects you picked are extremely high profile and have to contend with a lot of political forces and are probably the worst examples to pick
- jacques_chester 3y ago
  Bent Flyvbjerg calls (2) the "sublime". He points out that disastrous megaprojects get authorized because folks with the money and authority simply ignore the haters. https://en.wikipedia.org/wiki/Bent_Flyvbjerg#Megaproject_planning_and_management https://en.wikipedia.org/wiki/Bent_Flyvbjerg#Megaproject_pla...
- beryilma 3y ago
  I picked those projects because they were high-profile. But I'll give you another example that I am intimately involved with. Take the software/web accessibility remediation mess where universities and companies are being sued left and right for ADA (Americans with Disabilities Act) violations. These organizations know the real threat of legal liabilities, make projects to improve accessibility of their products (web sites, etc.), and routinely blow up their project timelines only to start with new projects. So the concept of "deadlines are real and have consequences" firmly applies here. Yet, NONE of these projects are completed on time. There are many reason for these project failures, but my original argument still stands in this case: "Estimating in the second type of environment is also bogus".
chaboud 3y ago
Financial and rigid business incentives (and, yes, regulatory requirements with teeth) are good forcing functions. I’m not sure “bureaucratic tangles” like the Big Dig would qualify. I’ll call this “tightness” in a program. Category two has more tight constraints than category one. I’ll give an example from my current job (more than a decade, which is crazy for Silicon Valley), where we make small devices (e.g., tablets, smart plugs, HDMI media players) and large devices (e.g., 75” televisions) as well as software-only features and products. If software slips, we might miss some business targets, so software’s tightness is highly variable. If we move on to small electronics, in a pinch, we can change initial shipment from ocean shipping to air shipping. This can buy 2-3 weeks of made up time, but at a cost of a few dollars per unit. The initial few hundred thousand devices might cost more to deliver if we slip, so we have more schedule tightness than pure software. There are de facto baseline business constraints. And then we move onto big devices like 75” TVs. It is cost prohibitive to air-ship TV’s, so we have to get at least the things necessary for initial use and updating ready in time for ocean shipment. Additionally, since large televisions are generally made where their panels are made and they are generally sold in brick and mortar stores that also serve as local warehouses (where space is planned in advance), there are commitments in manufacturing, logistics, and retail. TV’s, by nature, have more schedule tightness. In general, what I’ve observed with these sorts of constraints is that the added known downstream impacts of a program tend to call for greater scrutiny and inspection up front, leading to one of my most often cited rules of executive management: “no surprises”. I agree that there are folks who understand schedule tightness and folks who don’t (or at least, don’t yet). I think it’s less about a specific characteristic and more about broader downstream impact. I’ll note that I’ve seen people from completely different domains (e.g., automotive, aviation, military, software, cloud services, media) come in and be great at this or flub it, whether they had experience in our domain or not. Planning is a skill of its own.
nradov 3y ago
I won't attempt to excuse the failures on the Big Dig project, but the reality of politics and government budgeting is that project supporters are often forced to give unrealistically low estimates in order to get anything approved at all. Then once the inevitable delays and budget overruns start the project has already taken on a life of it's own and becomes impossible to cancel. And in the end the stakeholders are happy to have the new tunnel or fighter plane or whatever even if the final cost was way higher than they ever expected. It's a dysfunctional process, but there doesn't seem to be any other way to accomplish huge audacious goals. I wonder how late and over budget the Great Pyramid was?

Pannoniae 3y ago

"but mostly because of social and cultural reasons rather than technical or engineering reasons" Can you please expand on that why that is the case? Also, are there ways to improve estimation skills?

beryilma 3y ago
Inability to say no and push back. I am guilty of it myself. Most experienced software developers know in their gut a lower bound for a project's timeline, which always exceeds the timeline in managers' or PM's mind. When the VP asks how long a project will take, it is hard to not say something optimistic. Or when your release timeline is every six months, it is hard to say a project will take more than a release cycle. These are social issues more than technical. I know a mostly accurate and simple estimation approach, which is roughly the Pareto Principle: make a good effort to build a decent prototype and multiply the time it took by 4 to get to an estimate. But of course good luck selling this to your management! My advice to improve estimations would be around: - Building a prototype before making any estimations. Software is in a lucky position where you can build prototypes before the actual thing. - Applying the Pareto Principle, if you can. The reason for multiplying by 3-4 is because you have to account for research, design reviews, implementation, testing, bug fixes, documentation, and internationalization, etc.

treflop 3y ago

I feel like you are cherry picking extremely high-profile projects that you were personally not involved in, so (1) sometimes estimations do end up wrong and mistakes can happen, so being able to pick any example and choosing the ones that failed is disingenuous (2) picking someone else’s projects means you have absolutely no insight into the process. sometimes I will very intentionally under- or over-estimate a deadline for political - or really practical - reasons and someone on the outside will think I messed up. the projects you picked are extremely high profile and have to contend with a lot of political forces and are probably the worst examples to pick

jacques_chester 3y ago
Bent Flyvbjerg calls (2) the "sublime". He points out that disastrous megaprojects get authorized because folks with the money and authority simply ignore the haters. https://en.wikipedia.org/wiki/Bent_Flyvbjerg#Megaproject_planning_and_management https://en.wikipedia.org/wiki/Bent_Flyvbjerg#Megaproject_pla...
beryilma 3y ago
I picked those projects because they were high-profile. But I'll give you another example that I am intimately involved with. Take the software/web accessibility remediation mess where universities and companies are being sued left and right for ADA (Americans with Disabilities Act) violations. These organizations know the real threat of legal liabilities, make projects to improve accessibility of their products (web sites, etc.), and routinely blow up their project timelines only to start with new projects. So the concept of "deadlines are real and have consequences" firmly applies here. Yet, NONE of these projects are completed on time. There are many reason for these project failures, but my original argument still stands in this case: "Estimating in the second type of environment is also bogus".

chaboud 3y ago

Financial and rigid business incentives (and, yes, regulatory requirements with teeth) are good forcing functions. I’m not sure “bureaucratic tangles” like the Big Dig would qualify. I’ll call this “tightness” in a program. Category two has more tight constraints than category one. I’ll give an example from my current job (more than a decade, which is crazy for Silicon Valley), where we make small devices (e.g., tablets, smart plugs, HDMI media players) and large devices (e.g., 75” televisions) as well as software-only features and products. If software slips, we might miss some business targets, so software’s tightness is highly variable. If we move on to small electronics, in a pinch, we can change initial shipment from ocean shipping to air shipping. This can buy 2-3 weeks of made up time, but at a cost of a few dollars per unit. The initial few hundred thousand devices might cost more to deliver if we slip, so we have more schedule tightness than pure software. There are de facto baseline business constraints. And then we move onto big devices like 75” TVs. It is cost prohibitive to air-ship TV’s, so we have to get at least the things necessary for initial use and updating ready in time for ocean shipment. Additionally, since large televisions are generally made where their panels are made and they are generally sold in brick and mortar stores that also serve as local warehouses (where space is planned in advance), there are commitments in manufacturing, logistics, and retail. TV’s, by nature, have more schedule tightness. In general, what I’ve observed with these sorts of constraints is that the added known downstream impacts of a program tend to call for greater scrutiny and inspection up front, leading to one of my most often cited rules of executive management: “no surprises”. I agree that there are folks who understand schedule tightness and folks who don’t (or at least, don’t yet). I think it’s less about a specific characteristic and more about broader downstream impact. I’ll note that I’ve seen people from completely different domains (e.g., automotive, aviation, military, software, cloud services, media) come in and be great at this or flub it, whether they had experience in our domain or not. Planning is a skill of its own.

nradov 3y ago

I won't attempt to excuse the failures on the Big Dig project, but the reality of politics and government budgeting is that project supporters are often forced to give unrealistically low estimates in order to get anything approved at all. Then once the inevitable delays and budget overruns start the project has already taken on a life of it's own and becomes impossible to cancel. And in the end the stakeholders are happy to have the new tunnel or fighter plane or whatever even if the final cost was way higher than they ever expected. It's a dysfunctional process, but there doesn't seem to be any other way to accomplish huge audacious goals. I wonder how late and over budget the Great Pyramid was?