9 ms·

While overall the article is interesting, I'm really disgusted by two sentences: "In software it’s common to have something estimated at a day take a week inste

While overall the article is interesting, I'm really disgusted by two sentences: "In software it’s common to have something estimated at a day take a week instead." and "The inherent inaccuracy of software estimates creates extra tension that you don’t find in other industries." Oh, poor thing, you feel so special by being a software engineer? Have you never heard of construction projects taking many times the estimated time/budget? Maybe try digging a tunnel through a mountain, where the kind of rocks and caveats you find on the way can make you overrun your estimates? Maybe try sailing a container ship through an ocean, where a storm that appears in the middle can make your straight trip take a week more because you must go around? And we are not talking yet about building a subway line under a city, dealing with legal challenges, ... Wake up, this is called a project, this is called risk assessment and risk management. It's maybe not done in your projects, but you don't need a fancy name for it, it's what has been happening for millennias.

jdmichal 8y ago
It also does not help at all that software "estimates" are typically just sticking a finger in the air and seeing how hard the wind is blowing at the moment. How many engineers are doing things like using real numbers from previous work to build models before estimating? Do we really think that every project is such a unique snowflake that we can't build models? You don't even need that much data, as long as you're understanding that less data means less certainty. Which is also why every estimate should be either a range, or a single number with a confidence. "Previous work shows we can handle a project of this size in 6 months 85% of the time." Of course, this is not all on engineers either. I'm sure everyone has seen the project manager who takes estimates as commitments. Cultures that allow that to happen will see repressing of generally good estimates. (If you use modeling, you just boost your confidence interval appropriately. 6 months at 85% or 9 months at 95%? Pick which number you give based on your experience with the PM!)
- xb1010 8y ago
  Most projects I work on indeed consider themselves unique disruptive snowflakes. It is impossible to give estimates for such projects, since established prior art is frowned upon.
- humanrebar 8y ago
  > Previous work shows we can handle a project of this size in 6 months 85% of the time More like: Previous work shoes that this project should take between one week and eighteen months at a 95% confidence interval. ...there aren't many projects one can do 300 times to narrow that window. And asking for a date instead of a confidence interval is asking for an ambiguous estimate in the first place.
  - jdmichal 8y ago
    Another tenet is that work is work. You attempt to break the work into chunks that are close enough to previous work efforts to use your model. If you can't do that, then you don't actually have any idea of what you're doing yet. Which is fine, but you certainly can't estimate in that state.
- Cederfjard 8y ago
  Is this how you/your team/your company work? Do you know of any resources you would recommend for those of us who are interested in learning more?
  - jdmichal 8y ago
    This is generally part of the "No Estimates" movement. I learned about it from an internal talk, but I will ask the speaker if he has any resources I can point you to. We are not there yet, but are moving in that direction. We weren't previously even collecting the metrics required to create models in the first place, so we didn't have any data.
- repolfx 8y ago
  The conversion of estimates into commitments isn't some exception to the rule, it is the rule which is why only fools give the business estimates for things. Also ... Do we really think that every project is such a unique snowflake that we can't build models? Well, yes. That's inherent to what we do. Every project IS a special snowflake. If it wasn't we'd just download the software and install it. Oddly enough, installing and configuring software that is pre-made is very predictable and such projects rarely overrun. The sort of projects we're talking about are where pre-existing software won't do and thus, by definition, what you need is something special.
  - ianwalter 8y ago
    Exactly. It's hilarious to imagine management coming to a dev with a problem and a set of objectives and the dev replying mockingly "What are we some kind of special snowflake that we have to satisfy a different set of requirements than an off-the-shelf solution can satisfy?".
  - jdmichal 8y ago
    Every structure and bridge and road and tunnel project is obviously unique also. Unique terrain, unique architecture, unique engineering. If the idea is to look to other engineering to improve this, again, it's not like software is some kind of special.
    - repolfx 8y ago
      They're only unique in trivial ways. Roads are made of the same material everywhere - they're different only in some strict geometric sense. Meanwhile even very ambitious tunnelling projects like the Gotthard Base Tunnel, if you watch documentaries about them, they only hit a very small number of really unexpected issues even over long periods of time. Tunnelling is an incredibly predictable business. You buy some TBMs, set them going and worst case, you hit unexpected rock types or have an unexpected malfunction. The technology is advanced but rock is the same as rock has always been. Once the project planning is over requirements don't change. The course you're on doesn't change. Workers are largely trainable and replaceable - you never find you're unexpectedly relying on some random worker somewhere deep in the org chart who is 50x more skilled than their coworkers and who just got poached by a company that pays 3x as much as you can. The users/customers have no input: they get what you deliver and a tunnel is such a simple thing talking about whether they like it or not is a nonsensical question - it's a tunnel.
- LargeWu 8y ago
  In my experience, PM's or whoever usually want estimates on the spot. They want estimates fast, but then they treat them as though they were well researched. If you tell them "there's a 75% probability we can get it done in 2 months" they hear "we can get it done in 2 months". There's also a tendency to leave out important information. I recently worked on a project where 1) more than half of the features were not in the requirements because the product manager assumed we could re-use existing functionality from a completely separate application (we couldn't); 2) half of the requirements from the new work were left out of requirements because they didn't bother to specify them; 3) a critical requirement regarding how data was to be aggregated in search results (which is the primary feature) was not included because the architect didn't bother to tell anybody about it except for a passing "hey, what if we did this instead" before going on vacation. This caused 2 weeks of re-work after we were done, that would have taken 2 days if we did it from the start. Our engineering team reviews all requirements as a team before giving estimates and this STILL happened, because we couldn't know what was missing. Good estimates begin with good requirements, and usually the people responsible for those do not end up being accountable, it ALWAYS falls on engineering.
  - JoeAltmaier 8y ago
    That serial approach (get good requirements; design and code; test) always works that way. An iterative approach can work better. Why isn't the architect in standup at least during the beginning of the project? All this could have been shortcut.
    - shanghaiaway 8y ago
      Because developers want to be left alone and also because they think meetings are unnecessary
- zerkten 8y ago
  > How many engineers are doing things like using real numbers from previous work to build models before estimating? Fogbugz created a feature called Evidence-based Scheduling (http://help.fogcreek.com/7676/evidence-based-scheduling-ebs http://help.fogcreek.com/7676/evidence-based-scheduling-ebs). It's probably the only real quantitative approach that I have seen while working on web development projects. I'm probably wrong to assume that more "technical" fields like aerospace do much, much more than this, but it was really interesting to use this with what was mostly CRUD-work. EDIT: added link.
shriver 8y ago
While I agree with you that software projects aren't unique in having difficulty trying to estimate things, I think that emotionally it's different. If you're digging a tunnel and hit a seam of unobtanium you can go to your boss and point at the seam and say 'Look at this big rock - this big rock is very hard, we had no way of knowing it was here but now we know we have a problem". That is very different from going to your boss and saying "You know I was working on X, well X is harder than I thought, it's going to take longer". Well - is X harder? Or are you just slacking off? Or maybe you just took a stupid approach? It's much more difficult to actually confirm that the cause of the delay is an external factor, rather than a reflection of the person doing the work since most likely the only person with a real understanding of exactly why something is difficult is the person doing the work. So there's a stigma attached that isn't present in other professions.
- jdmichal 8y ago
  You will never know what you don't know. That doesn't change. The question is, what do you do about it? That tunneling project probably did account encountering different materials, but maybe not for unobtainium. If the area is known for unobtainium, that's obviously a research miss. If this is the first seam ever encountered within 500 miles, that's straight up unexpected findings. Most situations usually lie somewhere in between. You can always do more research, but research is at the cost of getting things done. So you balance that, with the understanding that sometimes you will go into projects with too little research. And no matter how much research you do, you will occasionally get genuine unexpected findings while doing the work that cause you to stop and go back into research mode. EDIT: I mean, just watch those DIY shows some. Almost every episode, there is something "unexpected". Foundation work, roof work, load bearing walls, pipes and vents in walls, electrical or plumbing needs updating, etc. And it rarely ever actually effects their timeline, because they've done it enough to model that something unexpected is probably going to happen.
  - 52-6F-62 8y ago
    I wholeheartedly agree. I think it’s however more of an unacceptable problem in non technical industries who hire technical workers. For instance, risk assessment isn’t all that helpful if your deadlines are bumped up by weeks or months and the client or your coworkers or bosses have no concept of why the project “can’t just be done immediately” when you’ve planned for added weeks. Even worse when they don’t understand why they can’t suddemly throw new variables at you after all estimates have been completed like new, potentially breaking features. Last minute scope creep is a real problem in some non-technical fields because the whole software solution is often some “nerd magic” and there’s no desire to understand the problem or solution except where to click and can they have it all by the end of the week. Theoretically maybe it could be done, but there goes all window provided for unexpecteds. Admittedly, this is probably more of an organizational flaw than directly associated with the subject at hand, but a very real tangent.
jrs95 8y ago
Yeah I don’t think he meant “you don’t find in other industries” as “you don’t find in ANY other industries”. I mean, you’re “disgusted”? Really? Reading all this seriously makes me think you need a vacation or something.
- deleted 8y ago
  [deleted]
  - ofj28 8y ago
    Programmers ARE a spoiled and privileged class because you’re not breaking your body the same way to survive as carpenter And that field is also pushing safety and mental health way more than it used to To suggest there’s an equivalence to their effort and struggle as a manual laborer is self-aggrandizing nonsense itself
ellius 8y ago
Software engineers seem to be prone to this sort of thinking in general. I know I was at first. I think it's common to just not be especially familiar with the practice of science, with other types of engineering, to not have exposure to other industries and thus have no other frame of reference, and to just be young and inexperienced. It's very easy to look around at your own experience and think "this is so complex and requires immense amounts of thought and coordination" and to somehow think that's unique to your situation.
alex_anglin 8y ago
A key difference is that people have been sailing for thousands of years and tunneling for hundreds. We've been developing software for decades now and are learning how to do it much more quickly than we're learning new ways of sailing or tunneling. In that regard, it's a creative process and more akin to writing a book than the examples you mention. Sure, risk assessment and management are all well and good - when they're practiced, but when it comes to doing something new (i.e. not writing a simple CRUD app in the software world), seeing it as a mechanical process is what leads to these tensions.
- tfigment 8y ago
  This capture my view as well. In the past decade few of the projects I worked on were mechanical. New languages/frameworks lead to new classes of problems and much of the overtime is troubleshooting and working around those. Some time is saved by experience but not as much as I'd like.
mightybyte 8y ago
Yes, I get that all engineering (not just software) has inherent uncertainty in estimates. And there is always the chance for unanticipated events causing major overruns. But I still think that the uncertainty in software is significantly higher than in other engineering disciplines. Of course I don't have any scientific evidence for this claim. But there are lots of phenomena that exist in the world that don't yet have scientific support. I think that software his higher uncertainty because it is much less constrained than physical engineering. There are a relatively limited number of ways that one part of a physical system can impact other parts, and those are constrained by the laws of physics. Software systems have no such laws. It's often possible for any component to write to arbitrary locations in memory controlled by any other component. This is kind of like a rusty hinge on a 34th floor bathroom door causing a problem with a doorbell on the third floor. Those kinds of actions at afar just don't happen in the physical world and if they do, they happen via pretty well understood physical mechanisms. To make matters worse, the discipline of software engineering is less than a hundred years old as opposed to thousands of years for physical engineering. It's not about feeling special. I think there is something fundamentally different about software.
- bluGill 8y ago
  The only thing different about software is we haven't been doing it long so we don't know how to estimate. If you are writing a CRUD web app, it is just like the million others that have been written before and you should have the ability to give reasonable estimates. If you are writing a self driving car - that is new ground that only a few have attempted: there are many "unknown unknowns" that are hard to account for.
  - DougWebb 8y ago
    I've been developing CRUD apps for my company's clients for eight years. They're all superficially the same, but every single one of them has had something unique about their schema design, and they all have different business rules (in a general sense and in specific details) that make every project unique. We have a model that gives us ballpark estimates before we get into detailed analysis, but every time a customer insists on taking our ballpark estimate and treating it as a fixed-price commitment, the project becomes uncomfortably stressful.
douglaswlance 8y ago
Software is the only discipline that exclusively creates practical results via ideas. In excavation, you can see the dirt and rocks, and say "This terrain is different than we expected." and someone else can see the same terrain and say, "You are correct. Our assumptions were wrong." In software, you can't show anyone that the situation is different than your assumptions, because the assumptions themselves are the material that you're working with. Building software is the process of turning assumptions into results.
- alex_anglin 8y ago
  >>Software is the only discipline that exclusively creates practical results via ideas. You're going a bit far there. How about writing, design, art, etc...? There are plenty of creative industries out there.
  - humanrebar 8y ago
    In those examples, a rush job is qualitatively worse, not something completely useless. Half finished tunnels, bridges, and (usually) programs provide basically no value.
  - nautilus12 8y ago
    Thats the point, software is a creative endeavor with more in common with those fields. Does anyone wonder why the novelist or the artist is taking too long. Why do they get that freedom but not in software?
    - alex_anglin 8y ago
      Probably because, by and large, they work for themselves and not as often in teams.
  - douglaswlance 8y ago
    Writing, design, art, and other creative disciplines generate products. They do not create results, by which I mean, software itself generates results. You can write a book, but the book doesn't do anything. A person can read it, but the book doesn't generate results.
ofj28 8y ago
Holy hell are you angry at nothing. Big whoop you’re good at pushing hard to build software. Wake up; it’ll be obsolete in 5 years, unlike a tunnel or bridge You’re comparing apples and oranges and peddling some crazy emotional rhetoric Who are you to make suggest others have to jump so high? I’ve built homes, it was a summer job in high school 3 years in a row. Framing, plumbing, wiring. I started writing code professionally after that last summer, in 1996. The required mindset is vastly different. Mr chip on his shoulder is scolding someone else for not carrying a big enough chip.
magicalhippo 8y ago
> Maybe try digging a tunnel through a mountain, where the kind of rocks and caveats you find on the way can make you overrun your estimates? Usually the client doesn't call halfway through the project and tells you that the tunnel you're digging really needs to start 10 miles further up the valley...
- legulere 8y ago
  Stuff like that happens also in construction. For instance the BER airport: Midway through the project they decided to build two instead of one floor.
bena 8y ago
You think software developers are unique in making claims that their field is unique in some regard? A lot of us have the issue of perspective. We believe we have problems no one else has and fail to consider that others may have problems we can't perceive. Out of sight, out of mind. Also, millennia is already a plural.
pjc50 8y ago
You can combine these effects by having a construction effort that's also heavily dependent on software - see Berlin Airport for an example. The unique problems with software I think are: a) some things are genuinely novel: how long will it take to make a self-driving car? Nobody really knows. b) big business projects may have incoherent or incompatible requirements, but the interactions between these appear late in development and the effort to clear them up is huge and fundamentally sociological - it has to feed back into changing the business process.
qaq 8y ago
The thing is in projects that you use as examples it's much easier to explain what is going on as people have pretty sane frame of reference even if they are not domain experts.
jaredklewis 8y ago
I had a similar reaction. > It’s not because we’re narcissists who need special labels. It’s because people keep running into difficulty trying to square the very definition of a project (something with “defined scope, schedule and resources”) with our real world experiences. Unless someone has an example of a project that can be easily scheduled and scoped, I am going to go with us being narcissists. Whether it is writing a screenplay, interior designing the living room, designing an aircraft, or building bridge, scope and schedule are always tricky. No software required. When something has a fixed scope and can be dependably completed in a fixed amount of time, the appropriate English word is “task.”
mbrameld 8y ago
A non-tech acquaintance of mine was struggling to start a cannabis business a few years ago. When he found out how much non cannabis-specific stuff went in to starting a cannabis business he lamented that he could be a successful as Bill Gates was if he only had to worry about one thing (writing software) the way Bill had.

jdmichal 8y ago

It also does not help at all that software "estimates" are typically just sticking a finger in the air and seeing how hard the wind is blowing at the moment. How many engineers are doing things like using real numbers from previous work to build models before estimating? Do we really think that every project is such a unique snowflake that we can't build models? You don't even need that much data, as long as you're understanding that less data means less certainty. Which is also why every estimate should be either a range, or a single number with a confidence. "Previous work shows we can handle a project of this size in 6 months 85% of the time." Of course, this is not all on engineers either. I'm sure everyone has seen the project manager who takes estimates as commitments. Cultures that allow that to happen will see repressing of generally good estimates. (If you use modeling, you just boost your confidence interval appropriately. 6 months at 85% or 9 months at 95%? Pick which number you give based on your experience with the PM!)

xb1010 8y ago
Most projects I work on indeed consider themselves unique disruptive snowflakes. It is impossible to give estimates for such projects, since established prior art is frowned upon.
humanrebar 8y ago
> Previous work shows we can handle a project of this size in 6 months 85% of the time More like: Previous work shoes that this project should take between one week and eighteen months at a 95% confidence interval. ...there aren't many projects one can do 300 times to narrow that window. And asking for a date instead of a confidence interval is asking for an ambiguous estimate in the first place.
- jdmichal 8y ago
  Another tenet is that work is work. You attempt to break the work into chunks that are close enough to previous work efforts to use your model. If you can't do that, then you don't actually have any idea of what you're doing yet. Which is fine, but you certainly can't estimate in that state.
Cederfjard 8y ago
Is this how you/your team/your company work? Do you know of any resources you would recommend for those of us who are interested in learning more?
- jdmichal 8y ago
  This is generally part of the "No Estimates" movement. I learned about it from an internal talk, but I will ask the speaker if he has any resources I can point you to. We are not there yet, but are moving in that direction. We weren't previously even collecting the metrics required to create models in the first place, so we didn't have any data.
repolfx 8y ago
The conversion of estimates into commitments isn't some exception to the rule, it is the rule which is why only fools give the business estimates for things. Also ... Do we really think that every project is such a unique snowflake that we can't build models? Well, yes. That's inherent to what we do. Every project IS a special snowflake. If it wasn't we'd just download the software and install it. Oddly enough, installing and configuring software that is pre-made is very predictable and such projects rarely overrun. The sort of projects we're talking about are where pre-existing software won't do and thus, by definition, what you need is something special.
- ianwalter 8y ago
  Exactly. It's hilarious to imagine management coming to a dev with a problem and a set of objectives and the dev replying mockingly "What are we some kind of special snowflake that we have to satisfy a different set of requirements than an off-the-shelf solution can satisfy?".
- jdmichal 8y ago
  Every structure and bridge and road and tunnel project is obviously unique also. Unique terrain, unique architecture, unique engineering. If the idea is to look to other engineering to improve this, again, it's not like software is some kind of special.
  - repolfx 8y ago
    They're only unique in trivial ways. Roads are made of the same material everywhere - they're different only in some strict geometric sense. Meanwhile even very ambitious tunnelling projects like the Gotthard Base Tunnel, if you watch documentaries about them, they only hit a very small number of really unexpected issues even over long periods of time. Tunnelling is an incredibly predictable business. You buy some TBMs, set them going and worst case, you hit unexpected rock types or have an unexpected malfunction. The technology is advanced but rock is the same as rock has always been. Once the project planning is over requirements don't change. The course you're on doesn't change. Workers are largely trainable and replaceable - you never find you're unexpectedly relying on some random worker somewhere deep in the org chart who is 50x more skilled than their coworkers and who just got poached by a company that pays 3x as much as you can. The users/customers have no input: they get what you deliver and a tunnel is such a simple thing talking about whether they like it or not is a nonsensical question - it's a tunnel.
LargeWu 8y ago
In my experience, PM's or whoever usually want estimates on the spot. They want estimates fast, but then they treat them as though they were well researched. If you tell them "there's a 75% probability we can get it done in 2 months" they hear "we can get it done in 2 months". There's also a tendency to leave out important information. I recently worked on a project where 1) more than half of the features were not in the requirements because the product manager assumed we could re-use existing functionality from a completely separate application (we couldn't); 2) half of the requirements from the new work were left out of requirements because they didn't bother to specify them; 3) a critical requirement regarding how data was to be aggregated in search results (which is the primary feature) was not included because the architect didn't bother to tell anybody about it except for a passing "hey, what if we did this instead" before going on vacation. This caused 2 weeks of re-work after we were done, that would have taken 2 days if we did it from the start. Our engineering team reviews all requirements as a team before giving estimates and this STILL happened, because we couldn't know what was missing. Good estimates begin with good requirements, and usually the people responsible for those do not end up being accountable, it ALWAYS falls on engineering.
- JoeAltmaier 8y ago
  That serial approach (get good requirements; design and code; test) always works that way. An iterative approach can work better. Why isn't the architect in standup at least during the beginning of the project? All this could have been shortcut.
  - shanghaiaway 8y ago
    Because developers want to be left alone and also because they think meetings are unnecessary
zerkten 8y ago
> How many engineers are doing things like using real numbers from previous work to build models before estimating? Fogbugz created a feature called Evidence-based Scheduling (http://help.fogcreek.com/7676/evidence-based-scheduling-ebs http://help.fogcreek.com/7676/evidence-based-scheduling-ebs). It's probably the only real quantitative approach that I have seen while working on web development projects. I'm probably wrong to assume that more "technical" fields like aerospace do much, much more than this, but it was really interesting to use this with what was mostly CRUD-work. EDIT: added link.

shriver 8y ago

While I agree with you that software projects aren't unique in having difficulty trying to estimate things, I think that emotionally it's different. If you're digging a tunnel and hit a seam of unobtanium you can go to your boss and point at the seam and say 'Look at this big rock - this big rock is very hard, we had no way of knowing it was here but now we know we have a problem". That is very different from going to your boss and saying "You know I was working on X, well X is harder than I thought, it's going to take longer". Well - is X harder? Or are you just slacking off? Or maybe you just took a stupid approach? It's much more difficult to actually confirm that the cause of the delay is an external factor, rather than a reflection of the person doing the work since most likely the only person with a real understanding of exactly why something is difficult is the person doing the work. So there's a stigma attached that isn't present in other professions.

jdmichal 8y ago
You will never know what you don't know. That doesn't change. The question is, what do you do about it? That tunneling project probably did account encountering different materials, but maybe not for unobtainium. If the area is known for unobtainium, that's obviously a research miss. If this is the first seam ever encountered within 500 miles, that's straight up unexpected findings. Most situations usually lie somewhere in between. You can always do more research, but research is at the cost of getting things done. So you balance that, with the understanding that sometimes you will go into projects with too little research. And no matter how much research you do, you will occasionally get genuine unexpected findings while doing the work that cause you to stop and go back into research mode. EDIT: I mean, just watch those DIY shows some. Almost every episode, there is something "unexpected". Foundation work, roof work, load bearing walls, pipes and vents in walls, electrical or plumbing needs updating, etc. And it rarely ever actually effects their timeline, because they've done it enough to model that something unexpected is probably going to happen.
- 52-6F-62 8y ago
  I wholeheartedly agree. I think it’s however more of an unacceptable problem in non technical industries who hire technical workers. For instance, risk assessment isn’t all that helpful if your deadlines are bumped up by weeks or months and the client or your coworkers or bosses have no concept of why the project “can’t just be done immediately” when you’ve planned for added weeks. Even worse when they don’t understand why they can’t suddemly throw new variables at you after all estimates have been completed like new, potentially breaking features. Last minute scope creep is a real problem in some non-technical fields because the whole software solution is often some “nerd magic” and there’s no desire to understand the problem or solution except where to click and can they have it all by the end of the week. Theoretically maybe it could be done, but there goes all window provided for unexpecteds. Admittedly, this is probably more of an organizational flaw than directly associated with the subject at hand, but a very real tangent.

jrs95 8y ago

Yeah I don’t think he meant “you don’t find in other industries” as “you don’t find in ANY other industries”. I mean, you’re “disgusted”? Really? Reading all this seriously makes me think you need a vacation or something.

deleted 8y ago
[deleted]
- ofj28 8y ago
  Programmers ARE a spoiled and privileged class because you’re not breaking your body the same way to survive as carpenter And that field is also pushing safety and mental health way more than it used to To suggest there’s an equivalence to their effort and struggle as a manual laborer is self-aggrandizing nonsense itself

ellius 8y ago

Software engineers seem to be prone to this sort of thinking in general. I know I was at first. I think it's common to just not be especially familiar with the practice of science, with other types of engineering, to not have exposure to other industries and thus have no other frame of reference, and to just be young and inexperienced. It's very easy to look around at your own experience and think "this is so complex and requires immense amounts of thought and coordination" and to somehow think that's unique to your situation.

alex_anglin 8y ago

A key difference is that people have been sailing for thousands of years and tunneling for hundreds. We've been developing software for decades now and are learning how to do it much more quickly than we're learning new ways of sailing or tunneling. In that regard, it's a creative process and more akin to writing a book than the examples you mention. Sure, risk assessment and management are all well and good - when they're practiced, but when it comes to doing something new (i.e. not writing a simple CRUD app in the software world), seeing it as a mechanical process is what leads to these tensions.

tfigment 8y ago
This capture my view as well. In the past decade few of the projects I worked on were mechanical. New languages/frameworks lead to new classes of problems and much of the overtime is troubleshooting and working around those. Some time is saved by experience but not as much as I'd like.

mightybyte 8y ago

Yes, I get that all engineering (not just software) has inherent uncertainty in estimates. And there is always the chance for unanticipated events causing major overruns. But I still think that the uncertainty in software is significantly higher than in other engineering disciplines. Of course I don't have any scientific evidence for this claim. But there are lots of phenomena that exist in the world that don't yet have scientific support. I think that software his higher uncertainty because it is much less constrained than physical engineering. There are a relatively limited number of ways that one part of a physical system can impact other parts, and those are constrained by the laws of physics. Software systems have no such laws. It's often possible for any component to write to arbitrary locations in memory controlled by any other component. This is kind of like a rusty hinge on a 34th floor bathroom door causing a problem with a doorbell on the third floor. Those kinds of actions at afar just don't happen in the physical world and if they do, they happen via pretty well understood physical mechanisms. To make matters worse, the discipline of software engineering is less than a hundred years old as opposed to thousands of years for physical engineering. It's not about feeling special. I think there is something fundamentally different about software.

bluGill 8y ago
The only thing different about software is we haven't been doing it long so we don't know how to estimate. If you are writing a CRUD web app, it is just like the million others that have been written before and you should have the ability to give reasonable estimates. If you are writing a self driving car - that is new ground that only a few have attempted: there are many "unknown unknowns" that are hard to account for.
- DougWebb 8y ago
  I've been developing CRUD apps for my company's clients for eight years. They're all superficially the same, but every single one of them has had something unique about their schema design, and they all have different business rules (in a general sense and in specific details) that make every project unique. We have a model that gives us ballpark estimates before we get into detailed analysis, but every time a customer insists on taking our ballpark estimate and treating it as a fixed-price commitment, the project becomes uncomfortably stressful.

douglaswlance 8y ago

Software is the only discipline that exclusively creates practical results via ideas. In excavation, you can see the dirt and rocks, and say "This terrain is different than we expected." and someone else can see the same terrain and say, "You are correct. Our assumptions were wrong." In software, you can't show anyone that the situation is different than your assumptions, because the assumptions themselves are the material that you're working with. Building software is the process of turning assumptions into results.

alex_anglin 8y ago
>>Software is the only discipline that exclusively creates practical results via ideas. You're going a bit far there. How about writing, design, art, etc...? There are plenty of creative industries out there.
- humanrebar 8y ago
  In those examples, a rush job is qualitatively worse, not something completely useless. Half finished tunnels, bridges, and (usually) programs provide basically no value.
- nautilus12 8y ago
  Thats the point, software is a creative endeavor with more in common with those fields. Does anyone wonder why the novelist or the artist is taking too long. Why do they get that freedom but not in software?
  - alex_anglin 8y ago
    Probably because, by and large, they work for themselves and not as often in teams.
- douglaswlance 8y ago
  Writing, design, art, and other creative disciplines generate products. They do not create results, by which I mean, software itself generates results. You can write a book, but the book doesn't do anything. A person can read it, but the book doesn't generate results.

ofj28 8y ago

Holy hell are you angry at nothing. Big whoop you’re good at pushing hard to build software. Wake up; it’ll be obsolete in 5 years, unlike a tunnel or bridge You’re comparing apples and oranges and peddling some crazy emotional rhetoric Who are you to make suggest others have to jump so high? I’ve built homes, it was a summer job in high school 3 years in a row. Framing, plumbing, wiring. I started writing code professionally after that last summer, in 1996. The required mindset is vastly different. Mr chip on his shoulder is scolding someone else for not carrying a big enough chip.

magicalhippo 8y ago

> Maybe try digging a tunnel through a mountain, where the kind of rocks and caveats you find on the way can make you overrun your estimates? Usually the client doesn't call halfway through the project and tells you that the tunnel you're digging really needs to start 10 miles further up the valley...

legulere 8y ago
Stuff like that happens also in construction. For instance the BER airport: Midway through the project they decided to build two instead of one floor.

bena 8y ago

You think software developers are unique in making claims that their field is unique in some regard? A lot of us have the issue of perspective. We believe we have problems no one else has and fail to consider that others may have problems we can't perceive. Out of sight, out of mind. Also, millennia is already a plural.

pjc50 8y ago

You can combine these effects by having a construction effort that's also heavily dependent on software - see Berlin Airport for an example. The unique problems with software I think are: a) some things are genuinely novel: how long will it take to make a self-driving car? Nobody really knows. b) big business projects may have incoherent or incompatible requirements, but the interactions between these appear late in development and the effort to clear them up is huge and fundamentally sociological - it has to feed back into changing the business process.

qaq 8y ago

The thing is in projects that you use as examples it's much easier to explain what is going on as people have pretty sane frame of reference even if they are not domain experts.

jaredklewis 8y ago

I had a similar reaction. > It’s not because we’re narcissists who need special labels. It’s because people keep running into difficulty trying to square the very definition of a project (something with “defined scope, schedule and resources”) with our real world experiences. Unless someone has an example of a project that can be easily scheduled and scoped, I am going to go with us being narcissists. Whether it is writing a screenplay, interior designing the living room, designing an aircraft, or building bridge, scope and schedule are always tricky. No software required. When something has a fixed scope and can be dependably completed in a fixed amount of time, the appropriate English word is “task.”

mbrameld 8y ago

A non-tech acquaintance of mine was struggling to start a cannabis business a few years ago. When he found out how much non cannabis-specific stuff went in to starting a cannabis business he lamented that he could be a successful as Bill Gates was if he only had to worry about one thing (writing software) the way Bill had.