5 ms·

It isn't just a "BigCorp" organizational issue. You touched on it, but gently: developers share some blame here. There's a real anti-engineering bias in this in

by sidlls 9y ago

It isn't just a "BigCorp" organizational issue. You touched on it, but gently: developers share some blame here. There's a real anti-engineering bias in this industry. I mean engineering as it's practiced in other contexts. There is a process of design and analysis that is done before build and implementation starts that in this industry is considered wasteful or work to be done by lesser mortals who aren't smart enough to code. That's a big reason why there's all this over-engineering of everything.

jerf 9y ago
I am honestly having trouble parsing through how you simultaneously accuse engineers of being too lazy to use process and then agreeing that there is too much process. I don't engineer as other engineering disciplines do, because my world is not like other engineering disciplines. If architects could push a button and see a skyscraper built in five minutes, then push another button to tear it down and build it with a slightly tweaked again, for basically no cost, often while clients are actually living in the building (!), you'd see them doing a lot less up-front planning from them, too. Stop having "real engineering" envy. As the characteristics of software are added to other engineering fields, they start engineering more like us, because that's the right answer. No, we are not in fact specially stupid or lazy, which is exactly the same mistake as thinking we are specially smart or industrious. We're neither, and we do the things we do for reasons every bit as good as the reasons for their processes.
- sidlls 9y ago
  I'm not accusing anyone of being lazy, rather inappropriately arrogant and consequently ignorant. Also, I don't agree there's too much process. I think there's too much bad process that isn't recognized as either bad or process. I strongly disagree with this: "As the characteristics of software are added to other engineering fields, they start engineering more like us, because that's the right answer." It's the wrong answer, and we get things like these recent Tesla and Uber incidents because of this attitude.
  - jerf 9y ago
    "recent Tesla and Uber incidents because of this attitude." That is not what I'm talking about; that's software engineers operating in non-software fields without taking in the appropriate expertise. There's nothing special about software there, excepting perhaps that because software is everywhere our field has greater exposure to that mistake, but I've read all sorts of stories where people didn't respect the expertise of people who came before them and made grave errors as a result. That's not even limited to engineering. (In fact it's a well-known common mistake of youth across the board....) Ironically, the automotive industry is actually the one I had in mind when I wrote that, because I live in Michigan, and personally know some automotive engineers, both on the design and testing sides. We've had several conversations over the years as to how their design processes have changed now that they can plug designs into the computers and do thing like finite element analysis in the computer before they build anything physically. In the 1980s, booking the crash chamber was really hard, because it was in constant use. Now it's easy to get into. They can't close it, because the real automotive engineers know they need the validation of reality, but they do many fewer crash tests than they used to. Because of the introduction of more of this software into their process, they're doing a lot less of the up-front planning that you might think they are doing, which they used to do, in favor of more exploratory engineering and experimentation in the computer before making anything physical. And you can see the results in modern cars... if you know how to look, anyhow. A lot of it is invisible to the end consumer [1], and some of the engineering gains get somewhat eaten up by additional requirements for safety and such, but a modern car is not just "sorta nicer than they used to be", they are orders of magnitude more sophisticated than they were 30 or 40 years ago, and a lot of that is a direct impact of the fact that they are now able to adopt more software-engineering-like processes into some parts of their engineering. If it weren't for that power, we'd never have cost-effective cars that also meet the safety and emissions requirements. [1]: "This car is quieter on the road because this strut is here instead of 1.5 cm closer to the front, which is why it has this weird bend in it." Not even an exaggeration. If you look around, you'll also see these weird bits of gluey stuff stuck to a piece of metal or something that doesn't seem to even be gluing something. Those are often for noise or vibration. Some of them are empirically discovered during testing, some during simulation. There's all kinds of little details like that that you don't notice if you don't know what you're looking for.
- TheOtherHobbes 9y ago
  >If architects could push a button and see a skyscraper built in five minutes, then push another button to tear it down and build it with a slightly tweaked again, for basically no cost, often while clients are actually living in the building (!), you'd see them doing a lot less up-front planning from them, too. Does this accurately reflect how software development works in the real world?
  - dragonwriter 9y ago
    > Does this accurately reflect how software development works in the real world? It accurately reflects what is both technically possible for software in general, as well as actual practice in many places. It's true that lots of places have heavier process that treats software engineering more like civil engineering. This is sometimes for good reasons based on the consequences of errors in the application domain, or because the software is a component of a hardware system that will be deployed in conditions where easy upgrades aren't practical; on the other hand, it's often not.
  - jerf 9y ago
    How often do you iterate through the edit-build-test cycle? If you're not able to do it in under a minute, you may want to invest some attention to your development processes. At a slightly higher level, CI integration should be building your product from scratch at the same rate that commits come in for projects in the 1-5 devs, and you can do a lot of building in 5 minutes. With continuous deployment you get the "clients living in the building at the time" part of the metaphor. I don't have anything like that myself, but it's certainly a thing. Cities would look a lot different if things were this changeable in the real world.
GiorgioG 9y ago
> There's a real anti-engineering bias in this industry. What we do is hardly real engineering in most cases. I don't mean medical or transportation systems. In BigCorp, most of us write line of business applications, which will generally speaking not directly kill anyone - unlike designing/engineering bridges, roof systems, etc.
- subway 9y ago
  I've watched the "my product isn't safety critical" line of thinking utterly fuck some companies. Any recommendations on which of your projects I can keep an eye on?
  - GiorgioG 9y ago
    I've done work for organizations where the practices ranged from extremely strict processes, to no process at all (from requirements gathering to devops.) I feel my best experiences were always in places that had some process that was developed over time, that had management in place that was willing to experiment with changes to said processes. But the way Agile is bought into at most big companies these days...it feels like management just wants to be lazy: "follow the process because hey that's what these smart guys say worked over in org X" even if it's a waste of time/resources/etc.
- zbentley 9y ago
  Part of the reason this becomes so dangerous is myopia. If I'm "just building an internal issue tracker", sure, I don't need to spend ages securing it . . . until someone else exposes it to the open internet because that's the only way to make it available to offshore partners (which in turn is because someone in networking made the--from their perspective reasonable--decision not to pay for the setup of a VPN because it was expensive). Each person is making more or less reasonable decisions, but the way they interact with the rest of the whole is what causes disasters and inflicts real harm on people's lives. A question I'm interested in is: "real" engineering has the same issue of scale and potential myopia. Consider the guy mixing concrete at the batch plant on a tight schedule who, faced with an impending shortage of aggregate, mixes a little less into his mud--he does this so the trucks lining up to fill won't have to wait for the other truck bringing more aggregate. Now, his concrete was headed for a bridge, which is genuinely structurally compromised by the lack of aggregate. Do things like this happen in other fields of engineering with anywhere near the same frequency they happen in software? Why or why not? (In the concrete example--and, to be honest, this is just 'cause I like talking about concrete--there is no "right" answer given the shortage. "Fatten" the mud by lacking aggregate? Structural issues. Let trucks wait? Jobs with unfinished pours may have premature cementing, which, unless their contractors are willing to bite the "tear down the partial pour; it's too cemented--wreck it, rebuild the forms, start over" bullet which pretty much always ends up pointed at them and not the batch plant, leads to . . . you guessed it, structural issues).
jdmichal 9y ago
> There is a process of design and analysis that is done before build and implementation starts that in this industry is considered wasteful or work to be done by lesser mortals who aren't smart enough to code. That's called "backlog refinement" in scrum. And it's supposed to be done by the engineers.
commandlinefan 9y ago
You make it sound as though I could do all of this up-front design and analysis work, and it would all be time well spent, and it would save time and effort in the long run, and it would produce a better end-product and the customers and my boss would be happier with the results, but I'm not doing that because... I'm just difficult that way? I'm a lazy, coddled, spoiled, petulant developer who'd rather play with his toys than put on his big-boy pants and do some real work? Pardon me while I bristle with indignation here. First of all, under the repressive regime of "agile" methodologies which demand that I: say exactly what I'm going to do and exactly how long it's going to take and provide ongoing evidence that I did exactly what I was going to do and that it took exactly how long I said it was going to take, have something to demo every single morning to prove that I wasn't wasting the whole day goofing around on Facebook, something open-ended like design is an impossible pipe dream. What does a "design" even look like? A word document? A visio diagram? A powerpoint? In this micromanaged, "I know that you programmers can accomplish any task of any difficulty of any level of lack of definition in any compressed timeline I can think of, but I know you're not doing it because you're a bunch of overeducated spoiled brats" management environment, we've gotten the message loud and clear that we had better do whatever produces what appears to be working software as fast as possible, regardless of how inefficient or anachronistic it might be because in spite of the supposed "talent shortage", every one of us is one angry boss away from unemployment and a line of skeptical prospective employers who don't care how much education or experience you have, but do you know Angular 5? Because we only use Angular 5.
- TheCoelacanth 9y ago
  WTF kind of Agile process is that? I've never heard of anyone having daily demos. The processes I've seen schedule them for every 2-3 weeks. If you're referring stand-up meetings, the people involved in those aren't supposed to be managers, they are supposed to be the people who you are directly working with.
  - DanielBMarkham 9y ago
    Sure. You can have daily sprints if you like --- if you know what you're doing. The rule with frameworks is this: the more time you spend thinking and screwing around with the framework, the more you're either doing it wrong or it's hurting more than helping. Scrum/Agile should just work, just like your C compiler just works. It's a tool that operates almost invisibly in the background while you make stuff people want. If it takes a long time to work with it, or you spend a lot of time fussing with it or arguing over whether you're doing it right or not? You've got something backwards somewhere. People end up with long sprints and tons of meetings either usually because they don't understand what they're doing or they're doing it the wrong way. Personally I prefer week or less sprints. Anything else tends to grow a bunch of meeting BS over time that's non-productive.

by sidlls 9y ago

jerf 9y ago

I am honestly having trouble parsing through how you simultaneously accuse engineers of being too lazy to use process and then agreeing that there is too much process. I don't engineer as other engineering disciplines do, because my world is not like other engineering disciplines. If architects could push a button and see a skyscraper built in five minutes, then push another button to tear it down and build it with a slightly tweaked again, for basically no cost, often while clients are actually living in the building (!), you'd see them doing a lot less up-front planning from them, too. Stop having "real engineering" envy. As the characteristics of software are added to other engineering fields, they start engineering more like us, because that's the right answer. No, we are not in fact specially stupid or lazy, which is exactly the same mistake as thinking we are specially smart or industrious. We're neither, and we do the things we do for reasons every bit as good as the reasons for their processes.

sidlls 9y ago
I'm not accusing anyone of being lazy, rather inappropriately arrogant and consequently ignorant. Also, I don't agree there's too much process. I think there's too much bad process that isn't recognized as either bad or process. I strongly disagree with this: "As the characteristics of software are added to other engineering fields, they start engineering more like us, because that's the right answer." It's the wrong answer, and we get things like these recent Tesla and Uber incidents because of this attitude.
- jerf 9y ago
  "recent Tesla and Uber incidents because of this attitude." That is not what I'm talking about; that's software engineers operating in non-software fields without taking in the appropriate expertise. There's nothing special about software there, excepting perhaps that because software is everywhere our field has greater exposure to that mistake, but I've read all sorts of stories where people didn't respect the expertise of people who came before them and made grave errors as a result. That's not even limited to engineering. (In fact it's a well-known common mistake of youth across the board....) Ironically, the automotive industry is actually the one I had in mind when I wrote that, because I live in Michigan, and personally know some automotive engineers, both on the design and testing sides. We've had several conversations over the years as to how their design processes have changed now that they can plug designs into the computers and do thing like finite element analysis in the computer before they build anything physically. In the 1980s, booking the crash chamber was really hard, because it was in constant use. Now it's easy to get into. They can't close it, because the real automotive engineers know they need the validation of reality, but they do many fewer crash tests than they used to. Because of the introduction of more of this software into their process, they're doing a lot less of the up-front planning that you might think they are doing, which they used to do, in favor of more exploratory engineering and experimentation in the computer before making anything physical. And you can see the results in modern cars... if you know how to look, anyhow. A lot of it is invisible to the end consumer [1], and some of the engineering gains get somewhat eaten up by additional requirements for safety and such, but a modern car is not just "sorta nicer than they used to be", they are orders of magnitude more sophisticated than they were 30 or 40 years ago, and a lot of that is a direct impact of the fact that they are now able to adopt more software-engineering-like processes into some parts of their engineering. If it weren't for that power, we'd never have cost-effective cars that also meet the safety and emissions requirements. [1]: "This car is quieter on the road because this strut is here instead of 1.5 cm closer to the front, which is why it has this weird bend in it." Not even an exaggeration. If you look around, you'll also see these weird bits of gluey stuff stuck to a piece of metal or something that doesn't seem to even be gluing something. Those are often for noise or vibration. Some of them are empirically discovered during testing, some during simulation. There's all kinds of little details like that that you don't notice if you don't know what you're looking for.
TheOtherHobbes 9y ago
>If architects could push a button and see a skyscraper built in five minutes, then push another button to tear it down and build it with a slightly tweaked again, for basically no cost, often while clients are actually living in the building (!), you'd see them doing a lot less up-front planning from them, too. Does this accurately reflect how software development works in the real world?
- dragonwriter 9y ago
  > Does this accurately reflect how software development works in the real world? It accurately reflects what is both technically possible for software in general, as well as actual practice in many places. It's true that lots of places have heavier process that treats software engineering more like civil engineering. This is sometimes for good reasons based on the consequences of errors in the application domain, or because the software is a component of a hardware system that will be deployed in conditions where easy upgrades aren't practical; on the other hand, it's often not.
- jerf 9y ago
  How often do you iterate through the edit-build-test cycle? If you're not able to do it in under a minute, you may want to invest some attention to your development processes. At a slightly higher level, CI integration should be building your product from scratch at the same rate that commits come in for projects in the 1-5 devs, and you can do a lot of building in 5 minutes. With continuous deployment you get the "clients living in the building at the time" part of the metaphor. I don't have anything like that myself, but it's certainly a thing. Cities would look a lot different if things were this changeable in the real world.

GiorgioG 9y ago

> There's a real anti-engineering bias in this industry. What we do is hardly real engineering in most cases. I don't mean medical or transportation systems. In BigCorp, most of us write line of business applications, which will generally speaking not directly kill anyone - unlike designing/engineering bridges, roof systems, etc.

subway 9y ago
I've watched the "my product isn't safety critical" line of thinking utterly fuck some companies. Any recommendations on which of your projects I can keep an eye on?
- GiorgioG 9y ago
  I've done work for organizations where the practices ranged from extremely strict processes, to no process at all (from requirements gathering to devops.) I feel my best experiences were always in places that had some process that was developed over time, that had management in place that was willing to experiment with changes to said processes. But the way Agile is bought into at most big companies these days...it feels like management just wants to be lazy: "follow the process because hey that's what these smart guys say worked over in org X" even if it's a waste of time/resources/etc.
zbentley 9y ago
Part of the reason this becomes so dangerous is myopia. If I'm "just building an internal issue tracker", sure, I don't need to spend ages securing it . . . until someone else exposes it to the open internet because that's the only way to make it available to offshore partners (which in turn is because someone in networking made the--from their perspective reasonable--decision not to pay for the setup of a VPN because it was expensive). Each person is making more or less reasonable decisions, but the way they interact with the rest of the whole is what causes disasters and inflicts real harm on people's lives. A question I'm interested in is: "real" engineering has the same issue of scale and potential myopia. Consider the guy mixing concrete at the batch plant on a tight schedule who, faced with an impending shortage of aggregate, mixes a little less into his mud--he does this so the trucks lining up to fill won't have to wait for the other truck bringing more aggregate. Now, his concrete was headed for a bridge, which is genuinely structurally compromised by the lack of aggregate. Do things like this happen in other fields of engineering with anywhere near the same frequency they happen in software? Why or why not? (In the concrete example--and, to be honest, this is just 'cause I like talking about concrete--there is no "right" answer given the shortage. "Fatten" the mud by lacking aggregate? Structural issues. Let trucks wait? Jobs with unfinished pours may have premature cementing, which, unless their contractors are willing to bite the "tear down the partial pour; it's too cemented--wreck it, rebuild the forms, start over" bullet which pretty much always ends up pointed at them and not the batch plant, leads to . . . you guessed it, structural issues).

jdmichal 9y ago

> There is a process of design and analysis that is done before build and implementation starts that in this industry is considered wasteful or work to be done by lesser mortals who aren't smart enough to code. That's called "backlog refinement" in scrum. And it's supposed to be done by the engineers.

commandlinefan 9y ago

You make it sound as though I could do all of this up-front design and analysis work, and it would all be time well spent, and it would save time and effort in the long run, and it would produce a better end-product and the customers and my boss would be happier with the results, but I'm not doing that because... I'm just difficult that way? I'm a lazy, coddled, spoiled, petulant developer who'd rather play with his toys than put on his big-boy pants and do some real work? Pardon me while I bristle with indignation here. First of all, under the repressive regime of "agile" methodologies which demand that I: say exactly what I'm going to do and exactly how long it's going to take and provide ongoing evidence that I did exactly what I was going to do and that it took exactly how long I said it was going to take, have something to demo every single morning to prove that I wasn't wasting the whole day goofing around on Facebook, something open-ended like design is an impossible pipe dream. What does a "design" even look like? A word document? A visio diagram? A powerpoint? In this micromanaged, "I know that you programmers can accomplish any task of any difficulty of any level of lack of definition in any compressed timeline I can think of, but I know you're not doing it because you're a bunch of overeducated spoiled brats" management environment, we've gotten the message loud and clear that we had better do whatever produces what appears to be working software as fast as possible, regardless of how inefficient or anachronistic it might be because in spite of the supposed "talent shortage", every one of us is one angry boss away from unemployment and a line of skeptical prospective employers who don't care how much education or experience you have, but do you know Angular 5? Because we only use Angular 5.

TheCoelacanth 9y ago
WTF kind of Agile process is that? I've never heard of anyone having daily demos. The processes I've seen schedule them for every 2-3 weeks. If you're referring stand-up meetings, the people involved in those aren't supposed to be managers, they are supposed to be the people who you are directly working with.
- DanielBMarkham 9y ago
  Sure. You can have daily sprints if you like --- if you know what you're doing. The rule with frameworks is this: the more time you spend thinking and screwing around with the framework, the more you're either doing it wrong or it's hurting more than helping. Scrum/Agile should just work, just like your C compiler just works. It's a tool that operates almost invisibly in the background while you make stuff people want. If it takes a long time to work with it, or you spend a lot of time fussing with it or arguing over whether you're doing it right or not? You've got something backwards somewhere. People end up with long sprints and tons of meetings either usually because they don't understand what they're doing or they're doing it the wrong way. Personally I prefer week or less sprints. Anything else tends to grow a bunch of meeting BS over time that's non-productive.