6 ms·

> I often hear people say that software engineering is a joke compared to civil engineering ... I think your example points out exactly why some think software

by clarry 7y ago

> I often hear people say that software engineering is a joke compared to civil engineering ... I think your example points out exactly why some think software engineering is a joke: they rush ass first to implement something (and they implement the wrong thing) instead of sitting down and figuring out the actual requirements first like any real engineering job would have you do. And it looks like everyone is pushing it more into this ass-first direction with agile. Let's not even try to plan ahead! You know what you want next week, so this short-sighted focus leads to chaotic development and once the real requirements are figured out 9 months into the project, well too bad it's too late to redesign and rewrite it all properly. So it's forever going to be a bumpy and crooked bridge on the verge of collapse, held together by more and more duck tape, hot glue and occult ceremony. And there's a plastic tunnel hanging underneath it. Only the brave will crawl through.

louwrentius 7y ago
Goddamn, this is exactly the sentiment I have about software development. Because it's not about bridges or tunnels, common sense is thrown out of the window, people start building something they conjure out of thin air. Few people say 'No!', we don't do anything. Anything. Until we have written a clear business case / usecase, a raison d'être for this project. Agile is often abused to not think, not design and just follow the whims of the customer. Nice for billable hours, but long term? Do you produce anything of value?
- bob33212 7y ago
  I have been in a position where I have said no. I was able to stop the feature factory for a few months. But after that we were right back to cranking on features because several end users "validated our roadmap".
- nradov 7y ago
  Santa Claus isn't going to fly down your chimney and deliver a fully formed business case. Developers share responsibility with other roles for figuring it out. Sometimes it takes multiple iterations of exploration and experimentation. Expecting to be told what to do is just lazy and unprofessional.
bmn__ 7y ago
> instead of sitting down and figuring out the actual requirements first like any real engineering job If it were so easy, then we would have done it already that way for the last seven decades. The simile topples over upon further contemplation: civil engineers: • client is not a domain expert • to an overwhelming part, client needs are easy to transport into the mind of c.eng. • can employ a wealth of standard solutions refined over the course of milleniums software authors: • client is a domain expert for the subject matter that the software is supposed to model • to an overwhelming part, client needs are difficult to express and transport into the mind of s.auth. • if a standard solution exists, the client would have already bought it off the shelf, so coming to a s.auth. always means customised development • the entire sector is still in its proverbial baby shoes We have to come to grips that we unfortunately can't apply what happens to work well for a different sector. The supposed value of non-traditional software methodology lies in that at least the client notices that the requirements are off the needs rather early.
- clarry 7y ago
  I agree it's not easy, but instead of trying harder to become good at it, it seems like this industry in its proverbial baby shoes threw a tantrum and decided that it doesn't want to learn at all. So instead of pushing for more effort & skill for up-front planning and modelling of the requirements and design (and trying to employ modelling tools & formal methods), we're just like fuckit nah let's just implement something and see where that leads us. It doesn't help that so many of the up-front design failures were not examples of engineers screwing up but sales people selling shit and then having engineers cook it. We'll be stuck in baby shoes for a long time with this approach.
  - vinceguidry 7y ago
    Companies are more than happy to churn through staff, losing millions in accumulated human knowledge, just to keep it out of the hands of employees. My company just shut down an entire office, they offered three out of four critical employees the opportunity to move to Atlanta. They didn’t want to move, opting to retire instead. Company just shrugged their shoulders rather than letting them work remote, gave them like a month to transfer knowledge and that’s that.
bostik 7y ago
A large part of the problem is that grand majority of software (at least if it's facing actual human clients) is constantly trying to search or validate market-fit. If you don't get it out first, you're going to lose to a competitor who does. And even when you do have a solid market-fit figured out, you still don't know what your customers will actually want. (And don't get me started on customer research. People lie all the time, and their accuracy on what features are honest-to-$deity blockers.. Nope.) This means that for most parts the incentives are misaligned and work actively against delivering high-quality software. Underlying the agile manifesto is the need for engineering to be properly aligned with business incentives. Continuous delivery, fast iteration cycles, incremental improvements and rapid deployments are all facets of the same fact: you don't actually know what your customers want (because your customers don't know what they want!), and the only true way to figure it out is to iterate as fast as you can. It's only when you have an established moat and a practically guaranteed, net-profitable income stream, when you can even imagine at having the opportunity of doing things the way you would assume you want to. Oddly enough.. the companies who have such moats and guaranteed profits tend to be the same ones that have been the slowest to modernise themselves. Banks - until the modern online-only challengers came along. Insurance, where the regulatory moats are even higher and is only now being "disrupted". Loan brokering. Energy. Water. Public transport (where monopolies and/or charters prevent competition). Healthcare. On and on and on... And even then, every one of these established, guaranteed-profit machines has the same problems: the moment one good new player enters the field and starts stealing customers, the old guard needs to find a way to either adapt - which is the best-case scenario - or rely on their power structures with politicians and quickly require new regulations to strangle the usurpers. Software is expensive to make. It's only cheap to deliver. And time spent making the wrong thing is really expensive. EDIT: I forgot to say this the first time around. Engineering career incentives are also misaligned. Maintenance is not appreciated, only shipping new stuff is. So the developers who would want to improve what exists and make it better will be looked down upon. The promotions and raises go to those who ship something new, even if it ends up setting the world on fire a year later.
markus_zhang 7y ago
Manager and Director are the people who are pushing "agile". They want quick results, and SE can only do what they want. You simply really can't "gather" requirement and finish a feature and do proper testing in a 2-week sprint. I do agree there are a lot of bad practices in SE as well, but let's not blame everything on them.
- clarry 7y ago
  Yeah I'm not really blaming individual software engineers (I'm in the same boat, fighting a slow trickle of feature after feature without really being given a chance to step back and consider the overall design & its future evolution), but the industry as a whole. As I said in another comment in this thread, all too often SEs are just tasked with cooking sh*t to vague spec (as laid out by sales & other non-engineers) instead of being active at the earlier stages where engineering should already begin. It's like sales people together with the customer decided the shape and colour of the bridge, as well as the number of pillars it has, and how many lanes it has, and what material it's made of -- here's the stack of things you build with -- and what other fancy features it has, and now finally we can call the engineer so they can hop on a bulldozer and start engineering with their hammer for the next two weeks and show the result so we can adjust the shape if it's not as pretty as we imagined. Actual engineering didn't happen.
- qznc 7y ago
  My understanding of Scrum is that requirement elicitation must happen before the sprint. Otherwise the story does not meet the Definition of Ready and the team should reject it. https://www.scruminc.com/definition-of-ready/ https://www.scruminc.com/definition-of-ready/
- nradov 7y ago
  It is completely possible to deliver a user story within a 2 week sprint provided that the acceptance criteria are reasonably clear. Many agile teams so this successfully. Features (or epics or whatever) are larger and contain multiple user stories. In most cases features extend over multiple sprints.
nhumrich 7y ago
Thats because we have tried it the other way. Its called waterfall. The problem is, the process is too slow in such a fast moving industry. If you are building a bridge, you dont have the problem of being half way done and all your customers say, "nah, nvm there is tunnel that just went up, that can kind of solve my needs. I will use that one instead."
- hwayne 7y ago
  Have you talked to civil engineers? I have. This happens all the time.
Kye 7y ago
People built a lot of ugly, useless, dangerous bridges before it was possible to plan and build a good one reliably. Even then, bridges still open with huge flaws. I think software engineering will get there eventually. Someone has to beat their head against the unknown a few times to map out its form.
celticninja 7y ago
Having worked on government projects, agile is definitely better than waterfall. The specs will change, better accept that early on and work with it, instead of trying to nail down the wrong thing.
wpietri 7y ago
> figuring out the actual requirements first like any real engineering job would have you do There are no "actual" requirements for most software projects. Software is generally co-evolved with the audience. Bridge designers can work like they do because bridges are hard to change and fundamentally pretty similar. But because software infinitely changeable and infinitely copyable, our field is different. If you're going to build 500 highway overpass bridges, you'll need engineers looking at each one, because local conditions and needs vary. But in software if we need to serve 500 users, we build one product, give each user a copy, and just keep adding features to solve the local problems. A big reason we think software can be like bridges is that in the early history of our profession, we were tackling problems that almost worked in a waterfall context. That's because we were taking existing paper processes and automating them. But the hard part of requirements discovery via process coevolution had been done for us by people tinkering with paper and human working relationships. That era is past. Today if the requirements for something are perfectly knowable, that's because it's either trivial or it's copying an existing piece of software. Any real product is discovering what's possible. It's growing along with its users' needs. It's solving new problems created by other people and groups. I know of no significant piece of software that, like a bridge, hit 1.0 and just stayed there for a decade while getting plenty of use because the requirements were perfectly foreseen. Not one. To me that's a sign that perfect foreknowledge is, if not impossible, at least incredibly rare. So I think the thing to do is to figure out how to build excellent software with the expectation that needs will change.
- qznc 7y ago
  There are plenty of embedded systems which hit 1.0 and stayed there for years and decades.
  - wpietri 7y ago
    In that case, I hope somebody points me at a few of them. My guess is that if we look at the circumstances, they won't hold a lot of lessons for us. As an example, the IRS is still running 60-year-old code: https://www.nextgov.com/it-modernization/2018/04/irs-60-year-old-it-system-failed-tax-day-due-new-hardware/147598/ https://www.nextgov.com/it-modernization/2018/04/irs-60-year... Is that because that code and hardware is especially good? Because the initial design and research process was so perfect that people are still entirely happy with it? I'm guessing not.
fuzzy2 7y ago
The problem is that you cannot figure out the actual requirements. In every non-trivial application project I had so far, the requirements (and there were lots of them) were simply not correct. The requirements the actual users created, that is. They _think_ they know the workflow. They don’t. If you don’t even ask the users, it’s even worse. But that’s okay. It is simply not possible to accurately judge how something will work until you get to use it. That goes for the creators, too. That’s why we iterate.
jrs235 7y ago
Posted to Hacker News a few days ago and I think relevent here: https://news.ycombinator.com/item?id=22365496 https://news.ycombinator.com/item?id=22365496

by clarry 7y ago

louwrentius 7y ago

Goddamn, this is exactly the sentiment I have about software development. Because it's not about bridges or tunnels, common sense is thrown out of the window, people start building something they conjure out of thin air. Few people say 'No!', we don't do anything. Anything. Until we have written a clear business case / usecase, a raison d'être for this project. Agile is often abused to not think, not design and just follow the whims of the customer. Nice for billable hours, but long term? Do you produce anything of value?

bob33212 7y ago
I have been in a position where I have said no. I was able to stop the feature factory for a few months. But after that we were right back to cranking on features because several end users "validated our roadmap".
nradov 7y ago
Santa Claus isn't going to fly down your chimney and deliver a fully formed business case. Developers share responsibility with other roles for figuring it out. Sometimes it takes multiple iterations of exploration and experimentation. Expecting to be told what to do is just lazy and unprofessional.

bmn__ 7y ago

> instead of sitting down and figuring out the actual requirements first like any real engineering job If it were so easy, then we would have done it already that way for the last seven decades. The simile topples over upon further contemplation: civil engineers: • client is not a domain expert • to an overwhelming part, client needs are easy to transport into the mind of c.eng. • can employ a wealth of standard solutions refined over the course of milleniums software authors: • client is a domain expert for the subject matter that the software is supposed to model • to an overwhelming part, client needs are difficult to express and transport into the mind of s.auth. • if a standard solution exists, the client would have already bought it off the shelf, so coming to a s.auth. always means customised development • the entire sector is still in its proverbial baby shoes We have to come to grips that we unfortunately can't apply what happens to work well for a different sector. The supposed value of non-traditional software methodology lies in that at least the client notices that the requirements are off the needs rather early.

clarry 7y ago
I agree it's not easy, but instead of trying harder to become good at it, it seems like this industry in its proverbial baby shoes threw a tantrum and decided that it doesn't want to learn at all. So instead of pushing for more effort & skill for up-front planning and modelling of the requirements and design (and trying to employ modelling tools & formal methods), we're just like fuckit nah let's just implement something and see where that leads us. It doesn't help that so many of the up-front design failures were not examples of engineers screwing up but sales people selling shit and then having engineers cook it. We'll be stuck in baby shoes for a long time with this approach.
- vinceguidry 7y ago
  Companies are more than happy to churn through staff, losing millions in accumulated human knowledge, just to keep it out of the hands of employees. My company just shut down an entire office, they offered three out of four critical employees the opportunity to move to Atlanta. They didn’t want to move, opting to retire instead. Company just shrugged their shoulders rather than letting them work remote, gave them like a month to transfer knowledge and that’s that.

bostik 7y ago

A large part of the problem is that grand majority of software (at least if it's facing actual human clients) is constantly trying to search or validate market-fit. If you don't get it out first, you're going to lose to a competitor who does. And even when you do have a solid market-fit figured out, you still don't know what your customers will actually want. (And don't get me started on customer research. People lie all the time, and their accuracy on what features are honest-to-$deity blockers.. Nope.) This means that for most parts the incentives are misaligned and work actively against delivering high-quality software. Underlying the agile manifesto is the need for engineering to be properly aligned with business incentives. Continuous delivery, fast iteration cycles, incremental improvements and rapid deployments are all facets of the same fact: you don't actually know what your customers want (because your customers don't know what they want!), and the only true way to figure it out is to iterate as fast as you can. It's only when you have an established moat and a practically guaranteed, net-profitable income stream, when you can even imagine at having the opportunity of doing things the way you would assume you want to. Oddly enough.. the companies who have such moats and guaranteed profits tend to be the same ones that have been the slowest to modernise themselves. Banks - until the modern online-only challengers came along. Insurance, where the regulatory moats are even higher and is only now being "disrupted". Loan brokering. Energy. Water. Public transport (where monopolies and/or charters prevent competition). Healthcare. On and on and on... And even then, every one of these established, guaranteed-profit machines has the same problems: the moment one good new player enters the field and starts stealing customers, the old guard needs to find a way to either adapt - which is the best-case scenario - or rely on their power structures with politicians and quickly require new regulations to strangle the usurpers. Software is expensive to make. It's only cheap to deliver. And time spent making the wrong thing is really expensive. EDIT: I forgot to say this the first time around. Engineering career incentives are also misaligned. Maintenance is not appreciated, only shipping new stuff is. So the developers who would want to improve what exists and make it better will be looked down upon. The promotions and raises go to those who ship something new, even if it ends up setting the world on fire a year later.

markus_zhang 7y ago

Manager and Director are the people who are pushing "agile". They want quick results, and SE can only do what they want. You simply really can't "gather" requirement and finish a feature and do proper testing in a 2-week sprint. I do agree there are a lot of bad practices in SE as well, but let's not blame everything on them.

clarry 7y ago
Yeah I'm not really blaming individual software engineers (I'm in the same boat, fighting a slow trickle of feature after feature without really being given a chance to step back and consider the overall design & its future evolution), but the industry as a whole. As I said in another comment in this thread, all too often SEs are just tasked with cooking sh*t to vague spec (as laid out by sales & other non-engineers) instead of being active at the earlier stages where engineering should already begin. It's like sales people together with the customer decided the shape and colour of the bridge, as well as the number of pillars it has, and how many lanes it has, and what material it's made of -- here's the stack of things you build with -- and what other fancy features it has, and now finally we can call the engineer so they can hop on a bulldozer and start engineering with their hammer for the next two weeks and show the result so we can adjust the shape if it's not as pretty as we imagined. Actual engineering didn't happen.
qznc 7y ago
My understanding of Scrum is that requirement elicitation must happen before the sprint. Otherwise the story does not meet the Definition of Ready and the team should reject it. https://www.scruminc.com/definition-of-ready/ https://www.scruminc.com/definition-of-ready/
nradov 7y ago
It is completely possible to deliver a user story within a 2 week sprint provided that the acceptance criteria are reasonably clear. Many agile teams so this successfully. Features (or epics or whatever) are larger and contain multiple user stories. In most cases features extend over multiple sprints.

nhumrich 7y ago

Thats because we have tried it the other way. Its called waterfall. The problem is, the process is too slow in such a fast moving industry. If you are building a bridge, you dont have the problem of being half way done and all your customers say, "nah, nvm there is tunnel that just went up, that can kind of solve my needs. I will use that one instead."

hwayne 7y ago
Have you talked to civil engineers? I have. This happens all the time.

Kye 7y ago

People built a lot of ugly, useless, dangerous bridges before it was possible to plan and build a good one reliably. Even then, bridges still open with huge flaws. I think software engineering will get there eventually. Someone has to beat their head against the unknown a few times to map out its form.

celticninja 7y ago

Having worked on government projects, agile is definitely better than waterfall. The specs will change, better accept that early on and work with it, instead of trying to nail down the wrong thing.

wpietri 7y ago

> figuring out the actual requirements first like any real engineering job would have you do There are no "actual" requirements for most software projects. Software is generally co-evolved with the audience. Bridge designers can work like they do because bridges are hard to change and fundamentally pretty similar. But because software infinitely changeable and infinitely copyable, our field is different. If you're going to build 500 highway overpass bridges, you'll need engineers looking at each one, because local conditions and needs vary. But in software if we need to serve 500 users, we build one product, give each user a copy, and just keep adding features to solve the local problems. A big reason we think software can be like bridges is that in the early history of our profession, we were tackling problems that almost worked in a waterfall context. That's because we were taking existing paper processes and automating them. But the hard part of requirements discovery via process coevolution had been done for us by people tinkering with paper and human working relationships. That era is past. Today if the requirements for something are perfectly knowable, that's because it's either trivial or it's copying an existing piece of software. Any real product is discovering what's possible. It's growing along with its users' needs. It's solving new problems created by other people and groups. I know of no significant piece of software that, like a bridge, hit 1.0 and just stayed there for a decade while getting plenty of use because the requirements were perfectly foreseen. Not one. To me that's a sign that perfect foreknowledge is, if not impossible, at least incredibly rare. So I think the thing to do is to figure out how to build excellent software with the expectation that needs will change.

qznc 7y ago
There are plenty of embedded systems which hit 1.0 and stayed there for years and decades.
- wpietri 7y ago
  In that case, I hope somebody points me at a few of them. My guess is that if we look at the circumstances, they won't hold a lot of lessons for us. As an example, the IRS is still running 60-year-old code: https://www.nextgov.com/it-modernization/2018/04/irs-60-year-old-it-system-failed-tax-day-due-new-hardware/147598/ https://www.nextgov.com/it-modernization/2018/04/irs-60-year... Is that because that code and hardware is especially good? Because the initial design and research process was so perfect that people are still entirely happy with it? I'm guessing not.

fuzzy2 7y ago

The problem is that you cannot figure out the actual requirements. In every non-trivial application project I had so far, the requirements (and there were lots of them) were simply not correct. The requirements the actual users created, that is. They _think_ they know the workflow. They don’t. If you don’t even ask the users, it’s even worse. But that’s okay. It is simply not possible to accurately judge how something will work until you get to use it. That goes for the creators, too. That’s why we iterate.

jrs235 7y ago

Posted to Hacker News a few days ago and I think relevent here: https://news.ycombinator.com/item?id=22365496 https://news.ycombinator.com/item?id=22365496