8 ms·

Sensible Software Engineering

164 points by myth_drannon 8y ago

painful 8y ago
I agree with the part that says Agile (and Scrum in particular) is drinking the koolaid. Managers who don't themselves write any code, but micromanage their workers with their version of Scrum, are the worst.
- gav 8y ago
  The majority of issues I see with Agile and Scrum is that it's a process added by people external to the teams and it's treated as a rigid process to be followed. I've worked with teams who have had to suffer through a 45-minute standup every morning that was immensely painful. They were just following the process as best as they understood it from a couple of days with an "agile coach" and didn't really understand what being agile really meant. > Managers who don't themselves write any code, but micromanage their workers with their version of Scrum, are the worst. I don't think anyone would disagree on the evils of micromanaging, but as a manager that doesn't code, I think that managers who do code are depriving their teams of the most valuable thing they have to offer--their support. If you take on a coding task and put yourself on the critical path of that sprint's work, you have to commit to putting the hours in to get it done. This is not a good use of your time. This is the difference between a manager and a Tech Lead.
  - Aeolun 8y ago
    Just the fact that a manager doesn’t code doesn’t mean they’re providing their teams with any support.
- jasonm23 8y ago
  Effectively Scrum is a poor fit because it's not based on engineering practices first. XP is better simply because it's based on engineering practices first. The trouble with any approach is that it's a template and some teams / people rest on those templates in place of actual thought.
jasode 8y ago
>Bugs are correlated with lines of code and TDD forces writing more code so how can it reduce bug counts? If the test code has no bugs then just write the rest of the code in the same style I'm not advocating for TDD (the programmer methodology in the IDE) but the author's explanation about "test code" isn't correct. Code written for explicit purposes of a test to exercise other code has been shown to increase correctness. E.g. SQLite database has 711x more test code than the core engine code.[1] (I made a previous comment why this is possible: https://news.ycombinator.com/item?id=15593121 https://news.ycombinator.com/item?id=15593121) Low-level infrastructure code like database engines, string manipulation libraries, crypto libraries, math libraries, network protocol routines, etc can benefit from a suite of regression tests. It's the high-level stuff like GUIs in webpages being tested with Selenium or LoadRunner that has conflicting business value because altering one pixel can have a cascading effect of breaking a bunch of fragile UI test scripts. [1] https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html
- herge 8y ago
  Maybe the spry takeaway is that you should write tests for things you want to make sure they will not break. Your UI being off by a pixel won't break your application, so if a test hangs on that, then it is not a good test. However, your business logic, or network protocol routine, those should not break even if you heavily refactor or add new features (especially business logic where a broken behaviour might seem correct), so those need to be heavily tested. If it is hard to test the juicy parts like business logic without also dragging in the UI, different OS/platform/db parts, etc, then you should look at how your application is structured and if it is really optimized for writing good tests.
  - jasode 8y ago
    >Your UI being off by a pixel won't break your application, so if a test hangs on that, then it is not a good test. For brevity in the previous comment, I didn't fully flesh out the background on why fragile UI tests get created. It happens accidentally. What sometimes happens is the the UI tester uses a "macro recorder" to record mouse movements and clicks. But then, a programmer shifts the position of a zipcode field by one pixel which throws the script off because it expected UI elements in a different spot. Fixing the broken UI tests is time consuming and can leave a bad impression that tests create a lot of effort for very little payback. The return-on-investment of UI tests depends on the business circumstances. I'm guessing Boeing and Airbus have automated UI tests that sometimes break when programmers change things which causes rework. However, the pain of fixing the UI tests and keeping it sync'd with the UI code is worth it for avionics software.
leetrout 8y ago
Really buried the lede here. I get that it was acknowledged as a rant / opinion piece but there's not a lot of really actionable advice for the general population of programmers, IMO. The article has good points, for sure, but the ending has the best part, IMO. > So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. Logic, statistics, and game theory are probably good bets. As things stand that kind of skill is probably the only thing that is going to survive the coming automation apocalypse because so far no one has figured out a way around rigorous and heuristic thinking. I think there's a lot of support there. I don't think using Kubernetes as an example of "sequestering the complexity behind a distributed control system" was a good follow up to TDD generating more lines of code. Containers are a step in the right direction and Kubernetes _is not_ the best option for using containers in production but it _is_ the most popular option and so if you want community mindshare and support then it probably is the best choice if you can manage it or use a managed service. "Serverless" is real, it's here, and containers / k8s are just a step along the way.
- eikenberry 8y ago
  I agree that Kubernetes is not the best technology out there, but I'm not 100% on which I would call the best. Did you have a winner in mind (it sounded like it)? If so why that one? Also it may not really matter as you said Kubernetes is the most popular and is only getting more so at this point and the network affect is so strong in tech like this that the "technically best" most likely will become a moot point.
  - leetrout 8y ago
    I am a big fan of Nomad's scheduler and the code is easy to pull in to an existing Go project. It doesn't have everything Kubernetes has but I don't think it needs it, either. Kubernetes has for sure won the popularity contest but the overhead involved in running it The Right Way™ on your own is a lot. Given what I've seen I would advocate for OpenShift if you like RedHat products / projects or sticking with Kubernetes from one of the well-known cloud providers.
deanalevitt 8y ago
This was really quite lovely to read. Like the others, I think the thoughts on TDD are somewhat fallacious but I certainly can't fault the conclusion.
scandox 8y ago
> The bottleneck has always been in understanding all the implicit assumptions baked into the system that for one reason or another are essential to its correctness. > It takes a particular kind of masochist to enjoy reverse engineering a black box from just poking at it with simple linear impulses. These are great observations and brilliantly put. In particular the second one I think rightly explains why some very smart people definitely do not take to programming as a profession.
ed_blackburn 8y ago
..and I came here thinking I would see an article about an amazing indie games studio from the 90s :-(
- jlarcombe 8y ago
  I thought exactly the same!
wildmanx 8y ago
> Bugs are correlated with lines of code and TDD forces writing more code so how can it reduce bug counts? If the test code has no bugs then just write the rest of the code in the same style Nobody has pointed out the fault in this reasoning yet, so I will. The linear relationship lines-of-code vs. total-bug-count is based on independence of bug introduction in different parts of the code. That is, introducing a bug in line 10000 is more-or-less independent of adding line 20000. For product code that is arguably the case, but adding test code into the mix, this basic assumption doesn't just not hold, but it's turned upside down. Test code is not customer-visible, but only dev-visible, with the sole purpose of finding bugs. Thus, adding test code to a code base decreases the average probability of a bug for lines of product code. More formally, if you have N lines of product code and thus c * N bugs for some fraction c, then adding M lines of test code does not increase the number of customer-visible bugs to c * (N + M). Instead, it reduces that number to c' * N with c'<c and the difference being caused by your test coverage. (A 100% test coverage, i.e., an exhaustive test without bugs or, equivalently, a formal verification, would bring c' to 0.) Sure, the M lines of test code may well have bugs on their own, but that only increases c' slightly while keeping it below c, and more importantly, those test bugs are not customer-visible. They only annoy developers. I agree with the rest of the post though.
- sombremesa 8y ago
  Tests don't exist for solving bugs as much as they exist to prevent regression. Writing exhaustive tests is not sufficient to bring c' to zero, because some of those bugs exist because of the developer's own presumptions about how code should behave.
  - chii 8y ago
    But testing exhaustively means the full range of inputs are used, and therefore, if a bug arises as inputs not considered by the dev initially, it would show up as a failure. What you're actually saying is that reaching full exhaustive testing is near impossible.
    - icebraining 8y ago
james_s_tayler 8y ago
>So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. Logic, statistics, and game theory are probably good bets. I think in an abstract sense control theory is a reasonably good bet. https://en.wikipedia.org/wiki/Control_theory https://en.wikipedia.org/wiki/Control_theory I can't say I know it deeply, but a lot of the ideas resonate when I think about software engineering. If you think of everything as basically an n-dimensional vehicle with an interface to control it and the control mechanism is used to set all the parameters relevant to the system then a few things follow: every system has a safe operating envelope parameters are usually linked to eachother such that turning one up turns another down I find there is a lot of mileage to be had about thinking about which parameters are linked to eachother and when you get excited about turning one of them up (ie you bring in TDD or a Kubernetes type solution) what is the effect you are having on the other parameters? That's where a new source of pain is going to come from. The biggest mistake I see in reasoning about these kind of things is being blind to the negative side of the trade-off due to the overwhelming excitement of finally being able to jump on the bandwagon and join the cargo cult. You have to train yourself to hunt for the parameter that is being affected indirectly as that's the most important side of the trade-off. You need to reason about whether or not the indirectly affected parameter's new value would take you outside of the safe operating envelope. With every decision we make about systems we build and run we are essentially trying to steer them, albeit clumsily, in this manner.
- keithnz 8y ago
  good recommendation, if the author understood control theory he'd understand that code that feedbacks on code is quite different than code with no feedback loop. while I find it hard to find anything I'd recommend about the authors article as most of the reasoning seems a little off, I can understand the sentiment of the article . Agile and TDD are really recognition of control theories ideas of feedback loops keep things in better control and can adapt faster vs long feedback loops going out of control far easier. This is more targeted at the human side of creating software. Nothing to say there aren't better strategies than TDD and Agile techniques, however I think that principle of feedback loops to give confidence will stay in some form. I think there is a LOT more to be said about engineering / designing correct, robust, and secure software.
myth2018 8y ago
The only part I tend to disagree with is regarding TDD. There is a study from Microsoft Research showing that TDD results in greater quality -- although you pay a price for it [0]. I believe this applies as a general rule, although I recognize I'm not aware of further studies. However I agree with the article's general idea. In the aviation industry there already are languages abstracting computers' internals and allowing programmers to reason about safety-critical programs using more high level constructs. Due to its nature, I think there won't be such a technology for general purpose languages -- in order to be general enough, you can't have too much things abstracted away. Maybe we couldn't go much farther than what languages like Basic allows us. On the other hand, I wish we had such languages for more specific tasks like ERP-like software, business web applications and so on. It's worth noticing that many of the biggest ERP companies in the world have their proprietary domain specific languages. [0] https://www.microsoft.com/en-us/research/blog/exploding-software-engineering-myths/ https://www.microsoft.com/en-us/research/blog/exploding-soft...
mynegation 8y ago
I enjoyed reading it and agree on many points. One thing that does not seem right is the claim that introduction of formal specification will increase the number of programmers. Yes, it takes a special kind of a person to be interested in fiddling with the code, but I believe the threshold for enjoying creation of formal specifications is even higher.
AdieuToLogic 8y ago
From the article: So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. No. If you want to be "a good programmer", then learn how to define the problem for which you are tasked to solve. The technology is irrelevant. The "language agnostic formalism" is irrelevant. Unless a person/team knows what must be done, then the rest really doesn't matter. Techniques which help to elicit repeatable delivery certainly are worthy to learn, even to advocate for. But without understanding what is needed, what use are they?
nishparadox 8y ago
"Whenever I’ve approached a new system the bottleneck has never been writing more code. The bottleneck has always been in understanding all the implicit assumptions baked into the system that for one reason or another are essential to its correctness". Well.... I know the feels... Assumptions without "understanding" is like the sword of damocles. As you sway into the program itself, there's always this notion that "something" is not right. Or maybe our assumption is not good enough. Most of the time, unplanned assumptions make it harder to integrate "more" code in the long run.
sinuhe69 8y ago
Basically, the author suggested a “back to basic” approach, which is of course not totally wrong. But ignore the very real challenge of the software industry is not going to help either. We can not sit down to plan and wait for the so-called “global optimal” design and then and only then proceed to implement. Because to achieve a “global optimal” design one has to consider each and every details and aspects of the problem, which is simply impossible in the software industry because so many are unknown. The Waterfall model is long dead. History has proven it does more harm than good. Don’t dig it up. PS: I don’t disagree that in some cases, academics can use the formal methods to find a (near) global optimal solution. But I don’t think it’s practical in a daily context, nor necessary. Our evolution is the best proof that local optima can lead over time to fantastic solutions.

164 points by myth_drannon 8y ago

I agree with the part that says Agile (and Scrum in particular) is drinking the koolaid. Managers who don't themselves write any code, but micromanage their workers with their version of Scrum, are the worst.

gav 8y ago
The majority of issues I see with Agile and Scrum is that it's a process added by people external to the teams and it's treated as a rigid process to be followed. I've worked with teams who have had to suffer through a 45-minute standup every morning that was immensely painful. They were just following the process as best as they understood it from a couple of days with an "agile coach" and didn't really understand what being agile really meant. > Managers who don't themselves write any code, but micromanage their workers with their version of Scrum, are the worst. I don't think anyone would disagree on the evils of micromanaging, but as a manager that doesn't code, I think that managers who do code are depriving their teams of the most valuable thing they have to offer--their support. If you take on a coding task and put yourself on the critical path of that sprint's work, you have to commit to putting the hours in to get it done. This is not a good use of your time. This is the difference between a manager and a Tech Lead.
- Aeolun 8y ago
  Just the fact that a manager doesn’t code doesn’t mean they’re providing their teams with any support.
jasonm23 8y ago
Effectively Scrum is a poor fit because it's not based on engineering practices first. XP is better simply because it's based on engineering practices first. The trouble with any approach is that it's a template and some teams / people rest on those templates in place of actual thought.

jasode 8y ago

>Bugs are correlated with lines of code and TDD forces writing more code so how can it reduce bug counts? If the test code has no bugs then just write the rest of the code in the same style I'm not advocating for TDD (the programmer methodology in the IDE) but the author's explanation about "test code" isn't correct. Code written for explicit purposes of a test to exercise other code has been shown to increase correctness. E.g. SQLite database has 711x more test code than the core engine code.[1] (I made a previous comment why this is possible: https://news.ycombinator.com/item?id=15593121 https://news.ycombinator.com/item?id=15593121) Low-level infrastructure code like database engines, string manipulation libraries, crypto libraries, math libraries, network protocol routines, etc can benefit from a suite of regression tests. It's the high-level stuff like GUIs in webpages being tested with Selenium or LoadRunner that has conflicting business value because altering one pixel can have a cascading effect of breaking a bunch of fragile UI test scripts. [1] https://www.sqlite.org/testing.html https://www.sqlite.org/testing.html

herge 8y ago
Maybe the spry takeaway is that you should write tests for things you want to make sure they will not break. Your UI being off by a pixel won't break your application, so if a test hangs on that, then it is not a good test. However, your business logic, or network protocol routine, those should not break even if you heavily refactor or add new features (especially business logic where a broken behaviour might seem correct), so those need to be heavily tested. If it is hard to test the juicy parts like business logic without also dragging in the UI, different OS/platform/db parts, etc, then you should look at how your application is structured and if it is really optimized for writing good tests.
- jasode 8y ago
  >Your UI being off by a pixel won't break your application, so if a test hangs on that, then it is not a good test. For brevity in the previous comment, I didn't fully flesh out the background on why fragile UI tests get created. It happens accidentally. What sometimes happens is the the UI tester uses a "macro recorder" to record mouse movements and clicks. But then, a programmer shifts the position of a zipcode field by one pixel which throws the script off because it expected UI elements in a different spot. Fixing the broken UI tests is time consuming and can leave a bad impression that tests create a lot of effort for very little payback. The return-on-investment of UI tests depends on the business circumstances. I'm guessing Boeing and Airbus have automated UI tests that sometimes break when programmers change things which causes rework. However, the pain of fixing the UI tests and keeping it sync'd with the UI code is worth it for avionics software.

leetrout 8y ago

Really buried the lede here. I get that it was acknowledged as a rant / opinion piece but there's not a lot of really actionable advice for the general population of programmers, IMO. The article has good points, for sure, but the ending has the best part, IMO. > So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. Logic, statistics, and game theory are probably good bets. As things stand that kind of skill is probably the only thing that is going to survive the coming automation apocalypse because so far no one has figured out a way around rigorous and heuristic thinking. I think there's a lot of support there. I don't think using Kubernetes as an example of "sequestering the complexity behind a distributed control system" was a good follow up to TDD generating more lines of code. Containers are a step in the right direction and Kubernetes _is not_ the best option for using containers in production but it _is_ the most popular option and so if you want community mindshare and support then it probably is the best choice if you can manage it or use a managed service. "Serverless" is real, it's here, and containers / k8s are just a step along the way.

eikenberry 8y ago
I agree that Kubernetes is not the best technology out there, but I'm not 100% on which I would call the best. Did you have a winner in mind (it sounded like it)? If so why that one? Also it may not really matter as you said Kubernetes is the most popular and is only getting more so at this point and the network affect is so strong in tech like this that the "technically best" most likely will become a moot point.
- leetrout 8y ago
  I am a big fan of Nomad's scheduler and the code is easy to pull in to an existing Go project. It doesn't have everything Kubernetes has but I don't think it needs it, either. Kubernetes has for sure won the popularity contest but the overhead involved in running it The Right Way™ on your own is a lot. Given what I've seen I would advocate for OpenShift if you like RedHat products / projects or sticking with Kubernetes from one of the well-known cloud providers.

deanalevitt 8y ago

This was really quite lovely to read. Like the others, I think the thoughts on TDD are somewhat fallacious but I certainly can't fault the conclusion.

scandox 8y ago

> The bottleneck has always been in understanding all the implicit assumptions baked into the system that for one reason or another are essential to its correctness. > It takes a particular kind of masochist to enjoy reverse engineering a black box from just poking at it with simple linear impulses. These are great observations and brilliantly put. In particular the second one I think rightly explains why some very smart people definitely do not take to programming as a profession.

ed_blackburn 8y ago

..and I came here thinking I would see an article about an amazing indie games studio from the 90s :-(

jlarcombe 8y ago
I thought exactly the same!

wildmanx 8y ago

> Bugs are correlated with lines of code and TDD forces writing more code so how can it reduce bug counts? If the test code has no bugs then just write the rest of the code in the same style Nobody has pointed out the fault in this reasoning yet, so I will. The linear relationship lines-of-code vs. total-bug-count is based on independence of bug introduction in different parts of the code. That is, introducing a bug in line 10000 is more-or-less independent of adding line 20000. For product code that is arguably the case, but adding test code into the mix, this basic assumption doesn't just not hold, but it's turned upside down. Test code is not customer-visible, but only dev-visible, with the sole purpose of finding bugs. Thus, adding test code to a code base decreases the average probability of a bug for lines of product code. More formally, if you have N lines of product code and thus c * N bugs for some fraction c, then adding M lines of test code does not increase the number of customer-visible bugs to c * (N + M). Instead, it reduces that number to c' * N with c'<c and the difference being caused by your test coverage. (A 100% test coverage, i.e., an exhaustive test without bugs or, equivalently, a formal verification, would bring c' to 0.) Sure, the M lines of test code may well have bugs on their own, but that only increases c' slightly while keeping it below c, and more importantly, those test bugs are not customer-visible. They only annoy developers. I agree with the rest of the post though.

sombremesa 8y ago
Tests don't exist for solving bugs as much as they exist to prevent regression. Writing exhaustive tests is not sufficient to bring c' to zero, because some of those bugs exist because of the developer's own presumptions about how code should behave.
- chii 8y ago
  But testing exhaustively means the full range of inputs are used, and therefore, if a bug arises as inputs not considered by the dev initially, it would show up as a failure. What you're actually saying is that reaching full exhaustive testing is near impossible.
  - icebraining 8y ago

james_s_tayler 8y ago

>So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. Logic, statistics, and game theory are probably good bets. I think in an abstract sense control theory is a reasonably good bet. https://en.wikipedia.org/wiki/Control_theory https://en.wikipedia.org/wiki/Control_theory I can't say I know it deeply, but a lot of the ideas resonate when I think about software engineering. If you think of everything as basically an n-dimensional vehicle with an interface to control it and the control mechanism is used to set all the parameters relevant to the system then a few things follow: every system has a safe operating envelope parameters are usually linked to eachother such that turning one up turns another down I find there is a lot of mileage to be had about thinking about which parameters are linked to eachother and when you get excited about turning one of them up (ie you bring in TDD or a Kubernetes type solution) what is the effect you are having on the other parameters? That's where a new source of pain is going to come from. The biggest mistake I see in reasoning about these kind of things is being blind to the negative side of the trade-off due to the overwhelming excitement of finally being able to jump on the bandwagon and join the cargo cult. You have to train yourself to hunt for the parameter that is being affected indirectly as that's the most important side of the trade-off. You need to reason about whether or not the indirectly affected parameter's new value would take you outside of the safe operating envelope. With every decision we make about systems we build and run we are essentially trying to steer them, albeit clumsily, in this manner.

keithnz 8y ago
good recommendation, if the author understood control theory he'd understand that code that feedbacks on code is quite different than code with no feedback loop. while I find it hard to find anything I'd recommend about the authors article as most of the reasoning seems a little off, I can understand the sentiment of the article . Agile and TDD are really recognition of control theories ideas of feedback loops keep things in better control and can adapt faster vs long feedback loops going out of control far easier. This is more targeted at the human side of creating software. Nothing to say there aren't better strategies than TDD and Agile techniques, however I think that principle of feedback loops to give confidence will stay in some form. I think there is a LOT more to be said about engineering / designing correct, robust, and secure software.

myth2018 8y ago

The only part I tend to disagree with is regarding TDD. There is a study from Microsoft Research showing that TDD results in greater quality -- although you pay a price for it [0]. I believe this applies as a general rule, although I recognize I'm not aware of further studies. However I agree with the article's general idea. In the aviation industry there already are languages abstracting computers' internals and allowing programmers to reason about safety-critical programs using more high level constructs. Due to its nature, I think there won't be such a technology for general purpose languages -- in order to be general enough, you can't have too much things abstracted away. Maybe we couldn't go much farther than what languages like Basic allows us. On the other hand, I wish we had such languages for more specific tasks like ERP-like software, business web applications and so on. It's worth noticing that many of the biggest ERP companies in the world have their proprietary domain specific languages. [0] https://www.microsoft.com/en-us/research/blog/exploding-software-engineering-myths/ https://www.microsoft.com/en-us/research/blog/exploding-soft...

mynegation 8y ago

I enjoyed reading it and agree on many points. One thing that does not seem right is the claim that introduction of formal specification will increase the number of programmers. Yes, it takes a special kind of a person to be interested in fiddling with the code, but I believe the threshold for enjoying creation of formal specifications is even higher.

AdieuToLogic 8y ago

From the article: So here’s the punchline: if you want to be a good programmer then learn a technology and language agnostic formalism. No. If you want to be "a good programmer", then learn how to define the problem for which you are tasked to solve. The technology is irrelevant. The "language agnostic formalism" is irrelevant. Unless a person/team knows what must be done, then the rest really doesn't matter. Techniques which help to elicit repeatable delivery certainly are worthy to learn, even to advocate for. But without understanding what is needed, what use are they?

nishparadox 8y ago

"Whenever I’ve approached a new system the bottleneck has never been writing more code. The bottleneck has always been in understanding all the implicit assumptions baked into the system that for one reason or another are essential to its correctness". Well.... I know the feels... Assumptions without "understanding" is like the sword of damocles. As you sway into the program itself, there's always this notion that "something" is not right. Or maybe our assumption is not good enough. Most of the time, unplanned assumptions make it harder to integrate "more" code in the long run.

sinuhe69 8y ago

Basically, the author suggested a “back to basic” approach, which is of course not totally wrong. But ignore the very real challenge of the software industry is not going to help either. We can not sit down to plan and wait for the so-called “global optimal” design and then and only then proceed to implement. Because to achieve a “global optimal” design one has to consider each and every details and aspects of the problem, which is simply impossible in the software industry because so many are unknown. The Waterfall model is long dead. History has proven it does more harm than good. Don’t dig it up. PS: I don’t disagree that in some cases, academics can use the formal methods to find a (near) global optimal solution. But I don’t think it’s practical in a daily context, nor necessary. Our evolution is the best proof that local optima can lead over time to fantastic solutions.