5 ms·

This is fundamentally what I call the Tower Defense[1] model, borrowed from my old manager here at LinkedIn, Rino Jose[2]. The Tower Defense model is an approa

by eldude 12y ago

This is fundamentally what I call the Tower Defense[1] model, borrowed from my old manager here at LinkedIn, Rino Jose[2]. The Tower Defense model is an approach to software reliability that focuses on catching bugs at the lowest level, to avoid the inevitable combinatorial explosion in test coverage surface area. In other words, deal with problems like these at the language level, so there is NO NEED to deal with them at a higher process-level. No one is disputing that processes, QA or Devops couldn't/shouldn't hypothetically catch these bugs before entering production. The problem of course is that they usually don't, because the lower level defenses are allowing too many bugs through, that they really shouldn't and the higher level processes become overwhelmed, fail, and allow bugs to cross the critical production threshold. This means always giving higher priority to lower-level methods of reliability. For example, * Language rules are more important than * Unit tests are more important than * Integration tests are more important than * Code reviews are more important than * QA is more important than * Monitoring is more important than * Bug reports [1] http://en.wikipedia.org/wiki/Tower_defense http://en.wikipedia.org/wiki/Tower_defense [2] https://www.linkedin.com/in/rinoj https://www.linkedin.com/in/rinoj

wellpast 12y ago
This if you're testing spaghetti code or a monolith. In a system of low coupling and small highly cohesive components the cost of catching defects at the upper levels is not high enough to justify the cost of blindly using the tools below.
blueblob 12y ago
Can't you make the argument that even in C/C++ they could have enforced different "Language rules" like forcing no warnings? When compiling with gcc you could use -Wunreachable-code. Then it is part of the process. EDIT: with -Werror it will make this bug an error but not necessarily the class of bugs.
- eldude 12y ago
  Of course. I didn't want to muddie the issue, but I would consider compilation flags, transpilation (e.g., coffeescript) and linting (e.g., jshint, etc...) to reside between "Language rules" and "Unit tests."
- Peaker 12y ago
  gcc has not warned about unreachable code for years[1] now :( clang -Wunreachable-code does, though. [1]: http://gcc.gnu.org/ml/gcc-help/2011-05/msg00360.html http://gcc.gnu.org/ml/gcc-help/2011-05/msg00360.html
  - blueblob 12y ago
    Wow, that's interesting! Thanks for that. I just saw it in a manpage, didn't realize it doesn't currently work. That's what I get for R-ingTFM :-D
- gweinberg 12y ago
  an unreachable code warning would have caught this particular defect, but it wouldn't have helped if the duplicated line were something other than a goto. I think the policy of always using curly braces for conditionals (even if only one line) in c like languages is a good one.
  - AnthonyMouse 12y ago
    > an unreachable code warning would have caught this particular defect, but it wouldn't have helped if the duplicated line were something other than a goto. In that case always using curly braces or using a language that requires an "end" statement after the conditionally executed code may not have helped either. Imagine the incorrectly repeated statement was "a = a + 1" or "error_mask ^= error_x" etc. Putting the erroneous line inside the conditional doesn't erase the error, it just modifies the conditions under which it executes. That's about as likely to hang you as save you.
    - lukeschlather 12y ago
      Is there any language rule that can save you from incorrectly repeating an operation that is not idempotent?
jeffdavis 12y ago
Shouldn't code review be higher than unit tests?
- arjunnarayan 12y ago
  Ideally your unit tests are run on each build, so you hit the tests locally, before you push your changes in for a code review.
- eldude 12y ago
  Most definitely not. Human processes are always inferior to software enforcement. Code reviews are heavily dependent upon leadership, training, and quality control factors likes comprehensibility[1], mood, problem domain familiarity, etc... [1] Ask a programmer to review 10 lines of code, he'll find 10 issues. Ask him to do 500 lines and he'll say it looks good. -- https://twitter.com/girayozil/status/306836785739210752 https://twitter.com/girayozil/status/306836785739210752
  - Too 12y ago
    Unit tests are also made by human processes :) The advantage is if they were made good that time they will give you nice regression coverage for free in the future.
  - Iftheshoefits 12y ago
    Unit tests aren't intrinsic or automatic. They're extra pieces of code that have to be written, debugged and maintained by human beings. They ought to be considered as little more than extremely limited, narrowly-scoped sanity checks; not definitive pass/fail gates for the code they're exercising.
nostrademons 12y ago
I've found that the most reliable systems I've worked with have a defense-in-depth model, with reliability baked into the system architecture. That includes things like timeouts & retries for RPCs, replicas, graceful fallback, backup implementations, subsystem isolation, shut-off switches, etc. This is the Erlang/OTP model, and much of Google Search & cloud infrastructure is also built on similar principles.

by eldude 12y ago

wellpast 12y ago

This if you're testing spaghetti code or a monolith. In a system of low coupling and small highly cohesive components the cost of catching defects at the upper levels is not high enough to justify the cost of blindly using the tools below.

blueblob 12y ago

Can't you make the argument that even in C/C++ they could have enforced different "Language rules" like forcing no warnings? When compiling with gcc you could use -Wunreachable-code. Then it is part of the process. EDIT: with -Werror it will make this bug an error but not necessarily the class of bugs.

eldude 12y ago
Of course. I didn't want to muddie the issue, but I would consider compilation flags, transpilation (e.g., coffeescript) and linting (e.g., jshint, etc...) to reside between "Language rules" and "Unit tests."
Peaker 12y ago
gcc has not warned about unreachable code for years[1] now :( clang -Wunreachable-code does, though. [1]: http://gcc.gnu.org/ml/gcc-help/2011-05/msg00360.html http://gcc.gnu.org/ml/gcc-help/2011-05/msg00360.html
- blueblob 12y ago
  Wow, that's interesting! Thanks for that. I just saw it in a manpage, didn't realize it doesn't currently work. That's what I get for R-ingTFM :-D
gweinberg 12y ago
an unreachable code warning would have caught this particular defect, but it wouldn't have helped if the duplicated line were something other than a goto. I think the policy of always using curly braces for conditionals (even if only one line) in c like languages is a good one.
- AnthonyMouse 12y ago
  > an unreachable code warning would have caught this particular defect, but it wouldn't have helped if the duplicated line were something other than a goto. In that case always using curly braces or using a language that requires an "end" statement after the conditionally executed code may not have helped either. Imagine the incorrectly repeated statement was "a = a + 1" or "error_mask ^= error_x" etc. Putting the erroneous line inside the conditional doesn't erase the error, it just modifies the conditions under which it executes. That's about as likely to hang you as save you.
  - lukeschlather 12y ago
    Is there any language rule that can save you from incorrectly repeating an operation that is not idempotent?

jeffdavis 12y ago

Shouldn't code review be higher than unit tests?

arjunnarayan 12y ago
Ideally your unit tests are run on each build, so you hit the tests locally, before you push your changes in for a code review.
eldude 12y ago
Most definitely not. Human processes are always inferior to software enforcement. Code reviews are heavily dependent upon leadership, training, and quality control factors likes comprehensibility[1], mood, problem domain familiarity, etc... [1] Ask a programmer to review 10 lines of code, he'll find 10 issues. Ask him to do 500 lines and he'll say it looks good. -- https://twitter.com/girayozil/status/306836785739210752 https://twitter.com/girayozil/status/306836785739210752
- Too 12y ago
  Unit tests are also made by human processes :) The advantage is if they were made good that time they will give you nice regression coverage for free in the future.
- Iftheshoefits 12y ago
  Unit tests aren't intrinsic or automatic. They're extra pieces of code that have to be written, debugged and maintained by human beings. They ought to be considered as little more than extremely limited, narrowly-scoped sanity checks; not definitive pass/fail gates for the code they're exercising.

nostrademons 12y ago

I've found that the most reliable systems I've worked with have a defense-in-depth model, with reliability baked into the system architecture. That includes things like timeouts & retries for RPCs, replicas, graceful fallback, backup implementations, subsystem isolation, shut-off switches, etc. This is the Erlang/OTP model, and much of Google Search & cloud infrastructure is also built on similar principles.