3 ms·

I think it's more appropriate to say, the quality of (enterprise?) software should not rely SOLELY on tests written by a software developer. Is it not common k

I think it's more appropriate to say, the quality of (enterprise?) software should not rely SOLELY on tests written by a software developer. Is it not common knowledge already that code review is the single most valuable tool in getting rid of bugs? I remember reading it on Code Complete, maybe The Pragmatic Programmer. Can't remember. I think there's still value in writing your own tests, at least with regard to refactoring (and sleeping better at night after deploying to production).

BlargMcLarg 6y ago
> Is it not common knowledge already that code review is the single most valuable tool in getting rid of bugs? No, it isn't. Because there isn't that much evidence it is. It is a good tool, but "most valuable tool"? That's a very bold claim. I would still place code analyzing tools, linters and fast feedback cycles (= fast compilation and ability to check changes) over code reviews, if only for the fact many code reviewers tend to divert into "I like this way better" arguments.
- b0afc375b5 6y ago
  Yeah definitely poor choice of words there. I misremembered the book. Anyway here's a cherry picked excerpt from Code Complete, 2nd edition: > Glenford Myers points out that human processes (inspections and walk-throughs, for instance) tend to be better than computer-based testing at finding certain kinds of errors and that the opposite is true for other kinds of errors (1979). > This result was confirmed in a later study, which found that code reading detected more interface defects and functional testing detected more control defects (Basili, Selby, and Hutchens 1986). Test guru Boris Beizer reports that informal test approaches typically achieve only 50–60 percent test coverage unless you're using a coverage analyzer (Johnson 1994). So I would conclude that the best approach is to use both code reviews/inspections and automated tests/linters/analyzers.
  - BlargMcLarg 6y ago
    I'm not debating automatic vs human approaches as much as how code reviews fit and why blind focus on them misses the forest for the trees in regards to other human approaches. When it takes 20 minutes from starting compilation to getting to the point of change, the Dev/QA/Review trifecta looks a lot different than when this takes 3 minutes. (Ideally,) developers are a lot less likely to make errors out of fatigue. They are more likely to pay attention to detail (when it takes 20 minutes, it's easy to go "whatever QA/reviewer will verify"). Reviewers start checking out the code and testing the change manually, rather than just looking at the code and pawning it off to QA. QA gets more time on their hands to proactively eliminate issues by being part of the design, instead of being the guys pointing out mistakes after the fact. Having shorter compilation cycles paves the way to increase the quality of all three segments of this cycle, yet it is often overlooked the most despite seeming so obvious. The part about QA is perhaps the most infuriating. Most business logic is so simple it can be presented as test/result matrices, which generally translate cleanly into requirements and automated tests, are easy to reason with, implicitly have three parties with different perspectives, and should be blind to technical details giving devs more freedom. Yet instead, this piece gets relegated to "software requirement engineers" writing walls of text, who tend to forget even the simplest cases (null, empty list/string, etc.), and any automated tests are then designed by the same developers making the implementation and only caught after the fact by manual QA a few months later.

BlargMcLarg 6y ago

> Is it not common knowledge already that code review is the single most valuable tool in getting rid of bugs? No, it isn't. Because there isn't that much evidence it is. It is a good tool, but "most valuable tool"? That's a very bold claim. I would still place code analyzing tools, linters and fast feedback cycles (= fast compilation and ability to check changes) over code reviews, if only for the fact many code reviewers tend to divert into "I like this way better" arguments.

b0afc375b5 6y ago
Yeah definitely poor choice of words there. I misremembered the book. Anyway here's a cherry picked excerpt from Code Complete, 2nd edition: > Glenford Myers points out that human processes (inspections and walk-throughs, for instance) tend to be better than computer-based testing at finding certain kinds of errors and that the opposite is true for other kinds of errors (1979). > This result was confirmed in a later study, which found that code reading detected more interface defects and functional testing detected more control defects (Basili, Selby, and Hutchens 1986). Test guru Boris Beizer reports that informal test approaches typically achieve only 50–60 percent test coverage unless you're using a coverage analyzer (Johnson 1994). So I would conclude that the best approach is to use both code reviews/inspections and automated tests/linters/analyzers.
- BlargMcLarg 6y ago
  I'm not debating automatic vs human approaches as much as how code reviews fit and why blind focus on them misses the forest for the trees in regards to other human approaches. When it takes 20 minutes from starting compilation to getting to the point of change, the Dev/QA/Review trifecta looks a lot different than when this takes 3 minutes. (Ideally,) developers are a lot less likely to make errors out of fatigue. They are more likely to pay attention to detail (when it takes 20 minutes, it's easy to go "whatever QA/reviewer will verify"). Reviewers start checking out the code and testing the change manually, rather than just looking at the code and pawning it off to QA. QA gets more time on their hands to proactively eliminate issues by being part of the design, instead of being the guys pointing out mistakes after the fact. Having shorter compilation cycles paves the way to increase the quality of all three segments of this cycle, yet it is often overlooked the most despite seeming so obvious. The part about QA is perhaps the most infuriating. Most business logic is so simple it can be presented as test/result matrices, which generally translate cleanly into requirements and automated tests, are easy to reason with, implicitly have three parties with different perspectives, and should be blind to technical details giving devs more freedom. Yet instead, this piece gets relegated to "software requirement engineers" writing walls of text, who tend to forget even the simplest cases (null, empty list/string, etc.), and any automated tests are then designed by the same developers making the implementation and only caught after the fact by manual QA a few months later.